While the Sobel operator gives us a good starting point by highlighting areas of high intensity change, the resulting "edges" can often be thick, noisy, and disconnected. For many applications, we need a more refined result: clean, thin, continuous lines that accurately represent object boundaries. This is where the Canny edge detector comes in. Developed by John F. Canny in 1986, it's a multi-stage algorithm designed specifically to produce superior edge maps and is arguably the most widely used edge detection algorithm today.

The Canny algorithm aims to satisfy three main criteria for good edge detection:

Good Detection: Find as many real edges as possible while minimizing false positives (detecting edges where none exist).
Good Localization: The detected edge pixels should be as close as possible to the center of the true edge.
Minimal Response: A single edge should only be detected once; minimize multiple responses to the same edge.

To achieve these goals, the Canny algorithm follows a sequence of well-defined steps:

1. Noise Reduction using Gaussian Smoothing

Real-world images almost always contain some level of noise (random variations in pixel intensity). Edge detection algorithms, especially those based on gradients (intensity changes), are very sensitive to noise. A small noisy fluctuation could be misinterpreted as an edge.

Therefore, the first step in the Canny algorithm is to smooth the image slightly to reduce noise. This is typically done using a Gaussian filter, just like the smoothing filters we discussed in the previous chapter. Applying a Gaussian blur helps suppress minor intensity variations that aren't part of significant edges. The amount of smoothing (controlled by the size and standard deviation, sigma, of the Gaussian kernel) is a parameter that can affect the final result; too much smoothing might blur out weaker edges, while too little might leave too much noise.

2. Finding Intensity Gradients

Once the image is smoothed, the next step is to find the intensity gradients, similar to how the Sobel operator works. This involves calculating the first derivative of the image intensity in both the horizontal ( $G_x$ ) and vertical ( $G_y$ ) directions. From these, we can compute the gradient magnitude (edge strength) and direction for each pixel:

Gradient Magnitude: $G = \sqrt{G_x^2 + G_y^2}$ (Indicates the strength of the edge)
Gradient Direction: $\theta = \operatorname{atan2}(G_y, G_x)$ (Indicates the orientation of the edge, perpendicular to the gradient direction)

The gradient magnitude gives us an image where brighter pixels indicate stronger potential edges. However, these edges are still usually thick. The gradient direction is important for the next step.

3. Non-Maximum Suppression

This crucial step aims to thin the thick edges found in the gradient magnitude image down to single-pixel width lines. The idea is simple: for any given pixel, we check if its gradient magnitude is the largest compared to its neighbors along the gradient direction.

Imagine an edge boundary. The gradient magnitude will be highest at the center of the edge and will fall off on either side. We want to keep only the pixels at the very peak of this gradient "ridge" and suppress the others (set their value to zero).

To do this, the algorithm looks at each pixel's gradient direction ( $\theta$ ). The direction is rounded to one of four main orientations (e.g., horizontal, vertical, or one of the two diagonals). Then, the pixel's gradient magnitude ( $G$ ) is compared with the gradient magnitudes of the two neighboring pixels along that direction.

If the current pixel's magnitude is greater than both neighbors along the gradient direction, it's considered a local maximum, and its value is preserved.
If the current pixel's magnitude is not the largest among its neighbors in that direction, it is suppressed (set to 0).

This process effectively thins the edges, leaving mostly single-pixel-wide lines representing the locations with the sharpest intensity change.

Flow of Non-Maximum Suppression for a pixel P and its neighbors N1, N2 along the gradient direction.

4. Double Thresholding

The non-maximum suppression step gives us thin edges, but some might be caused by noise or minor color variations that we don't consider "real" edges. Double thresholding helps filter these out based on their gradient magnitude (strength).

Instead of using a single threshold, Canny uses two:

A high threshold ( $T_{high}$ )
A low threshold ( $T_{low}$ )

These thresholds are applied to the image resulting from non-maximum suppression:

Any pixel with a gradient magnitude greater than $T_{high}$ is immediately marked as a strong edge pixel. These are very likely to be part of a genuine edge.
Any pixel with a gradient magnitude less than $T_{low}$ is immediately discarded (marked as not an edge). These are considered too weak to be significant.
Any pixel with a gradient magnitude between $T_{low}$ and $T_{high}$ is marked as a weak edge pixel. These are candidates; their fate depends on the next step.

Choosing appropriate threshold values is important. If $T_{high}$ is too high, strong edges might be broken into segments. If $T_{low}$ is too low, noise might start appearing in the final edge map. Typically, $T_{high}$ is set to be 2 or 3 times $T_{low}$ .

5. Edge Tracking by Hysteresis

This final step determines which of the weak edge pixels (those between $T_{low}$ and $T_{high}$ ) should be kept. The logic is based on connectivity: a weak edge pixel is kept only if it is connected to a strong edge pixel.

The algorithm scans the image and performs the following check for each weak pixel:

Examine the 8 neighboring pixels around the weak pixel.
If any of these neighbors is a strong edge pixel (magnitude > $T_{high}$ ), then the weak pixel is "promoted" and marked as a strong edge pixel itself.
This process continues recursively: newly promoted weak pixels can, in turn, cause their connected weak neighbors to be promoted.

Essentially, we start from the definite "strong" edge pixels and follow connections through the "weak" edge pixels. Any weak pixel that cannot be reached from a strong edge pixel via a path of other weak pixels is discarded (set to 0).

This hysteresis process helps achieve two goals:

It connects segments of edges that might have been broken if only a single high threshold was used (because some parts of the edge might fall slightly below $T_{high}$ ).
It helps eliminate isolated weak pixels caused by noise, as they are unlikely to be connected to any strong edge pixels.

The final output is a binary image where white pixels represent the detected edges, and black pixels represent the background. These edges are typically thin, well-localized, and more continuous compared to simpler methods.

The Canny edge detector provides a robust way to identify significant edges while suppressing noise and insignificant details. Understanding its steps helps in interpreting its results and tuning its parameters (Gaussian sigma, $T_{low}$ , $T_{high}$ ) for specific applications, which you will practice in the hands-on section.