**Computer Vision** Lecture 13 Roger S. Gaborski Roger S. Gaborski

**http://www.youtube.com/watch?v=NPcMS49V5hg** • SIFT • http://www.youtube.com/watch?v=U0wqePj4Mx0 • SIFT Object Recognition using VLfeat • http://www.youtube.com/watch?v=PU6BKIh1y4o • http://www.youtube.com/watch?v=0KLuBlGdFvw Roger S. Gaborski

**Harris Edge Detection:Not Invariant to Scaling** Corner All points will be classified as edges Roger S. Gaborski

**Blob Detection** Roger S. Gaborski

**Recall: Edge detectionDerivative of Gaussian** Edge f Derivativeof Gaussian Edge = maximumof derivative Roger S. Gaborski * Is the convolution operator, NOT correlation (otherwise polarity flipped) Source: S. Seitz

**Edge detection – Second Derivative of Gaussian** Edge f Second derivativeof Gaussian (Laplacian) Edge = zero crossingof second derivative Roger S. Gaborski Source: S. Seitz

**Scale Issue – Laplacianwide pulse** edge Note Value of Response – about -0.05 Roger S. Gaborski

**Scale Issue – Laplaciannarrower pulse** Roger S. Gaborski

**Scale Issue – Laplacian Pulse matches Laplacian – Max** Response Note Value of Response for Matched Pulse – about -0.10 Roger S. Gaborski

**Blob Detection** • The magnitude of Laplacian response is maximum when scale of Laplacian is matched to the blob • Convolve the blob with Laplacians at several different scales and look for the maximum • BUT – Laplacian response decays as scale increases see next slide Roger S. Gaborski

**Note Response Values** Increasing scale – sigma Decreasing Response Values Roger S. Gaborski

**Scale selection** We want to find the characteristic scale of the blob by convolving it with Laplacians at several scales and looking for the maximum response However, Laplacian response decays as scale increases: original signal(radius=8) increasing σ Roger S. Gaborski

**Scale normalization** The response of a derivative of Gaussian filter to a perfect step edge decreases as σ increases Roger S. Gaborski

**Scale normalization** The response of a derivative of Gaussian filter to a perfect step edge decreases as σ increases To keep response the same (scale-invariant), must multiply Gaussian derivative by σ Laplacian is the second Gaussian derivative, so it must be multiplied by σ2 Roger S. Gaborski

**Effect of scale normalization** Scale-normalized Laplacian response maximum Original signal Unnormalized Laplacian response Roger S. Gaborski

**Blob detection in 2D** Laplacian of Gaussian: Circularly symmetric operator for blob detection in 2D Roger S. Gaborski

**Blob detection in 2D** Laplacian of Gaussian: Circularly symmetric operator for blob detection in 2D Scale-normalized: Roger S. Gaborski

**Scale selection** At what scale does the Laplacian achieve a maximum response to a binary circle of radius r? r Roger S. Gaborski image Laplacian

**Scale selection** At what scale does the Laplacian achieve a maximum response to a binary circle of radius r? To get maximum response, the zeros of the Laplacian have to be aligned with the circle The Laplacian is given by (up to scale): The maximum response occurs at r circle Roger S. Gaborski image Laplacian

**Characteristic scale** We define the characteristic scale of a blob as the scale that produces peak of Laplacian response in the blob center characteristic scale T. Lindeberg (1998). "Feature detection with automatic scale selection."International Journal of Computer Vision30 (2): pp 77--116. Roger S. Gaborski

**Scale-space blob detector** Convolve image with scale-normalized Laplacian at several scales Find maxima of squared Laplacian response in scale-space Roger S. Gaborski

**Scale-space blob detector: Example** Roger S. Gaborski

**Scale-space blob detector: Example** Roger S. Gaborski

**Scale-space blob detector: Example** Roger S. Gaborski

**Approximating the Laplacian with a Difference of Gaussians:** Efficient implementation (Laplacian) (Difference of Gaussians) Roger S. Gaborski

**Scale Space** • Start with original image • Create progressively blurred images • Blur with Gaussian function • Progressive increase blur • One octave: doubling of blur • Resize original image to 50% • Repeat creating progressively blurred images • Repeat … Roger S. Gaborski

**DoG implementation** David G. Lowe. "Distinctive image features from scale-invariant keypoints.”IJCV 60 (2), pp. 91-110, 2004. Roger S. Gaborski

**Gaussian Blurred Images and Difference of Gaussian Images** Grouped by octave From Striker Roger S. Gaborski

**Scale Invariant Feature Transform** • Feature Detector and Feature Descriptor • Detection stages for SIFT features: • Scale-space extrema detection • Keypoint localization • Orientation assignment • Generation of keypoint descriptors. Following slides based on Strider, Univ. Toronto and Lowe Roger S. Gaborski

**Keypoints are maxima of Difference of Gaussians (DoG) that** occur at multiple scales • Start with Gaussian blurred image: L(x; y; σ ) = G(x; y; σ )* I(x; y) Where: G(x; y; σ ) = 1/(2 pi*σ2) (exp-(x2 +y2 )/σ2 ) Roger S. Gaborski

**Difference of Gaussian Filter:** G(x; y; kσ ) – G(x; y; σ) • Convolve blurred image with Difference of Gaussian: D(x; y; σ ) = L(x; y; k σ) - L(x; y; σ) Which is simply the difference of Gaussian blurred images at scales σ and k σ Roger S. Gaborski

**Blurred Images at Different Scales Computation of DoG** REF: Lowe , 2004 Roger S. Gaborski

**Scale Space Extrema Detection** • Group convolved images by octave Octave doubling of σ • Keypoints are idenitied as local maxima or minima of the DoG images across scales. • Each pixel in the DoG images is compared to its 8 neighbors at the same scale, plus the 9 corresponding neighbors at neighboring scales. • If the pixel is a local maximum or minimum, it is selected as a candidate keypoint Roger S. Gaborski

**Local extrema detection – pixel marked with X compared to** 26 neighbors in 3x3x3 space of adjacent DoGimages X is a key point if X is either greater (or less than) than all of the neighbors Roger S. Gaborski

**Keypoint Selection** • Keypoints with low contrast are deleted • Keypoints on edges are deleted • Each remaining keypoint is assigned an orientation Roger S. Gaborski

**Keypoint Orientation** • For each remaining keypoint • Compute gradient orientation histogram in the neighborhood of keypoint • Contribution of each neighboring pixel is weighted by the gradient magnitude and a Gaussian window with a σ that is 1.5 the scale of the keypoint • Peaks in histogram correspond to dominate orientations • Separate keypoint for direction corresponding to maximum in histogram and any direction within 80% of the maximum Roger S. Gaborski

**16x16 Pixels Around Keypoint** Divide 16x16 pixels into 16 , 4x4 sub windows Calculate gradient magnitude and direction in each sub window Store gradient and magnitude in 8 bin histograms Roger S. Gaborski

**Gradients with an angle between 0 and 44 degrees goes in the** first bin, etc. The magnitude depends on the gradients magnitude and the gradient’s distance from the key point Roger S. Gaborski

**Key Point Vector** • Each 4x4 sub window has 8 numbers associate with the sub window (the histogram) • 16 sub windows • 16 * 8 = 128 numbers • Normalize the vector (divide vector by the sqrt of the sum of the elements squared) • The resulting normalized vector is the feature vector associated with each key point Roger S. Gaborski

**SIFT Descriptor** • Histogram contains 8 bins • Each descriptor has 16 histograms around keypoint • 4 x 4 x 8 = 128 long feature vector for each keypoint • Normalize feature for illumination invariance Roger S. Gaborski

**Recognition using SIFT** • Compute SIFT features for input image • Match features in SIFT database Roger S. Gaborski

**Gaussian Blurred Images and Difference of Gaussian Images** Grouped by octave From Striker Roger S. Gaborski

**Maxima of DoG across scales Keypoints after** removing low contrast points Roger S. Gaborski

**Removal of Edge Points** Roger S. Gaborski

**Arrows indicate Scale and Orientation** Roger S. Gaborski

**VLFeat.org** • MSER • SURF (Speeded Up Robust Features) Roger S. Gaborski

**Image Features** http://videolectures.net/mcvc08_gousseau_cmlf/ • The Future of Image Search author: Jitendra Malik, UC Berkeley, University of California, http://videolectures.net/kdd08_malik_fis/ Roger S. Gaborski

**SIFT Feature Representation** • Keypoint Feature Descriptor • Set of orientation histograms on 4x4 neighborhoods (Lowe 2004) 8x8 set of samples 2x2 Descriptor Array computer from 8x8 set of samples Roger S. Gaborski