Computer Vision

180 Views

Download Presentation
## Computer Vision

- - - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - - -

**Computer Vision**Lecture 13 Roger S. Gaborski Roger S. Gaborski**http://www.youtube.com/watch?v=NPcMS49V5hg**• SIFT • http://www.youtube.com/watch?v=U0wqePj4Mx0 • SIFT Object Recognition using VLfeat • http://www.youtube.com/watch?v=PU6BKIh1y4o • http://www.youtube.com/watch?v=0KLuBlGdFvw Roger S. Gaborski**Harris Edge Detection:Not Invariant to Scaling**Corner All points will be classified as edges Roger S. Gaborski**Blob Detection**Roger S. Gaborski**Recall: Edge detectionDerivative of Gaussian**Edge f Derivativeof Gaussian Edge = maximumof derivative Roger S. Gaborski * Is the convolution operator, NOT correlation (otherwise polarity flipped) Source: S. Seitz**Edge detection – Second Derivative of Gaussian**Edge f Second derivativeof Gaussian (Laplacian) Edge = zero crossingof second derivative Roger S. Gaborski Source: S. Seitz**Scale Issue – Laplacianwide pulse**edge Note Value of Response – about -0.05 Roger S. Gaborski**Scale Issue – Laplaciannarrower pulse**Roger S. Gaborski**Scale Issue – Laplacian Pulse matches Laplacian – Max**Response Note Value of Response for Matched Pulse – about -0.10 Roger S. Gaborski**Blob Detection**• The magnitude of Laplacian response is maximum when scale of Laplacian is matched to the blob • Convolve the blob with Laplacians at several different scales and look for the maximum • BUT – Laplacian response decays as scale increases see next slide Roger S. Gaborski**Note Response Values**Increasing scale – sigma Decreasing Response Values Roger S. Gaborski**Scale selection**We want to find the characteristic scale of the blob by convolving it with Laplacians at several scales and looking for the maximum response However, Laplacian response decays as scale increases: original signal(radius=8) increasing σ Roger S. Gaborski**Scale normalization**The response of a derivative of Gaussian filter to a perfect step edge decreases as σ increases Roger S. Gaborski**Scale normalization**The response of a derivative of Gaussian filter to a perfect step edge decreases as σ increases To keep response the same (scale-invariant), must multiply Gaussian derivative by σ Laplacian is the second Gaussian derivative, so it must be multiplied by σ2 Roger S. Gaborski**Effect of scale normalization**Scale-normalized Laplacian response maximum Original signal Unnormalized Laplacian response Roger S. Gaborski**Blob detection in 2D**Laplacian of Gaussian: Circularly symmetric operator for blob detection in 2D Roger S. Gaborski**Blob detection in 2D**Laplacian of Gaussian: Circularly symmetric operator for blob detection in 2D Scale-normalized: Roger S. Gaborski**Scale selection**At what scale does the Laplacian achieve a maximum response to a binary circle of radius r? r Roger S. Gaborski image Laplacian**Scale selection**At what scale does the Laplacian achieve a maximum response to a binary circle of radius r? To get maximum response, the zeros of the Laplacian have to be aligned with the circle The Laplacian is given by (up to scale): The maximum response occurs at r circle Roger S. Gaborski image Laplacian**Characteristic scale**We define the characteristic scale of a blob as the scale that produces peak of Laplacian response in the blob center characteristic scale T. Lindeberg (1998). "Feature detection with automatic scale selection."International Journal of Computer Vision30 (2): pp 77--116. Roger S. Gaborski**Scale-space blob detector**Convolve image with scale-normalized Laplacian at several scales Find maxima of squared Laplacian response in scale-space Roger S. Gaborski**Scale-space blob detector: Example**Roger S. Gaborski**Scale-space blob detector: Example**Roger S. Gaborski**Scale-space blob detector: Example**Roger S. Gaborski**Approximating the Laplacian with a Difference of Gaussians:**Efficient implementation (Laplacian) (Difference of Gaussians) Roger S. Gaborski**Scale Space**• Start with original image • Create progressively blurred images • Blur with Gaussian function • Progressive increase blur • One octave: doubling of blur • Resize original image to 50% • Repeat creating progressively blurred images • Repeat … Roger S. Gaborski**DoG implementation**David G. Lowe. "Distinctive image features from scale-invariant keypoints.”IJCV 60 (2), pp. 91-110, 2004. Roger S. Gaborski**Gaussian Blurred Images and Difference of Gaussian Images**Grouped by octave From Striker Roger S. Gaborski**Scale Invariant Feature Transform**• Feature Detector and Feature Descriptor • Detection stages for SIFT features: • Scale-space extrema detection • Keypoint localization • Orientation assignment • Generation of keypoint descriptors. Following slides based on Strider, Univ. Toronto and Lowe Roger S. Gaborski**Keypoints are maxima of Difference of Gaussians (DoG) that**occur at multiple scales • Start with Gaussian blurred image: L(x; y; σ ) = G(x; y; σ )* I(x; y) Where: G(x; y; σ ) = 1/(2 pi*σ2) (exp-(x2 +y2 )/σ2 ) Roger S. Gaborski**Difference of Gaussian Filter:**G(x; y; kσ ) – G(x; y; σ) • Convolve blurred image with Difference of Gaussian: D(x; y; σ ) = L(x; y; k σ) - L(x; y; σ) Which is simply the difference of Gaussian blurred images at scales σ and k σ Roger S. Gaborski**Blurred Images at Different Scales Computation of DoG**REF: Lowe , 2004 Roger S. Gaborski**Scale Space Extrema Detection**• Group convolved images by octave Octave doubling of σ • Keypoints are idenitied as local maxima or minima of the DoG images across scales. • Each pixel in the DoG images is compared to its 8 neighbors at the same scale, plus the 9 corresponding neighbors at neighboring scales. • If the pixel is a local maximum or minimum, it is selected as a candidate keypoint Roger S. Gaborski**Local extrema detection – pixel marked with X compared to**26 neighbors in 3x3x3 space of adjacent DoGimages X is a key point if X is either greater (or less than) than all of the neighbors Roger S. Gaborski**Keypoint Selection**• Keypoints with low contrast are deleted • Keypoints on edges are deleted • Each remaining keypoint is assigned an orientation Roger S. Gaborski**Keypoint Orientation**• For each remaining keypoint • Compute gradient orientation histogram in the neighborhood of keypoint • Contribution of each neighboring pixel is weighted by the gradient magnitude and a Gaussian window with a σ that is 1.5 the scale of the keypoint • Peaks in histogram correspond to dominate orientations • Separate keypoint for direction corresponding to maximum in histogram and any direction within 80% of the maximum Roger S. Gaborski**16x16 Pixels Around Keypoint**Divide 16x16 pixels into 16 , 4x4 sub windows Calculate gradient magnitude and direction in each sub window Store gradient and magnitude in 8 bin histograms Roger S. Gaborski**Gradients with an angle between 0 and 44 degrees goes in the**first bin, etc. The magnitude depends on the gradients magnitude and the gradient’s distance from the key point Roger S. Gaborski**Key Point Vector**• Each 4x4 sub window has 8 numbers associate with the sub window (the histogram) • 16 sub windows • 16 * 8 = 128 numbers • Normalize the vector (divide vector by the sqrt of the sum of the elements squared) • The resulting normalized vector is the feature vector associated with each key point Roger S. Gaborski**SIFT Descriptor**• Histogram contains 8 bins • Each descriptor has 16 histograms around keypoint • 4 x 4 x 8 = 128 long feature vector for each keypoint • Normalize feature for illumination invariance Roger S. Gaborski**Recognition using SIFT**• Compute SIFT features for input image • Match features in SIFT database Roger S. Gaborski**Gaussian Blurred Images and Difference of Gaussian Images**Grouped by octave From Striker Roger S. Gaborski**Maxima of DoG across scales Keypoints after**removing low contrast points Roger S. Gaborski**Removal of Edge Points**Roger S. Gaborski**Arrows indicate Scale and Orientation**Roger S. Gaborski**VLFeat.org**• MSER • SURF (Speeded Up Robust Features) Roger S. Gaborski**Image Features**http://videolectures.net/mcvc08_gousseau_cmlf/ • The Future of Image Search author: Jitendra Malik, UC Berkeley, University of California, http://videolectures.net/kdd08_malik_fis/ Roger S. Gaborski**SIFT Feature Representation**• Keypoint Feature Descriptor • Set of orientation histograms on 4x4 neighborhoods (Lowe 2004) 8x8 set of samples 2x2 Descriptor Array computer from 8x8 set of samples Roger S. Gaborski