Stanford CS223B Computer Vision, Winter 2006 Lecture 3 More On Features

1. Stanford CS223B Computer Vision, Winter 2006Lecture 3 More On Features Professor Sebastian Thrun CAs: Dan Maynes-Aminzade and Mitul Saha [with slides by D Forsyth, D. Lowe, M. Polleyfeys, C. Rasmussen, G. Loy, D. Jacobs, J. Rehg, A, Hanson, G. Bradski,…]

2. Today’s Goals • Canny Edge Detector • Harris Corner Detector • Hough Transform • Templates and Image Pyramid • SIFT Features

3. Features in Matlab im = imread('bridge.jpg'); bw = double(im(:,:,1)) ./ 256; edge(im,’sobel’) - (almost) linear edge(im,’canny’) - not local, no closed form

4. Sobel Operator Edge Magnitude = 2 2 S1 + S1 S1 tan-1 S2 Edge Direction = -1 -2 -1 0 0 0 1 2 1 -1 0 1 -2 0 2 -1 0 1 S1= S2 =

5. Sobel in Matlab edge(im,’sobel’)

6. Canny Edge Detector edge(im,’canny’)

7. Comparison Sobel Canny

8. Canny Edge Detection Steps: • Apply derivative of Gaussian • Non-maximum suppression • Thin multi-pixel wide “ridges” down to single pixel width • Linking and thresholding • Low, high edge-strength thresholds • Accept all edges over low threshold that are connected to edge over high threshold

9. Non-Maximum Supression • Non-maximum suppression: • Select the single maximum point across the width of an edge.

10. Linking to the Next Edge Point Assume the marked point q is an edge point. Take the normal to the gradient at that point and use this to predict continuation points (either r or p).

11. Edge Hysteresis • Hysteresis:A lag or momentum factor • Idea: Maintain two thresholds khigh and klow • Use khigh to find strong edges to start edge chain • Use klow to find weak edges which continue edge chain • Typical ratio of thresholds is roughly khigh/klow=2

12. Canny Edge Detection (Example) gap is gone Strong + connected weak edges Original image Strong edges only Weak edges courtesy of G. Loy

13. Canny Edge Detection (Example) Using Matlab with default thresholds

14. Bridge Example Again edge(im,’canny’)

15. Corner Effects

16. Today’s Goals • Canny Edge Detector • Harris Corner Detector • Hough Transform • Templates and Image Pyramid • SIFT Features

17. Finding Corners Edge detectors perform poorly at corners. Corners provide repeatable points for matching, so are worth detecting. • Idea: • Exactly at a corner, gradient is ill defined. • However, in the region around a corner, gradient has two or more different values.

18. The Harris corner detector Form the second-moment matrix: Gradient with respect to x, times gradient with respect to y Sum over a small region around the hypothetical corner Matrix is symmetric Slide credit: David Jacobs

19. Simple Case First, consider case where: This means dominant gradient directions align with x or y axis If either λ is close to 0, then this is not a corner, so look for locations where both are large. Slide credit: David Jacobs

20. General Case It can be shown that since C is rotationally symmetric: So every case is like a rotated version of the one on last slide. Slide credit: David Jacobs

21. So, To Detect Corners • Filter image with Gaussian to reduce noise • Compute magnitude of the x and y gradients at each pixel • Construct C in a window around each pixel (Harris uses a Gaussian window – just blur) • Solve for product of ls (determinant of C) • If ls are both big (product reaches local maximum and is above threshold), we have a corner (Harris also checks that ratio of ls is not too high)

22. Gradient Orientation Closeup

23. Corner Detection Corners are detected where the product of the ellipse axis lengths reaches a local maximum.

24. Harris Corners • Originally developed as features for motion tracking • Greatly reduces amount of computation compared to tracking every pixel • Translation and rotation invariant (but not scale invariant)

25. Harris Corner in Matlab % Harris Corner detector - by Kashif Shahzad sigma=2; thresh=0.1; sze=11; disp=0; % Derivative masks dy = [-1 0 1; -1 0 1; -1 0 1]; dx = dy'; %dx is the transpose matrix of dy % Ix and Iy are the horizontal and vertical edges of image Ix = conv2(bw, dx, 'same'); Iy = conv2(bw, dy, 'same'); % Calculating the gradient of the image Ix and Iy g = fspecial('gaussian',max(1,fix(6*sigma)), sigma); Ix2 = conv2(Ix.^2, g, 'same'); % Smoothed squared image derivatives Iy2 = conv2(Iy.^2, g, 'same'); Ixy = conv2(Ix.*Iy, g, 'same'); % My preferred measure according to research paper cornerness = (Ix2.*Iy2 - Ixy.^2)./(Ix2 + Iy2 + eps); % We should perform nonmaximal suppression and threshold mx = ordfilt2(cornerness,sze^2,ones(sze)); % Grey-scale dilate cornerness = (cornerness==mx)&(cornerness>thresh); % Find maxima [rws,cols] = find(cornerness); % Find row,col coords. clf ; imshow(bw); hold on; p=[cols rws]; plot(p(:,1),p(:,2),'or'); title('\bf Harris Corners')

26. Example (s=0.1)

27. Example (s=0.01)

28. Example (s=0.001)

29. Harris: OpenCV Implementation

30. Harris Corners for Optical Flow Will talk about this later (optical flow)

31. Today’s Goals • Canny Edge Detector • Harris Corner Detector • Hough Transform • Templates and Image Pyramid • SIFT Features

32. Features? Local versus global

33. Vanishing Points

34. Vanishing Points

35. Vanishing Points A. Canaletto [1740], Arrival of the French Ambassador in Venice

36. Vanishing Points…? A. Canaletto [1740], Arrival of the French Ambassador in Venice

37. From Edges to Lines

38. Hough Transform

39. Hough Transform: Quantization m m Detecting Lines by finding maxima / clustering in parameter space

40. Hough Transform: Algorithm • For each image point, determine • most likely line parameters b,m (direction of gradient) • strength (magnitude of gradient) • Increment parameter counter by strength value • Cluster in parameter space, pick local maxima

41. Hough Transform: Results Image Edge detection Hough Transform

42. Hough Transform?

43. Summary Hough Transform • Smart counting • Local evidence for global features • Organized in a table • Careful with parameterization! • Subject to Curse of Dimensionality • Works great for simple features with 3 unknowns • Will fail for complex objects (e.g., all faces)

44. Today’s Goals • Canny Edge Detector • Harris Corner Detector • Hough Transform • Templates and Image Pyramid • SIFT Features

45. Problem: Features for Recognition … in here Want to find

46. Templates • Find an object in an image! • Want Invariance! • Scaling • Rotation • Illumination • Deformation

47. Convolution with Templates % read image im = imread('bridge.jpg'); bw = double(im(:,:,1)) ./ 256;; imshow(bw) % apply FFT FFTim = fft2(bw); bw2 = real(ifft2(FFTim)); imshow(bw2) % define a kernel kernel=zeros(size(bw)); kernel(1, 1) = 1; kernel(1, 2) = -1; FFTkernel = fft2(kernel); % apply the kernel and check out the result FFTresult = FFTim .* FFTkernel; result = real(ifft2(FFTresult)); imshow(result) % select an image patch patch = bw(221:240,351:370); imshow(patch) patch = patch - (sum(sum(patch)) / size(patch,1) / size(patch, 2)); kernel=zeros(size(bw)); kernel(1:size(patch,1),1:size(patch,2)) = patch; FFTkernel = fft2(kernel); % apply the kernel and check out the result FFTresult = FFTim .* FFTkernel; result = max(0, real(ifft2(FFTresult))); result = result ./ max(max(result)); result = (result .^ 1 > 0.5); imshow(result) % alternative convolution imshow(conv2(bw, patch, 'same'))

48. Template Convolution

49. Template Convolution

50. Aside: Convolution Theorem Fourier Transform of g: Fis invertible