1 / 51

Advance Computer Vision

Advance Computer Vision. Lectures 02 Roger S. Gaborski. Features. Match corresponding feature points in images In video, two temporally adjacent frames In still images, two different images of the same scene Or two different images of similar scene. Issues. Lighting variations

mura
Download Presentation

Advance Computer Vision

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Advance Computer Vision Lectures 02 Roger S. Gaborski

  2. Features • Match corresponding feature points in images • In video, two temporally adjacent frames • In still images, two different images of the same scene • Or two different images of similar scene

  3. Issues • Lighting variations • View point • Scale • Rotation

  4. NEED TO FIND CORRESPONDENCE BETWEEN FEATURE POINTS IN TWO DIFFERENT IMAGES • Cannot match individual pixels • Need to use a window containing many pixels (5x5, 7x7, 21x21, etc)

  5. Matching on a continuum like texture and edges not very robust • Many edges (and parts of edges) will match • At the very least: • Need to find interest points • Extract patches around interest points • Match patches

  6. Feature Points / Correspondence • Points should be extracted consistently over different views • Points should be invariant to scaling, rotation, changes in illumination • Information in the neighborhood of the point should be unique so that it can be matched

  7. Select Window RegionMatch Region in Second Image Calculate difference between the two patches WindowMatching_ACV.m

  8. Convert to Grayscale

  9. Randomly Select Patch patch = I(80:110,200:230); For Demonstration Use Only Strip Containing Patch

  10. Error=Sum of Absolute Difference Between Patch and Strip

  11. This simple approach works because the patch was extracted from the image • We need a method that will work with ‘similar’ patches

  12. Normalized Cross Correlation (Refer to equation on page p313)Also MATLAB docs • w is template • w is average value of elements in template • f is the image • f is the average of the image where f and w overlap • Denominator normalizes resulting in an output range of -1, +1 • High value for absolute value of output is a good match

  13. MATLAB Cross Correlation Function g = normxcorr2(template, f)

  14. Find Max Value in |g| d = abs(g); [ypeak, xpeak] = find(d == max(d(:))); %Adjust location by size of template %since g is larger than original image %by size of template ypeak = ypeak-(size(patch,1)-1)/2; xpeak = xpeak-(size(patch,2)-1)/2; fprintf('\n Center of Patch: ypeak is %d and xpeak is %d \n\n\n', ypeak, xpeak); figure, imshow(Igray) hold on plot(xpeak, ypeak, 'ro')

  15. Red – strongest response Green – second strongest response (wheel rotation slightly different)

  16. HW#1- Due Tuesday 12/04: rsg.advcv@gmail.com • Read in Purple Flower Image and Select a 31x31 patch • Extract image strip that contains patch • Calculate Error=Absolute Difference Between Patch and Strip and plot • Repeat using Normalized Cross Correlation on whole image • Plot max location on image • CONTINUED NEXT SLIDE

  17. HW#1- Due Tuesday 12/04: rsg.advcv@gmail.com • Repeat Normalized Cross Correlation using several different templates – choose templates from uniform areas and compare with templates containing strong edges, try different size templates • Explain your results, what are the characteristics of ‘good’ templates • Email code, results and writeupto:rsg.advcv@gmail.com • You should work individually or in teams of 2- if team, list both names on report and code • WindowMatching_ACV_your_names.m

  18. Selection of Patches • We don’t want to just randomly select patches from an image • We want to select patches that have high information content • We want to do this automatically, not manually chosen

  19. But How Do We Select Points? • Junctions or Corners • Stable over changes in viewpoint

  20. Feature Points?

  21. Subtract First Window from Second Window Sightly Offset from First

  22. Moravec’s Corner Detector • Overview: • Select window size • Shift window over image region • If window over uniform region, shifts in all directions will result in small changes • If window over edge, shifts along edge will results in small changes, but shifts across edge will result in large changes • along edge – no change • Perpendicular to edge – large change • If window over corner, than shifts in all directions will result will result in large changes • Detect corner by finding regions that have large changes in all directions

  23. Subtract First Window from Second Window Window moved vertically, no change Window moved horizontally, no change Window moved in either direction, large change

  24. Corner Detection: Basic Idea We should easily recognize the point by looking through a small window Shifting a window in anydirection should give a large change in intensity “flat” region:no change in all directions “edge”:no change along the edge direction “corner”:significant change in all directions Roger S. Gaborski Source: A. Efros

  25. Wikipedia: Corner Detection-No shortage of corner detectors • Moravec • Harris • Laplacian of gaussians, Difference of Gaussian and Determinant of the Hessian • SIFT • SUSAN Corner Detector • Trajkovic – Hedley Corner Detector Roger S. Gaborski

  26. Corner Detection: Weighted Sum of Differences Window function Shifted intensity Intensity Window function w(u,v) = Area of patch: (u,v) or 1 in window, 0 outside Gaussian Change in appearance for the shift [x,y]: E(x,y) = ΣΣ w(u,v) (I(u+x, v+y) – I(u,v))2 u v Roger S. Gaborski Source: Wikipedia

  27. Corner Detection: Mathematics Change in appearance for the shift [x,y]: I(u,v) E(x,y) E(3,2) E(0,0) Roger S. Gaborski Source: R. Szeliski

  28. Corner Detection Change in appearance for the shift [x,y]: • E(x,y) = ΣΣ w(u,v) (I(u+x, v+y) – I(u,v))2 We want to find out how this function behaves for small shifts Roger S. Gaborski

  29. Taylor Expansion Change in appearance for the shift [x,y]: • E(x,y) = ΣΣ w(u,v) (I(u+x, v+y) – I(u,v))2 Taylor expansion of I(u+x, v+y)= I(u,v) + Ix(u,v)*x + Iy(u,v)*y Ix and Iy are partial derivatives of x and y So, (I(u,v) + Ix(u,v)*x + Iy(u,v)*y – I(u,v))2 Results in E(x,y) = ΣΣ w(u,v) (Ix(u,v)*x+Iy(u,v)*y)2 Roger S. Gaborski

  30. Corner Detection: Mathematics The approximation simplifies to E(x,y) = (x,y) A (x,y)T Where Ais a second moment matrixcomputed from image derivatives: A = ΣΣ w(u,v) Ix2IxIy IxIy Iy2 Roger S. Gaborski

  31. Interpreting the second moment matrix Analyze eigenvalues of A to determine if a corner exists If λ1and λ2= 0, then pixel (x,y) is not of interest If one eigenvalue λ is close to 0, and the other is large, (x,y) is on an edge If both eigenvalues λ are large, (x,y) is a corner Roger S. Gaborski

  32. Interpreting the eigenvalues Classification of image points using eigenvalues of M: 2 “Edge”2 >> 1 “Corner”1 and 2 are large,1 ~ 2;E increases in all directions 1 and 2 are small;E is almost constant in all directions “Edge”1 >> 2 “Flat” region 1 Roger S. Gaborski

  33. Defining Corner Response Function, R Instead of calculating eigenvalues, Harris defined the following function: R = det(A)- α trace(A)2 = λ1λ2 - α (λ1 + λ2) Where α = .04 to .06 Recall: A = [ a b; c d] Det(A) = ad – bc Roger S. Gaborski

  34. Corner Response Function R < 0 R > 0 Edge Corners |R| small R < 0 “Flat” region Edge Roger S. Gaborski

  35. Harris Detector Algorithm • Compute Derivatives at each point • Compute Second Moment Matrix A • Compute Corner Response Function • Threshold R • Find Local Maxima Roger S. Gaborski

  36. Harris Corner Detector • Reference: C.G. Harris and M.J. Stephens “A Combined Corner and Edge Detector” • Code inspired by Peter Kovesi • Derivative Masks: dx = [-1, 0, 1;-1, 0, 1;-1, 0, 1] • dy = dx’ • Image Derivatives; • Ix = imfilter(im, dx, 'conv',‘same’); • Iy = imfilter(im, dy, 'conv',‘same’); • Gaussian Filter • g = fspecial(‘gaussian’, 6*sigma, sigma); Roger S. Gaborski

  37. Smooth squared image derivative • Ix2 = imfilter (Ix.^2, g, 'conv', ‘same’); • Iy2 = imfilter (Iy.^2, g, 'conv', ‘same’); • IxIy = imfilter (Ix .* Iy, g, 'conv', ‘same’); c = (Ix2.*Iy2 – IxIy.^2)./(Ix2+Iy2); Roger S. Gaborski

  38. Non-maximal Suppression and Threshold • Extract local maxima – gray scale morphological dilation • size = 2*radius+1; %radius is parameter • mx = imdilate(c,ones(size)); %gray scale dilate • cc = (c==mx)&(c>thresh); %find maxima • [r,c] = find(cc) %find row, col coordinates • figure, imagesc( im), colormap(gray) • hold on • plot(c,r, ‘rs’), title(‘Corners) Roger S. Gaborski

  39. 100x100 Grid background =0, lines = 1 Roger S. Gaborski

  40. Roger S. Gaborski

  41. Image rotated 45 Degrees Roger S. Gaborski

  42. Image rotated 45 Degreessame parameters Roger S. Gaborski

  43. Roger S. Gaborski

  44. Roger S. Gaborski

  45. Porsche Image Roger S. Gaborski

  46. Harris Points Roger S. Gaborski

  47. 1983 Porsche Roger S. Gaborski

  48. HW#2 – Due Thursday, 12/06 10am • Work individually or in teams of 2 • Write a Harris Detector Function (do not simply copy one from web, write your own) • Experiment with ‘grid image’ and Purple Flower image and two ‘interesting’ images of your choice. NOTE: Create your own grid image using Matlab (for example, a matrix of all zeros with white lines, pixels = 1) • Goals: - Find all intersections on grid image • Detect all petal end points on flower image – better results that class lecture slide • Email: rsg.advcv@gmail.com • 1- write up including result images, observations and 2- MATLAB code Roger S. Gaborski

  49. Pedestrian Problem • http://www.pedestrian-detection.com/ • Databases – CALTECH, others • Toolbox - Piotr's Matlab Toolbox • Papers and Code

More Related