Lecture 08

1. Lecture 08 27/12/2011 ShaiAvidan הבהרה: החומר המחייב הוא החומר הנלמד בכיתה ולא זה המופיע / לא מופיע במצגת.

2. Today • Hough Transform • Generalized Hough Transform • Implicit Shape Model • Video Google

3. Hough Transform & Generalized Hough Transform

4. y ρ x θ Hough Transform • Origin: Detection of straight lines in clutter • Basic idea: each candidate point votes for all lines that it is consistent with. • Votes are accumulated in quantized array • Local maxima correspond to candidate lines • Representation of a line • Usual form y = a x + b has a singularity around 90º. • Better parameterization: x cos() + y sin() = K. Grauman, B. Leibe

5. Examples • Hough transform for a square (left) and a circle (right) K. Grauman, B. Leibe

6. Hough Transform: Noisy Line ρ • Problem: Finding the true maximum θ Tokens Votes K. Grauman, B. Leibe

7. Hough Transform: Noisy Input ρ • Problem: Lots of spurious maxima θ Tokens Votes K. Grauman, B. Leibe

8. Generalized Hough Transform [Ballard81] • Generalization for an arbitrary contour or shape • Choose reference point for the contour (e.g. center) • For each point on the contour remember where it is located w.r.t. to the reference point • Remember radius r and angle relative to the contour tangent • Recognition: whenever you find a contour point, calculate the tangent angle and ‘vote’ for all possible reference points • Instead of reference point, can also vote for transformation  The same idea can be used with local features! K. Grauman, B. Leibe Slide credit: Bernt Schiele

9. Implicit Shape Model

10. Gen. Hough Transform with Local Features • For every feature, store possible “occurrences” • For new image, let the matched features vote for possible object positions K. Grauman, B. Leibe

11. 3D Object Recognition [Lowe99] • Gen. HT for Recognition • Typically only 3 feature matches needed for recognition • Extra matches provide robustness • Affine model can be used for planar objects K. Grauman, B. Leibe Slide credit: David Lowe

12. View Interpolation • Training • Training views from similar viewpoints are clusteredbased on feature matches. • Matching features between adjacent views are linked. • Recognition • Feature matches may bespread over several training viewpoints.  Use the known links to “transfer votes” to other viewpoints. [Lowe01] K. Grauman, B. Leibe

13. Recognition Using View Interpolation K. Grauman, B. Leibe

14. Location Recognition Training K. Grauman, B. Leibe

15. Applications • Sony Aibo(Evolution Robotics) • SIFT usage • Recognize docking station • Communicate with visual cards • Other uses • Place recognition • Loop closure in SLAM K. Grauman, B. Leibe Slide credit: David Lowe

16. Video Google

17. Indexing local features • Each patch / region has a descriptor, which is a point in some high-dimensional feature space (e.g., SIFT) K. Grauman, B. Leibe

18. Indexing local features • When we see close points in feature space, we have similar descriptors, which indicates similar local content. K. Grauman, B. Leibe Figure credit: A. Zisserman

19. Indexing local features • We saw in the previous section how to use voting and pose clustering to identify objects using local features Figure credit: David Lowe K. Grauman, B. Leibe

20. Indexing local features • With potentially thousands of features per image, and hundreds to millions of images to search, how to efficiently find those that are relevant to a new image? • Low-dimensional descriptors : can use standard efficient data structures for nearest neighbor search • High-dimensional descriptors: approximate nearest neighbor search methods more practical • Inverted file indexing schemes K. Grauman, B. Leibe

21. Indexing local features: inverted file index • For text documents, an efficient way to find all pages on which a word occurs is to use an index… • We want to find all images in which a feature occurs. • To use this idea, we’ll need to map our features to “visual words”. K. Grauman, B. Leibe

22. Visual words • More recently used for describing scenes and objects for the sake of indexing or classification. Sivic & Zisserman 2003; Csurka, Bray, Dance, & Fan 2004; many others. K. Grauman, B. Leibe

23. Inverted file index for images comprised of visual words List of image numbers Word number K. Grauman, B. Leibe Image credit: A. Zisserman

24. Bags of visual words • Summarize entire image based on its distribution (histogram) of word occurrences. • Analogous to bag of words representation commonly used for documents. K. Grauman, B. Leibe Image credit: Fei-Fei Li

25. Video Google System Query region • Collect all words within query region • Inverted file index to find relevant frames • Compare word counts • Spatial verification Sivic & Zisserman, ICCV 2003 • Demo online at : http://www.robots.ox.ac.uk/~vgg/research/vgoogle/index.html Retrieved frames K. Grauman, B. Leibe

26. Visual vocabulary formation Issues: • Sampling strategy • Clustering / quantization algorithm • What corpus provides features (universal vocabulary?) • Vocabulary size, number of words K. Grauman, B. Leibe

27. Sampling strategies Sparse, at interest points Dense, uniformly Randomly • To find specific, textured objects, sparse sampling from interest points often more reliable. • Multiple complementary interest operators offer more image coverage. • For object categorization, dense sampling offers better coverage. • [See Nowak, Jurie & Triggs, ECCV 2006] Multiple interest operators K. Grauman, B. Leibe Image credits: F-F. Li, E. Nowak, J. Sivic

28. Clustering / quantization methods • k-means (typical choice), agglomerative clustering, mean-shift,… K. Grauman, B. Leibe