1 / 75

Algorithms & Applications in Computer Vision

Algorithms & Applications in Computer Vision. Lihi Zelnik-Manor Lecture 11: Structure from Motion. Image segmentation. The goals of segmentation. Group together similar-looking pixels for efficiency of further processing “Bottom-up” process Unsupervised. “superpixels”.

orli
Download Presentation

Algorithms & Applications in Computer Vision

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Algorithms & Applications in Computer Vision Lihi Zelnik-Manor Lecture 11: Structure from Motion

  2. Image segmentation

  3. The goals of segmentation • Group together similar-looking pixels for efficiency of further processing • “Bottom-up” process • Unsupervised “superpixels” X. Ren and J. Malik. Learning a classification model for segmentation. ICCV 2003.

  4. The goals of segmentation • Separate image into coherent “objects” • “Bottom-up” or “top-down” process? • Supervised or unsupervised? human segmentation image Berkeley segmentation database:http://www.eecs.berkeley.edu/Research/Projects/CS/vision/grouping/segbench/

  5. Water back Grass Tiger Tiger Sand head eye legs tail mouth shadow From Pixels to Perception outdoor wildlife

  6. Berkeley Segmentation DataSet [BSDS] D. Martin, C. Fowlkes, D. Tal, J. Malik. "A Database of Human Segmented Natural Images and its Application to Evaluating Segmentation Algorithms and Measuring Ecological Statistics", ICCV, 2001

  7. Contour detection ~2004 Local 7

  8. Contour detection ~2008 (color) Global 8

  9. Inspiration from psychology • The Gestalt school: Grouping is key to visual perception The Muller-Lyer illusion http://en.wikipedia.org/wiki/Gestalt_psychology

  10. The Gestalt school • Elements in a collection can have properties that result from relationships • “The whole is greater than the sum of its parts” occlusion subjective contours familiar configuration http://en.wikipedia.org/wiki/Gestalt_psychology

  11. Emergence http://en.wikipedia.org/wiki/Gestalt_psychology

  12. Gestalt factors • These factors make intuitive sense, but are very difficult to translate into algorithms

  13. Segmentation as clustering Source: K. Grauman

  14. Two components to clustering • Similarity between pixels • For now lets assume it is the distance between RGB values. • The clustering algorithm

  15. What is Clustering? • Organizing data into classes such that: • high intra-class similarity • low inter-class similarity • Finding the class labels and the number of classes directly from the data (in contrast to classification).

  16. What is a natural grouping?

  17. What is a natural grouping? Clustering is subjective Simpson's Family Females Males School Employees

  18. What is Similarity?

  19. Defining Distance Measures Definition: Let O1 and O2 be two objects from the universe of possible objects. The distance (dissimilarity) between O1 and O2 is a real number denoted by D(O1,O2) Peter Piotr 0.23 3 342.7

  20. Segmentation as clustering Distance based on color only Source: K. Grauman

  21. Partitional Clustering • Nonhierarchical, each instance is placed in exactly one of K nonoverlapping clusters. • Since only one set of clusters is output, the user normally has to input the desired number of clusters K.

  22. Squared Error 10 9 8 7 6 5 4 3 2 1 1 2 3 4 5 6 7 8 9 10 Objective Function

  23. Algorithmk-means • 1. Decide on a value for k. • 2. Initialize the k cluster centers (randomly, if necessary). • 3. Decide the class memberships of the N objects by assigning them to the nearest cluster center. • 4. Re-estimate the k cluster centers, by assuming the memberships found above are correct. • 5. If none of the N objects changed membership in the last iteration, exit. Otherwise goto 3.

  24. k3 k1 k2 K-means Clustering: Step 1 Distance Metric: Euclidean Distance 5 4 3 2 1 0 0 1 2 3 4 5

  25. k2 k1 k3 K-means Clustering: Step 2 Distance Metric: Euclidean Distance 5 4 3 2 1 0 0 1 2 3 4 5

  26. k2 k1 k3 K-means Clustering: Step 3 Distance Metric: Euclidean Distance 5 4 3 2 1 0 0 1 2 3 4 5

  27. k2 k1 k3 K-means Clustering: Step 4 Distance Metric: Euclidean Distance 5 4 3 2 1 0 0 1 2 3 4 5

  28. k1 k2 k3 K-means Clustering: Step 5 Distance Metric: Euclidean Distance

  29. Segmentation as clustering • K-means clustering based on intensity or color is essentially vector quantization of the image attributes • Clusters don’t have to be spatially coherent Image Intensity-based clusters Color-based clusters

  30. Segmentation as clustering Distance based on color and position Source: K. Grauman

  31. Segmentation as clustering • Clustering based on (r,g,b,x,y) values enforces more spatial coherence

  32. K-Means for segmentation • Pros • Very simple method • Converges to a local minimum of the error function • Cons • Memory-intensive • Need to pick K • Sensitive to initialization • Sensitive to outliers • Only finds “spherical” clusters

  33. Mean shift clustering and segmentation • An advanced and versatile technique for clustering-based segmentation http://www.caip.rutgers.edu/~comanici/MSPAMI/msPamiResults.html D. Comaniciu and P. Meer, Mean Shift: A Robust Approach toward Feature Space Analysis, PAMI 2002.

  34. Mean shift algorithm • The mean shift algorithm seeks modes or local maxima of density in the feature space Feature space (L*u*v* color values) image

  35. Mean shift Searchwindow Center of mass Mean Shift vector Slide by Y. Ukrainitz & B. Sarel

  36. Mean shift Searchwindow Center of mass Mean Shift vector Slide by Y. Ukrainitz & B. Sarel

  37. Mean shift Searchwindow Center of mass Mean Shift vector Slide by Y. Ukrainitz & B. Sarel

  38. Mean shift Searchwindow Center of mass Mean Shift vector Slide by Y. Ukrainitz & B. Sarel

  39. Mean shift Searchwindow Center of mass Mean Shift vector Slide by Y. Ukrainitz & B. Sarel

  40. Mean shift Searchwindow Center of mass Mean Shift vector Slide by Y. Ukrainitz & B. Sarel

  41. Mean shift Searchwindow Center of mass Slide by Y. Ukrainitz & B. Sarel

  42. Mean shift clustering • Cluster: all data points in the attraction basin of a mode • Attraction basin: the region for which all trajectories lead to the same mode Slide by Y. Ukrainitz & B. Sarel

  43. Mean shift clustering/segmentation • Find features (color, gradients, texture, etc) • Initialize windows at individual feature points • Perform mean shift for each window until convergence • Merge windows that end up near the same “peak” or mode

  44. Mean shift segmentation results http://www.caip.rutgers.edu/~comanici/MSPAMI/msPamiResults.html

  45. More results

  46. More results

  47. Mean shift pros and cons • Pros • Does not assume spherical clusters • Just a single parameter (window size) • Finds variable number of modes • Robust to outliers • Cons • Output depends on window size • Computationally expensive • Does not scale well with dimension of feature space

  48. j wij i Images as graphs • Node for every pixel • Edge between every pair of pixels (or every pair of “sufficiently close” pixels) • Each edge is weighted by the affinity or similarity of the two nodes Source: S. Seitz

  49. j wij i Segmentation by graph partitioning • Break Graph into Segments • Delete links that cross between segments • Easiest to break links that have low affinity • similar pixels should be in the same segments • dissimilar pixels should be in different segments A B C Source: S. Seitz

  50. Measuring affinity • Suppose we represent each pixel by a feature vector x, and define a distance function appropriate for this feature representation • Then we can convert the distance between two feature vectors into an affinity with the help of a generalized Gaussian kernel:

More Related