1 / 69

Image segmentation using Eigenvectors

Image segmentation using Eigenvectors. Speaker : Sameer Agarwal Course : Learning and Vision Seminar Date : 09/10/2001.

roxanne
Download Presentation

Image segmentation using Eigenvectors

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Image segmentation using Eigenvectors Speaker : Sameer Agarwal Course : Learning and Vision Seminar Date : 09/10/2001

  2. “Theoretically I might say there are 327 brightnesses and nuances of color. Do I have “327” No. I have sky, house, and trees. It is impossible to achieve “327” as such. And yet even though such droll calculations are possible--- and implied, say, for the house 120, the trees 90 and the sky 117– I should at least have this arrangement and division of the total, and not, say, 127 and 100 and 100; or 150 and 177.” Laws of Organization in Perceptual Forms Max Wertheimer (1923)

  3. What is Image Segmentation ? • Partitioning of an image into related regions.

  4. Why do Image Segmentation ? • Image Compression - Identify distinct components within an image and use the most suitable compression algorithm for each component to get a higher compression ratio. • Medical Diagnosis - Automatic segmentation of MRI images for identification of cancerous regions • Mapping and Measurement - Automatic analysis of remote sensing data from satellites to identify and measure regions of interest. e.g. Petroleum reserves.

  5. How many groups ? Out of the various possible partitions, which is the correct one ?

  6. The bayesian view Given prior knowledge about the structure of the data, choose the partition which is most probable. Problem : How do you specify a prior for knowledge which is composed of knowledge on multiple scales. e.g. • Coherence • Symmetry

  7. A simple implementation • Assume that the image was generated by a mixture of multiple models • Segmentation is done in two steps : • Estimate the parameters of the mixture model • For each point calculate the posterior probabilities of it belonging to a cluster. Assign to the cluster with the maximum posterior.

  8. Why doesn’t it work ? • The model selection problem. • Number of components ? • The structure of the components? • Estimation problem transforms into a hard optimization problem. No guarantee of convergence to the global optima.

  9. Prior Work • k-means • Mixture Models (Expectation Maximization) • k-Medoid • k-Harmonic • Self Organizing Maps • Neural Gas • Linkage based graph methods.

  10. Outline of the talk • The Gestalt approach to perceptual grouping • Graph theoretic formulation of the segmentation problem • The normalized cut • Experimental results • Relation to other methods • Conclusions

  11. The Gestalt approach • Gestalt : a structure, configuration, or pattern of physical, biological, or psychological phenomena so integrated as to constitute a functional unit with properties not derivable by summation of its parts “The whole is different from the sum of the parts”

  12. The Gestalt Movement • Formed by Max Werthheimer, Wolfgang Kohler and Kurt Koffka. • Rejected structuralism and its assumptions of atomicity and empiricism. • Adopted a “Holistic” approach to perception.

  13. An Example Emergent properties of a configuration. The arrangement of several dots in a line gives rise to emergent properties, such as length, orientation and curvature, that are different from the properties of the dots that compose it.

  14. Gestalt Cues

  15. And the moral of the story is .. • Image segmentation based on low level cues cannot and should not aim to produce a complete final “correct” segmentation. • Instead use low-level attributes like color, brightness to sequentially come up with hierarchical partitions. • Mid and high-level knowledge can be used to either confirm or select some partition for further attention.

  16. A graph theoretic approach • A weighted undirected graph G = (V,E) • Nodes are points in the feature space • Fully connected graph • Edge weight w(i,j) is a function of the similarity between nodes i and j. Task: Partition the set V into disjoint sets V1,..,Vn, s.t. similarity among nodes in Vi is high and similarity across Vi and Vj is low.

  17. Issues • What is a good partition ? • How can you compute such a partition efficiently ?

  18. Graph Cut • G=(V,E) • Sets A and B are a disjoint partition of V Cut(A,B) is a measure of similarity between the two groups.

  19. The temptation Cut is a measure of association Minimizing it will give a partition with the maximum disassociation. Efficient poly-time algorithms algorithms exist to solve the MinCut problem. So why not use it ?

  20. The problem with MinCut

  21. The Normalized Cut Given a partition (A,B) of the vertex set V. Ncut(A,B) measures similarity between two groups, normalized by the “volume” they occupy in the whole graph.

  22. Matrix formulation Definitions: D is an n x n diagonal matrix with entries W is an n x n symmetrical matrix

  23. NP-Complete After some linear algebra we get.. • Subject to the constraints: • y(i)ε {1,-b} • ytD1=0

  24. Real numbers to the rescue • Relax the constraints on y, and allow it to take real value. • Claim : The real valued MinNcut(G) can then be solved for by solving the generalized eigenvalue problem for the second smallest generalized eigenvector.

  25. Proof • Rewrite the equation as Here Lemma 1: is an eigenvector of the above eigensystem with eigenvalue 0.

  26. Proof(contd.) Lemma 2 : is a positive definite matrix since (D-W) is known to be positive semi-definite. Lemma 3 : z0 is the smallest eigenvector of eigensystem. Lemma 4 : z1 is perpendicular to z0

  27. Proof (contd.) Lemma 5 : Let A be a real symmetric matrix, Under the constraint that x is orthogonal to the j-1 smallest eigenvectors x1,…,xj-1,the quotient is minimized by the next smallest eigenvector.

  28. Finally.. • By lemma 1 we have y0=1 is an eigenvector of the eigensystem with eigenvalue 0. • It is the “smallest” eigenvector. • Hence by lemma 2, the second smallest eigenvector (y1) will minimize the Ncut equation. • By lemma 3 and 4 z1tz0= y1tD1=0

  29. What about the first constraint ? • The second smallest eigenvector is only an approximation to the optimal normalized cut. y1 minimizes Y will take similar values for nodes with with high similarity value.

  30. The grouping algorithm • Given an image, set up the weighted graph G=(E,V). Set the weight on the edges connecting two nodes as a measure of the similarity between the nodes. • Solve (D-W)x=λDx for eigenvectors with the smallest eigenvalues. • Use the second smallest eigenvector to bipartition the graph.

  31. Details.. • The eigenvector takes continuous values, how do use it to segment the image ? • Choose 0 as the splitting point. • Find the median of the eigenvector and use that as the splitting point • Search amongst l evenly spaced points for one which gives the best exact Ncut value. • Impose a stability criterion on the eigenvector.

  32. Stability ? • Since we allow the eigenvectors to take real values. Some eigenvectors might take a smooth continuous form. • We want vectors that have sharp discontinuities, indicating separation between regions. • Measure the smoothness of the vector, and stop partitioning when the smoothness value falls below a threshold.

  33. Detail.. (Contd.) • How do you partition images with multiple segments ? 1. The higher order eigenvectors contain information about sub-partitions. Keep splitting till Ncut exceeds some pre-specified value. Problem : Numerical Error 2. Recursively run the algorithm on successive subgraphs. Problem : Computationally Expensive and the stability criterion might prevent correct partitioning.

  34. Simultanous P-way cut • Use the first n eigenvectors as n-dimensional indicator vectors of each point. This is equivalent to imbedding each point in an n-dimensional space. • Perform k-means clustering in this new space to create p’>p clusters. • Use the original 2-way Ncut or a greedy strategy to merge these p’ partitions into p partitions.

  35. How good is the approximation ? The normalized cheeger constant h is defined as : We know that the second eigenvalue is bounded by : This is only a qualitative indication of the quality of approximation, it does not say anything about how close the eigenvector is to the optimal Ncut vector.

  36. Example I

  37. Distance Matrix

  38. The second generalized eigenvector

  39. The first partition

  40. The second generalized eigenvector

  41. The second partition

  42. The fourth generalized eigenvector

  43. The third partition

  44. Example II

  45. The structure of the affinity matrix

  46. Generalized eigenvalues

  47. The first partition

  48. The second partition

  49. The third partition

  50. The fourth partition

More Related