1 / 29

Graph-based Segmentation

Graph-based Segmentation. Main Ideas . Convert image into a graph Vertices for the pixels Edges between the pixels Additional vertices and edges to encode other constraints Manipulate the graph to segment the image. Papers.

yorick
Download Presentation

Graph-based Segmentation

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Graph-based Segmentation

  2. Main Ideas • Convert image into a graph • Vertices for the pixels • Edges between the pixels • Additional vertices and edges to encode other constraints • Manipulate the graph to segment the image

  3. Papers • Interactive graph cuts for optimal boundary & region segmentation ofobjects in N-D images • Boykov and Jolly • Minimize an energy function • Efficient Graph-based Segmentation • Felzenszwalb and Huttenlocher • Cluster the vertices based on edge weight

  4. Boykov and Jolly • Binary image segmentation • Classify pixels as object or background • Their contribution is adding interactivity • Minimise an energy function • E(A) = B(A) + λR(A) • A: Segmentation (assign each pixels to object or background) • B(A): The cost of all the edges between object pixels and background pixels • R(A): The cost of deciding a pixel to be object or background

  5. Creating the Graph • Each pixel has a corresponding vertex • Additionally, a source (“object”) and a sink (“background”) • Each pixel vertex has an edge to its neighbours (e.g. 8 adjacent neighbours in 2D), an edge to the source, an edge to the sink

  6. Edge Weights between pixels • Weight of edges between pixel vertices are determined by the B() function • Low score when boundary is likely to pass between the vertices • high score when vertices are probably part of the same element • E.g. the difference in pixel intensities, the gradient

  7. Edges to Source/Sink • If pixel is known to be an object, use a high weight (K) to the source, zero weight to the sink • K is chosen so that it will never be cut • Conversely, if pixel is background, use weight K to the sink, zero weight to the source • Otherwise, weigh edges to source and sink appropriately using the R() function • Note that the edge to the source is the “likelihood” for the pixel being the background – we break this edge when the pixel is assigned to the background

  8. Applications • Handles arbitrary number of dimensions • Finds global minimum energy • Needs “good” user input to work effectively • Need intelligent functions • Need to select λ

  9. Felzenszwalb and Huttenlocher • Download the program from the webpage: • http://people.cs.uchicago.edu/~pff/segment/ • Minimal documentation • Short README file • Paper

  10. The program • Comes as a tar.gz archive and .zip archive • Process • Extract archive • ‘make’ (Makefile supplied) • Program consists of • A .cpp “wrapper” file (only calls the functions) • Actual algorithm functions are in .h files • Basic portable C++ code

  11. Program Testing • Built on Mac OS X, Linux, Windows cygwin • Gcc toolchain, but any C++ compiler should work • Supplied basic Makefile • Results were basically the same between platforms • Colors are chosen randomly • Results obtained are not the same as posted on the website • Image files on the website may be modified (scaled/compressed/downsampled)

  12. Algorithm • Create a graph • Each vertex corresponds to a vertex • Edges are between “neighbouring” vertices • Choose a small neighbourhood to reduce computation time (otherwise we have a complete graph) • Weight on the edge is the 5D distance between the points (for a 2D image) • 5D vector = x position, y position and 3 color components

  13. Parameters • σ: Use this value and do Gaussian smoothing (preprocessing the image to reduce noise) • k: threshold value for doing the clustering • min: “hack” parameter • the smallest cluster size must contain at least this many vertices – clusters that are too small will be merged with other clusters until sufficiently large

  14. Clustering • Put each vertex in a component • Sort edges by weight • Take each edge in turn • If the edge is between vertices in two different components A and B, we can merge if the edge weight is lower enough than the threshold • Threshold is the minimum of the following value, computed on A and B • (Lowest weight edge in minimum spanning tree of the component) + (k / size of component)

  15. Notes • Low edge weights between vertices that are likely to be in the same cluster • As a cluster gets larger, it becomes harder to add vertices to it • Heuristic – not really minimising a particular energy function • More similar to “region growing” • User has select “good” parameters to get good results

  16. Effect of σ • Increased smoothing results in removal of noise • Can cause “bleeding” – the algorithm has difficulty separating background from the object if the boundaries are too smooth

  17. Reference Images

  18. Increasing σ • Clouds are recognised as one object • Palm tree gets confused with ocean

  19. Grain • Increasing σ introduces more blurring (reduces the edge weight between pixels)

  20. Vertebrae MRI • Gets rid of noise (bottom left, right hand side), but purple vertebrate piece bleeds out

  21. Increasing Threshold • Clusters more aggresively • Palm tree is confused with ocean and clouds

  22. Grain • Non-grain pixels are almost all clustered together • Measure of how “similar” all the pixels of an object should be

  23. MRI • Vertebrae and region next to vertebrae are very similar shade so are easily confused

  24. Increasing ‘min’ value • Segmentation is the same, but small components are merged with neighboring ones

  25. Grain • Easy to control change, gets rid of small artifacts

  26. MRI • Not much effect if regions are already large

  27. Parameter Tweaking • Need to manually tune parameters to get a good image • Image of MRI after selecting • σ = 0.6 • k = 200 • min = 60

  28. Performance • Need to tune parameters by hand • Very fast • Program usually takes a couple seconds on the test images provided • Only takes ppm images in RAW data format • Theoretically, algorithm generalises to arbitrary number of dimensions and arbitrary number of features per pixel

  29. Command-line tools • Working with images involves opening up the results in an image viewer – can get messy

More Related