Segment ácia farebného obrazu

1. Segmentácia farebného obrazu

2. Image segmentation

3. Image segmentation

4. Segmentation • Segmentation means to divide up the image into a patchwork of regions, each of which is “homogeneous”, that is, the “same” in some sense - intensity, texture, colour, … • The segmentationoperation only subdivides an image; • it does not attempt to recognizethe segmented imageparts!

5. Complete vs. partial segmentation Complete segmentation - divides an image into nonoverlapping regions that match to the real world objects. • Cooperation with higher processing levels which use specific knowledge of the problem domain is necessary.

6. Complete vs. partial segmentation • Partial segmentation- in which regions do notcorrespond directly with image objects. • Image is divided into separate regions that are homogeneous withrespect to a chosen property such as brightness, color, texture, etc.

7. Gestalt (celostné) laws of perceptual organization • The emphasis in the Gestalt approach was on the configurationof the elements. Closure: Humans tend toenclose a space by completing a contour and ignoring gaps. Proximity: Objects that arecloser to one another tend to begrouped together.

8. Gestalt laws of perceptual organization Continuation: Humans tend to continue contours whenever the elements of the pattern establish an implied direction. Similarity: Elements that look similar will be perceived as part of the same form. (color, shape, texture, and motion).

9. Gestalt laws • A series of factors affect whether elements should be grouped together. • Proximity: tokens that are nearby tend to be grouped. • Similarity: similar tokens tend to be grouped together. • Common fate: tokens that have coherent motion tend to be grouped together. • Common region: tokens that lie inside the same closed region tend to be grouped together. • Parallelism: parallel curves or tokens tend to be grouped together.

10. Gestalt laws • Closure: tokens or curves that tend to lead to closedcurves tend to be grouped together. • Symmetry: curves that lead to symmetric groupsare grouped together. • Continuity: tokens that lead to “continuous” curves tend to be grouped. • Familiar configuration: tokens that, when grouped, lead to a familiar object, tend to be grouped together.

11. Gestalt laws

12. Gestalt laws

13. Consequence:Groupings by Invisible Completions Stressing the invisible groupings: * Images from Steve Lehar’s Gestalt papers: http://cns-alumni.bu.edu/pub/slehar/Lehar.html

14. Image segmentation • Segmentation criteria: a segmentation is a partition of an image I into a set of regions S satisfying: •  Si = S Partition covers the whole image. • Si Sj = , i  j No regions intersect. •  Si, P(Si) = true Homogeneity predicate is satisfied byeach region. • P(Si Sj) = false, Union of adjacent regionsi  j, Si adjacent Sj does not satisfy it.

15. Image segmentation So, all we have to do is to define and implement the similarity predicate. • But, what do we want to be similar in each region? • Is there any property that will cause the regions to be meaningful objects?

16. Segmetnation methods • Pixel-based • Histogram • Clustering • Region-based • Region growing • Split and merge • Edge-based • Model-based • Physics-based • Graph-based

17. Histogram-based segmentation • How many “orange” pixels arein this image? • This type of question can be answeredby looking at the histogram.

18. Histogram-based segmentation • How many modes are there? • Solve this by reducing the number of colors to K and mapping each pixel to the closest color. • Here’s what it looks like if we use two colors.

19. Clustering-based segmentation • How to choose the representative colors? • This is a clustering problem! • K-means algorithm can be used for clustering.

20. Clustering-based segmentation K-means clustering of color.

21. Clustering-based segmentation K-means clustering of color.

22. Results of K-Means Clustering: Image Clusters on intensity Clusters on color K-means clustering using intensity alone and color alone * From Marc Pollefeys COMP 256 2003

23. Clustering-based segmentation • Clustering can also be used with other features (e.g., texture) in addition to color. Original Images Color Regions Texture Regions

24. Clustering-based segmentation • K-means variants: • Different ways to initialize the means. • Different stopping criteria. • Dynamic methods for determining the right number of clusters (K) for a given image. • Problem: histogram-based and clustering-based segmentation can produce messy regions. • How can these be fixed?

25. Clustering-based segmentation • Expectation-Maximization (EM) algorithm can be used as a probabilistic clustering method where each cluster is modeled using a Gaussian. • The clusters are updated iteratively by computing the parameters of the Gaussians. Example from the UC Berkeley’s Blobworld system.

26. Clustering-based segmentation Examples from the UC Berkeley’s Blobworld system.

27. Region growing • Region growing techniques start with one pixel of a potential region and try to grow it by adding adjacent pixels till the pixels being compared are too dissimilar. • The first pixel selected can be just the first unlabeled pixel in the image or a set of seed pixels can be chosen from the image. • Usually a statistical test is used to decide which pixels can be added to a region. • Region is a population with similar statistics. • Use statistical test to see if neighbor on border fits into the region population.

28. Region growing image segmentation

29. Split-and-merge • Start with the whole image. • If the variance is too high, break into quadrants. • Merge any adjacent regions that are similar enough. • Repeat steps 2 and 3, iteratively until no more splitting or merging occur.  Idea: goodResults: blocky

30. Split-and-merge

31. Split-and-merge

32. Split-and-merge A satellite image. A large connected region formed by merging pixels labeled as residential after classification. More compact sub-regions after the split-and-merge procedure.

33. Mean Shift Segmentation http://www.caip.rutgers.edu/~comanici/MSPAMI/msPamiResults.html

34. Mean Shift Algorithm • Mean Shift Algorithm • Choose a search window size. • Choose the initial location of the search window. • Compute the mean location (centroid of the data) in the search window. • Center the search window at the mean location computed in Step 3. • Repeat Steps 3 and 4 until convergence. The mean shift algorithm seeks the “mode” or point of highest density of a data distribution:

35. Mean Shift Segmentation • Mean Shift Setmentation Algorithm • Convert the image into tokens (via color, gradients, texture measures etc). • Choose initial search window locations uniformly in the data. • Compute the mean shift window location for each initial position. • Merge windows that end up on the same “peak” or mode. • The data these merged windows traversed are clustered together. *Image From: Dorin Comaniciu and Peter Meer, Distribution Free Decomposition of Multivariate Data, Pattern Analysis & Applications (1999)2:22–30

36. Mean Shift Segmentation Extension Is scale (search window size) sensitive. Solution, use all scales: • Gary Bradski’s internally published agglomerative clustering extension: • Mean shift dendrograms • Place a tiny mean shift window over each data point • Grow the window and mean shift it • Track windows that merge along with the data they transversed • Until everything is merged into one cluster Best 4 clusters: Best 2 clusters: Advantage over agglomerative clustering: Highly parallelizable

37. Mean Shift Segmentation Results: http://www.caip.rutgers.edu/~comanici/MSPAMI/msPamiResults.html

38. Graph Cut

39. Graph Cut • First: select a region of interest

40. Graph Cut • How to select the object automatically? ?

41. Graph Cut • What are graphs? • Nodes • usually pixels • sometimes samples • Edges • weights associated(W(i,j)) • E.g. RGB value difference

42. Graph Cut • What are cuts? • Each “cut” -> points, W(I,j) • Optimization problem • W(i,j) = |RGB(i) – RGB(j)|

43. Graph Cut • Go back to our selected region • Each “cut” -> points, W(I,j) • Optimization problem • W(i,j) = |RGB(i) – RGB(j)|

44. Graph Cut • Go back to our selected region • Each “cut” -> points, W(I,j) • Optimization problem • W(i,j) = |RGB(i) – RGB(j)|

45. Graph Cut • We want highest sum of weights • Each “cut” -> points, W(I,j) • Optimization problem • W(i,j) = |RGB(i) – RGB(j)|

46. Graph Cut • We want highest sum of weights • Each “cut” -> points, W(I,j) • Optimization problem • W(i,j) = |RGB(i) – RGB(j)| • These cuts give low points • W(i,j) = |RGB(i) – RGB(j)|is low

47. Graph Cut • We want highest sum of weights • Each “cut” -> points, W(I,j) • Optimization problem • W(i,j) = |RGB(i) – RGB(j)| • These cuts give high points • W(i,j) = |RGB(i) – RGB(j)|is high

48. Graph-based segmentation • An image is represented by a graph whose nodes are pixels or small groups of pixels. • The goal is to partition the nodes into disjoint sets so that the similarity within each set is high and across different sets is low. http://www.cs.berkeley.edu/~malik/papers/SM-ncut.pdf

49. Graph-based segmentation • Let G = (V,E) be a graph. Each edge (u,v) has a weight w(u,v) that represents the similarity between u and v. • Graph G can be broken into 2 disjoint graphs with node sets A and B by removing edges that connect these sets. • Let cut(A,B) =  w(u,v). • One way to segment G is to find the minimal cut. uA, vB

50. Graph-based segmentation