REGION SEGMENTATION

1. REGION SEGMENTATION By Ajay Gupta Computer Vision

2. Overview Definition Application Classification of methods Region based segmentation Computer Vision

3. Definition Segmentation refers to the process of partitioning a image into multiple regions. Regions:- A group of connected pixels with similar properties. Regions are used to interpret images. A region may correspond to a particular object, or different parts of an object. Computer Vision

4. Definition (contd..) • In most cases, segmentation should provide a set of regions having the following properties • Connectivity and compactness • Regularity of boundaries • Homogeneity in terms of color or texture • Differentiation from neighbor regions Computer Vision

5. Applications Automatic car assembly in robotic vision Airport security systems Object recognition Criminal investigation Computer graphics Medical Imaging MPEG-4 video object (VO) segmentation Computer Vision

6. Why do we need segmentation? The goal of segmentation is to simplify the representation of an image into something that is more meaningful and easier to analyze. Image segmentation is typically used to locate objects and boundaries in images. For correct interpretation, image must be partitioned into regions that correspond to objects or parts of an object. Computer Vision

7. Basic Formulation Let R represent the entire image region. We want to partition R into n sub regions, R1, R2, . . ., Rn, such that: (a) Summation of Ri =R (b) Ri is a connected region for i=1, 2, . . , n (c) Ri intersection Rj =φ for all i and j , I= j (d) P(Ri) = TRUE for i=1, 2, . . . n (e) P( Ri summation Rj)= False , i = j Computer Vision

8. Basic Formulation (a) segmentation must be complete – all pixels must belong to a region (b) pixels in a region must be connected (c) Regions must be disjoint (d) states that pixels in a region must all share the same property – The logic predicate P(Ri) over a region must return TRUE for each point in that region (e) indicates that regions are different in the sense of the predicate P. Computer Vision

9. Segmentation Effect Region Segmented Image Computer Vision

10. Difficulties • Image segmentation is considered as a very difficult task • The main reasons for these difficulties are • No universal definition of objects or regions • Images suffer from several weaknesses • Irregular illumination, shadows and reflections • Objects with textured surfaces • Noisy and blurred images • Partially occluded objects • Segmentation results are very sensitive to parameter choices Computer Vision

11. Approaches to segmentation • Region based approaches • group together pixels with similar properties • combining proximity and similarity Computer Vision

12. Classification • Region based approaches are based on pixel properties such as • Homogeneity • Spatial proximity • The most used methods are • Thresholding • Clustering • Pixel aggregation • Region growing • Split and merge Computer Vision

13. Classification (contd..) • Homogeneity: Pixels that form an object have same gray level. • Spatial proximity: The distance between the two pixels are measured by using the Euclidean distance. • Euclidean distance: It is the straight line distance between two points. If p1 at (x1, y1) and p2 at (x2, y2) are the two points in a plane then the Euclidean distance is √((x1 - x2)² + (y1 - y2)²). Computer Vision

14. Thresholding A parameter θ called the brightness threshold is chosen and applied to an image A[x,y]. If A[x,y] ≥ θ A[m,n] = object = 1 else A[m,n] = background = 0 The results is a binary image, but that alone is not yet segmentation. This must further be worked out by joining pixels together into one or more regions or areas. Computer Vision

15. Thresholding • Pixels are classified by comparing their values • (intensity or other function) with one or several thresholds • Thresholds are determined by histograms and can be fixed • 1. Globally • 2. Locally • 3. Dynamically • Correct thresholding leads to better segmentation • The choice of good thresholds is mandatory • They exist if the histogram peaks are separated by deep valleys Computer Vision

16. Thresholding • Thresholding do not consider spatial properties such as proximity • Thresholding can be used when • Regions are homogeneous • Objects and background have a strong contrast • Thresholding is inappropriate when • Regions have an important gradient • Regions are strongly textured Computer Vision

17. Thresholding ● Advantages : ● Drawbacks : Very fast The threshold segmentation method is easy to grasp The threshold method depends on the possibility to define a threshold that works well everywhere in the image Require a region growing or other technique of segmentation if two objects have the same color Computer Vision

18. Region Representation • Three classes of region representations : • Array representations. • Hierarchical representations. • Symbolic representations. Computer Vision

19. Array Representation • Use an array of the same size as the original image with entries that indicate the region to which a pixel belongs. • Use membership arrays (mask, bitmap) Computer Vision

20. Hierarchical Representations • Allows image representation at multiple resolutions • Commonly used forms: • Pyramids • Quad trees Computer Vision

21. Pyramids • Pyramid representation of an n × n image contains the image and k reduced versions of the image. n is the power of 2. • The other images are n/2 ×n/2, n/4 ×n/4… • The pixel at level i is obtained by combining information from several pixels in the image at level i + 1. • An implementation point is that the entire pyramid fits into a linear array of size: 2(22*level) Computer Vision

22. Pyramids Computer Vision

23. Quad Trees • An extension of pyramids for binary images. • Contains three types of nodes : white, black, and gray. • Is obtained by recursive splitting an image. • A region in an image is split into four subregions of identical size. Computer Vision

24. Picture of Quad Tree Computer Vision

25. Symbolic Representations • Some Commonly used symbolic characteristics: • Enclosing Rectangle • Centroid • Moments • Euler number • Others: • Mean, variance of intensity values Computer Vision

26. Clustering • Principle • Given N points in feature space (for instance RGB pixel values) and K categories • Pick randomly K points as initial centers m1,… mK. • Repeat the following step • Assign each points, to the cluster with nearest mi • Recompute mean mi of each cluster from its member points (make sure no cluster is empty) • Until no mean has changed more than some ε Computer Vision

27. Clustering results are not always unique ! Computer Vision

28. Clustering method ● Advantages : ● Drawbacks : Many clusters Necessity to identify the clusters. Depends on the initial number of cluster. Computer Vision

29. Pixel Aggregation • The basic idea is to grow from a seed pixel • At a labeled pixel, check each of its neighbors • If its attributes are similar to those of the already labeled pixel,label the neighbor accordingly • Repeat until there is no more pixel that can be labeled • A simple case • The attribute of a pixel is its pixel value • The similarity is defined as the difference between adjacent pixel values • If the difference is smaller than a threshold, they are assigned to the same region, otherwise not Computer Vision

30. Region Growing : Region Adjacency Graphs • Region growing consists of • starting with a very fine segmentation • merging together similar adjacent regions • Region Adjacency Graphs are used to represent segmentation data • each node represents a region • an edge exits between two nodes if the corresponding regions are adjacent Computer Vision

31. RAG • Nodes represent regions; arcs between nodes represent a common boundary between regions. • Different properties of regions stored in node data structure. • After initial segmentation, results stored in RAG and regions combined to obtain a better segmentation. Computer Vision

32. RAG Example Computer Vision

33. Region Growing : Homogeneity predicate • A homogeneity predicate H(R) is a function that takes a region R and returns true or false according to the pixel properties • A segmentation result is given by a partition {R1,R2,...,Rn} such that • ∀ k Rk is connex • ∀ k H(Rk) is true • ∀ i ≠ j if Ri is a neighbor of Rj then H(Ri∪Rj) is false • As predicates we can choose • Absolute intensity homogeneity • ∀ p,q ∈ S | I(p) − I(q) | ≤ s • Differential intensity homogeneity • ∀ p,q ∈ S and (p neighbor of q)| I(p) − I(q) | ≤ s Computer Vision

34. Region Split and Merge • After segmentation the regions may need to be refined or reformed. • Split operation adds missing boundaries by splitting regions that contain part of different objects. • Merge operation eliminates false boundaries and spurious regions by merging adjacent regions that belong to the same object. Computer Vision

35. Split and merge • Split into a quad-tree representation • A region is split if the homogeneity predicate is not satisfied • This may lead to over-segmentation ! • Two adjacent regions are merged if the homogeneity predicate is satisfied • This can be done by growing each region toward its neighbors Computer Vision

36. Region Merging • Determines the similarity between two regions. • Approaches to judge the similarity • on the gray value of regions • on the weakness of boundaries between the regions Computer Vision

37. Algorithm: Region Merging (1) Form initial regions in the image using thresholding ( or a similar approach) followed by component labeling. (2) Prepare a region adjacency graph (RAG) for the image. (3) For each region in an image, perform the following steps: (a) Consider its adjacent region and test to see if they are similar. (b) For regions that are similar, merge them and modify the RAG. (4) Repeat step 3 until no regions are merged. Computer Vision

38. Region Splitting • If some property of a region is not constant • Regular decomposition Methods: divide the region into a fixed number of equal-sized regions. Computer Vision

39. Algorithm: Region Splitting • Form initial region in the image • For each region in an image, recursively perform: • Compute the variance in the gray values for the region • If the variance is above a threshold, split the region along the appropriate boundary Computer Vision

40. Algorithm: Split and Merge Region Segmentation • Start with the entire image as a single region • Split: Pick a region R. If H(R) is false, then split the region into four subregions. • H(R)=1 if the variance is small, 0 otherwise • Merge: Consider any two or more neighboring subregions, R1, R2,…, Rn, in the image. If H(R1UR2U…Rn) is true, merge the n regions into a single region • Repeat these steps until no further splits or merges. Computer Vision

41. Conclusion A universal algorithm of segmentation doesn't exist, as each type of image corresponds to a specific approach. Computer Vision

42. Links [1] http://en.wikipedia.org/wiki/Data_clustering [2] MÉMOIRE DE STAGE DE MASTER - NICOLAS PASSALACQUA - Méthodes de clustering flou appliquées au "filtrage parental" d'Internet – 2006 - Effectué au LGI2P sous la direction de GERARD DRAY [3] 5th Slovakian-Hungarian Joint Symposium on Applied Machine Intelligence and Informatics - Marián Bakoš - Active Contours and their Utilization at Image Segmentation – 2007 - http://www.bmf.hu/conferences/sami2007/31_Bakos.pdf [4] http://iacl.ece.jhu.edu/projects/gvf/ [5] PCVO2 Photogrammetric Computer Vision ISPRS Commission III, Symposium 2002 September 9 - 13, 2002, Graz, Austria - Marco Roggero - OBJECT SEGMENTATION WITH REGION GROWING AND PRINCIPAL COMPONENT ANALYSIS [6] ENSTA Module D9 – ES322 - Jean-Christophe Baillie - Traitement d'Image et Vision Artificielle - 2003 Computer Vision

43. Links [7] http://uei.ensta.fr/baillie/assets/ES322%20-` %20Segmentation.ppt [8] http://www.ph.tn.tudelft.nl/Courses/FIP/noframes/fip- Segmenta.html#Heading118 [9]http://perso.enst.fr/~maitre/BETI/snakes_coul/MBAI.html [10] http://www.cs.ru.nl/~ths/rt2/col/h7/7contourENG.html Computer Vision