Shape-based Image Retrieval for Scene Analysis using Split-and-Merge Algorithm

1. Shape-based Image Retrieval

2. 4 Image Retrieval by Shape Similarity

3. Shape Here: shape is geometry

4. QBIC – Search by shape ** Images courtesy : Yong Rao

7. 在设计描述子时需要考虑的要素 完整性 可计算性 不变性 鲁棒性

8. 周边 离心率 凸包 自回归 弹性

10. Region-based similar Contour based similar

14. 利用x,y两个平面的投影 （视图）描述物体的形状

15. Region Finding（segmentation) • Instead of (or in addition to) detecting edges in an image, we can also search for homogeneous regions as the basis for scene analysis. • In such regions, the intensity, texture, or other features do not change abruptly (突然地). • Often, both edge detection and region finding are used to complement each other.

16. Region Finding • Definition of a region: • A region is a set of connected pixels meeting two requirements: • A region is homogeneous. This could be defined as, for example, the maximum difference in intensity values between two pixels being no greater than some threshold . （内部特性） • The union of the pixels of two adjacent regions never satisfies the homogeneity criterion. （不可再扩大性）

17. The Split-and-Merge Algorithm • 由两个阶段组成 • 1）由上而下的，通过对图像进行划分来发现同质的区域 • 2）由下而上地，通过对邻近同质的区域的合并满足第2个性质。

18. The Split-and-Merge Algorithm • First, we perform splitting: • At the start of the algorithm, the entire image is considered as the candidate region. • If the candidate region does not meet the homogeneity criterion, we split it into four smaller candidate regions. • This is repeated until there are no candidate regions to be split any more. • Then, we perform merging: • Check all pairs of neighboring regions and merge them if it does not violate the homogeneity criterion.

19. The Split-and-Merge Algorithm • Sample image 阈值定义为2 即差值<2

20. The Split-and-Merge Algorithm • First split

21. The Split-and-Merge Algorithm • Second split

22. The Split-and-Merge Algorithm • Third split

23. The Split-and-Merge Algorithm • Merge

24. The Split-and-Merge Algorithm • Final result

25. Region Finding • There are a variety of algorithms for region finding. • Another example is “Smooth Region Analysis” (AT&T Cambridge): • Design a filter that detects smoothness of color and texture, • apply the filter to create a smoothness image, • find closed regions of high smoothness.

26. Region Finding • Sample image

27. Region Finding • Smoothness image

28. Region Finding • Resulting regions shaded to their mean color

29. Chain Codes（Contour description) • An interesting way of describing a contour is using chain codes. • A chain code is an ordered list of local orientations (directions) of a contour. • These local directions are given through the locations of neighboring pixels, so there are only eight different possibilities. • We assign each of these directions a code:

30. Chain Codes • Then we start at the first edge in the list and go clockwise around the contour. • We add the code for each edge to a list, which becomes our chain code.

31. Chain Codes • What happens if in our chain code for a given contour we replace every code n with (n mod 8) + 1 ? • The contour will berotated clockwise by 45 degrees. • We can also compute the derivative of a chain code, also referred to as difference code. • Difference codes are rotation-invariant descriptions of contours. • Some features of regions, such as their corners or areas, can be directly computed from chain or difference codes.

32. Similarity computing • Given two strings A=a1a2…an, B=b1b2…bm • how to measure the dissimilarity between A and B •  Given two strings A=a1a2…an, B=b1b2…bm • how to change A to B character by character • using three types of edit operation • (1) insert a character • (2) delete a character • (3) replace one character with a different character • such that the number of edit steps is minimum •  Dissimilarity = minimum number of edit steps

33. Distance computing for two sets

34. The shape through transformation approach • Shapes can be distinguished by measuring the effort needed to transform one shape into another • Similarity is measured as a transformational distance

35. 垂直 伸长率

36. Use of global features • All shapes are represented as binary images .A set of 22 features is used for their representation: • Area,computed as the number of pixels, set in the binary image • Circularity ,expressed as the ratio between the squared perimeter and the area • Eccentricity (离心率), computed as the ratio between the smallest and the largest eigenvalues

37. (

39. ( 凹度树)

40. Based on digital moments • If we consider binary images ,where region R (whose shape we want to describe) is the set of points so that ,the digital i, j-th moment of the R is represented by • is the count of and represent the area of R

41. Based on digital moments • The centroid of R is expressed as :

42. Based on digital moments • Powerful descriptors based on digital moments are functions of moments that are invariant under scaling , translation , rotation and squeezing . • Normalized moments ,are invariant under translation , scaling , stretching and squeezing transformations

43. Based on digital moments • Normalized moments are defined by • Considering the central i , j -th moments

44. Based on digital moments • Normalizing coordinates by their standard deviations and

45. Based on digital moments • And then normalized by the area : (4.1)

46. Based on digital moments normalized central moments are obtained from central moments according to the following transformation: (4.2) (4.3)

47. Based on digital moments • A set of functions are defined from these moments . Six of them are rotation invariant ,and one is both skew and rotation invariant:

48. Based on digital moments

49. Based on edge image correlation

50. SHAPE-BASED RETRIEVAL • Shape transformationapproaches model shape similarity more closely to human perception • These methods are typically more robust in order to cope with shape distortions , support a higher precision in retrieving similar images and allow comparison with partially occluded shapes • Indexing with these approaches is impossible • 具体检索，可以根据基于chain code的相似性计算，面积计算等。更主要的应用是对象发现。