Outline

1. Multiresolution Analysis of ConnectivityAtul Sajjanhar, Deakin University, Australia Guojun Lu, Monash University, Australia Dengsheng Zhang, Monash University, Australia Tian Qi, I2R, Singapore

2. Outline • Background of Traditional Approach • Motivation of Coherent Features • Coherence Features in Contour Shapes • Connectivity in Region Shapes • Multiresolution Shape Analysis • Principal Component Analysis of Shape Features • Experiment Results • Conclusions

3. Background • CBIR—content based image retrieval, using low level features such as color, texture and shape • Shape—one of the important image features • Contour based approach • Region based approach • Existing techniques ignore the geodesic feature of the shape which is an important information • We attempt to employ coherence feature in shape to represent shape more accurately

4. Traditional Histogram Approach • Consider contour based shape retrieval • Shape is first abstracted by the centroidal distance function which is the distance from the centroid to the points on the boundary. • The centroidal distances are discretised into buckets • The values of the buckets are normalised as the shape features • Distance between shapes is computed as the sum of differences between corresponding buckets.

5. rN-1 (xc, yc) r0 r1 ri+1 ri r0 rN-1 (b) (a) Traditional Histogram

6. Motivation of Coherent Features • Drawback of traditional histograms is the lack of spatial continuity information, as a result, shapes which are completely different may have the same histograms • An analogy is drawn from Color Coherence Vectors (CCV) proposed by Pass and Zabih. CCV is used for image retrieval based on colour: Pixels are classified as coherent or incoherent. Coherent pixels are part of a sizable contiguous region of similar colour while incoherent pixels are not.

7. Coherence • In the case of shape representation, we define “distance coherence” of pixels on the shape boundary. • If the centroidal distance of a sample point on the shape boundary is part of a segment in which all the sample points have the same discretised centroidal distance then it is coherent. • On the other hand, if the discretised centroidal distance of a sample point is different from the discretised centroidal distances of it’s immediate neighbours then it is incoherent.

8. Coherence Based Histogram Each bucket is decomposed into coherent and incoherent

9. Limitation • Coherence can be conveniently defined for contour based shape retrieval • However, it cannot be directly applied to region based shape like the image below.

10. Connectivity In the case of region based shape, we define “connectivity” of pixels in an image. The state of the nearest 8-neighbours is computed for each dark pixel. Connectivity of a dark pixel is obtained as the number of dark pixels amongst the nearest 8-neighbours. Decomposition by Connectivity

11. Distance Histogram of Region Shapes • A distance function is computed by finding the number of shape pixels on concentric circles centered at the shape centroid • The distance function is then discretised into buckets to obtain the distance histogram • The similarity between shapes is determined by the sum of differences between corresponding buckets.

12. Connectivity Based Histograms

13. Multiresolution Analysis • Shapes can be corrupted by noise • Require multiresolution analysis

14. Multiresolution Analysis • Multiple resolutions of each image is obtained using Gaussian filters. • Each resolution of the image is decomposed by connectivity. The family of 3d-histograms is obtained for each image. • The process of obtaining histograms is preceded by pixel intensity thresholding. Pixel intensity thresholding is especially useful for fine shapes which would otherwise have significant rates of change in histogram densities. • Principal components are obtained for the histogram family and used as an index for each image. • During querying, the similarity between two images is determined by the L2-difference of their principal components

15. Multiresolution Images Multiple resolutions are obtained by using Gaussian filters. Family of histograms is obtained for multiple resolutions of an image.

16. Multiresolution Distance Histogram It is observed that the rates of change of histograms is high for fine edges. Hence, it is important to use pixel intensity thresholding.

17. PCA • Principal Component Analysis (PCA) is a statistical approach for reducing the dimensionality of data. We apply PCA to feature vectors and the resulting low dimensional data is used to index images. • Singular value decomposition for the data matrix A is computed as U S V'. • Where V is consisted of the eigenvectors of A'xA and U is consisted of the eigenvectors of AxA'. • The eigenvalues obtained from the products of AxA' and A'xA, when square-rooted, make up the columns of S. The diagonal of S is said to be the singular values of the original matrix, A. • Each eigenvector described above represents a principle component.

18. PCA The first eigenvector (P1) has the largest eigenvalues to the direction of the largest variance. The second (P2) and the third (P3) eigenvectors are orthogonal to the first one. In the example shown, the first eigenvalue for the first eigenvector is 1 = 81.65. The other eigenvalues are 2 = 5.3 and 3 = 1.54. Thus, the first eigenvector contains almost all the energy. The data could thus be approximated in one-dimension.

19. Experiment • Dataset is MPEG-7. It consists of • Traditional approach – 2d histograms are obtained for each image • Proposed approach – 3d histograms are obtained for each image • Principal components are obtained for the histogram family and used as an index for each image. • During querying, the similarity between two images is determined by the L2-difference of their principal components

20. Results

21. Conclusions • A set of new region based shape feature is proposed based on coherence distance histogram • Multiresolution image analysis and PCA feature analysis have been used to obtain the accurate and compact shape features • Experiment results show the new shape features outperforms the traditional distance histogram approach • In future, the technique will be applied to gray level or color images