General Image Retrieval Using Shape and Combined Features

1. General Image Retrieval Using Shape and Combined Features Dengsheng Zhang and Guojun Lu Gippsland School of Computing and Information Technology Monash University, Australia

2. Outline • CBIR—Content-based Image Retrieval • Shape Feature—Generic Fourier Descriptor • Texture Feature—Gabor Filter • Semi-automatic Segmentation and Indexing • Experimental Results • Conclusions and Future Work

3. Content-based Image Retrieval • General CBIR Problem: given a query image, find similar images from database. • General Methodology of CBIR: • Finding effective features to represent images • Index the images in DB with the extracted features using certain type of data structure • Matching the query submitted by user with images in DB using certain type of distance measure and Interface • Our focus: finding effective perceptual features

4. Content-based Image Retrieval • Represent images with content features • Color: RGB, HSV, LUV • Shape: moments, Fourier descriptors, scale space method • Texture: statistic method, fractal method, spectral method • Represent images with combined features • Combine several content features to represent images more effectively than individual features

5. Generic Fourier Descriptor (GFD) • Basically, GFD is acquired from 2D Fourier transform on a polar-raster sampled shape image.

6. Generic Fourier Descriptor (GFD) • After applying the 2D FT on the polar-raster sampled image, a set of transformed coefficients are obtained, which is used as the feature vector where m is the maximum number of the radial frequencies selected and n is the maximum number of angular frequencies selected.

7. Gabor Filter • Basically, Gabor filters are a group of wavelets, with each wavelet capturing energy at a specific frequency and a specific direction. • For a given image I(x, y) with size PQ, its discrete Gabor wavelet transform is given by a convolution: where Extracted Energy: Extracted feature: f = (00, 00 , 01 , 01 , …, 45, 45)

8. Semi-automatic Segmentation and Indexing

9. Experiments • A database of 1,000 images from over 40 varieties is created from a classified collection of 10,000 natural pictures. The types of images include land animals, marine animals, flying animals, buildings, aircrafts, flowers and other real world objects. • Each image is segmented and indexed using the semi-automatic segmentation tool. For each image, up to 5 objects are allowed to index the image. The similarity between the query image and the target image is measured by the similarity between the two most similar objects in the two images.

10. Performance Measurement • Precision and Recall

11. Distance Measurement • The similarity between two images are measured by the city block distance between the two feature vectors of the images. • For the combined retrieval, assuming the rank of an image using shape retrieval is r1 and the rank of the image using texture retrieval is r2, then the rank of the image using combined retrieval is given by (r1+r2)/2.

12. Performance Results

13. Retrieval Using Shape

14. Retrieval Using Combined Features Using shape Using combined

15. Conclusions • A general image retrieval technique using shape and combined features has been presented. • A semi-automatic object segmentation and indexing method has been presented. • On average, the retrieval effectiveness of shape is comparable with that of texture. • Retrieval using combined shape and texture features is more powerful than retrieval using individual features. • Combined features should be an option rather than replacement of individual features. • In the future, we plan to segment image automatically into homogenous texture regions using split and merging technique, so that indexing of images using shape and texture features can be done automatically.