Content-Based Image Retrieval Using Multiresolution Color and Texture Features

1. Young Deok Chun, Nam Chul Kim, Member, IEEE, and IckHoon Jang, Member, IEEE IEEE TRANSACTIONS ON MULTIMEDIA,OCTOBER 2008 Content-Based Image Retrieval Using Multiresolution Color and Texture Features

2. Outline • Introduction • Conventional features • Proposed image retrieval method • Experimental results • Conclusion

3. Introduction • Typical CBIR • Extract features related to visual content from a query image • Compute similarity between the features of the query image and target images in DB • Target images are next retrieved which are most similar to the query image • Extraction of good features is one of the important tasks in CBIR • Shape • Describes the contours of objects in an image • Usually extracted from segmenting the image into meaningful regions or objects

4. Introduction • Color • Most widely used visual features • Invariant to image size and orientation • Texture • Visual feature that refers to innate surface properties of an object • Relationship to the surrounding environment • A feature extracted from an image is generally represented as a vector of finite dimension • Feature vector dimension is one of the most important factors that determine • The amount of storage space for the vector • Retrieval accuracy • Retrieval time (or computational complexity)

5. Conventional features • Explain the conventional features which are used in the proposed retrieval method • color feature • color autocorrelogram • texture features • BDIP • BVLC

6. Color Autocorrelogram • Color autocorrelogram • Captures the spatial correlation between identical colors only • Probability of finding a pixel p’ of the identical color at a distance k from a given pixel p of the lth color • measure the distance between pixels • As k varies, spatial correlation between identical colors in an image can be obtained in various resolutions

7. BDIP • Block Difference of Inverse Probabilities • Texture feature that effectively extracts edges and valleys • Object boundaries are extracted well , andthe intensity variation in dark regions is emphasized Representative (maximum) of intensity variation in a block Intensity of pixel(x,y) Block size (k+1)*(k+1) Representative value in a block Original image BDIPoperator

8. BVLC • Block Variation of Local Correlation coefficients • Represents the variation of block-based local correlation coefficients according to four orientations • Local correlation • BVLC Local covariance normalized by local variance standard deviation of the lth block shifted by k Difference between the max and min values of block-based local correlation coefficients (four orientations)

9. Proposed image retrieval method 4-band wavelet decomposition • Overall structure A color feature vector fc of dimensionNcis then formed with color autocorrelograms extracted from the and Each component image is wavelet decomposed into a wavelet image A texture feature vector ft of dimension Ntwith BDIP and BVLC moments extracted from configuration of 2-level wavelet decomposed images

10. Color Feature Extraction

11. Color Feature Extraction • The and are first quantized into and • LL band • uniformly quantized • other subbands • non-uniformly quantized by the generalized Lloyd algorithm • Given the total number of quantization levels L of all subbands, Li is allocated to the ithsubband

12. Color Feature Extraction • Numbers of quantization levels for all subbands are decided • Color autocorrelogram is extracted • To reduce computational complexity, we modify the color autocorrelogram the set of pixels having the lth color

13. Color Feature Extraction • The color feature vector fcis finally formed with the color autocorrelogram • color autocorrelogram • probability of finding a pixel p’ of the lth color among the two causal neighboring pixels at a distance k from a given pixel p of the lth color in the

14. Texture Feature Extraction

15. Texture Feature Extraction • is first divided into nonoverlapping blocks of a given size, where the BDIP and BVLC are computed • BDIP • the denominator in may yield negative BDIP values in wavelet domain, which leads to invalid measurement of intensity variation Pixel values are nonnegative in LL band

16. Texture Feature Extraction • BVLC • Reduce computational complexity • Local covariance • Mean absolute difference of pixels between two blocks • Local variance • Mean absolute difference of four end pixels in the block

17. Texture Feature Extraction • Modified BVLC • Difference between the max and min values of the local correlation coefficients according to two orientations

18. Texture Feature Extraction • The first and second moments of the BDIP and BVLC for each subband are extracted • The texture feature vector

19. Feature Vector Combination and Similarity Measurement

20. Feature Vector Combination • After color and texture feature vectors are extracted, the retrieval system combines these feature vectors • Each of the color and texture feature components is normalized by its dimension and standard deviation • reducing the effect of different feature vector dimensions and component variances in the similarity computation

21. Similarity Measurement • Use generalized Minkowski-form distance of metric order one • Feature dimension • Color feature • NC : determined as the total number of quantization levels L • Texture feature • NT: 16Z • N= NC + NT • The number of additions for a query image in the similarity measurement of the retrieval is given as

22. Similarity Measurement • Huge Database • Progressively implement • reduce the computational complexity in CBIR • Set of candidate images is selected by feature matching at the lowest level • Progressive refinement is performed as the level increases • Progressive retrieval is composed of Z+1 steps • First step • the color feature vector fC and the texture feature vector fT are combined for m=LL and n=1 • Second step • m={LL,HL,LH,HH} and n=1 • Z+1 step • m={LL,HL,LH,HH} and n={1,….,Z}

23. Similarity Measurement • At each step (jth step) • query • the combined feature vector fjq of dimension Nj • Target • the combined feature vector fjt of the same dimension • for each of kj-1 target images • kj target images with the best similarity are retrieved • Total number of additions for a query in the similarity measurement of the progressive retrieval • where kj is determined to decrease and the Nj increases as the retrieval step j increases

24. EXPERIMENTAL RESULTS • Image Database • The Corel DB • 990 RGB color images • 192 x 128 pixels • 11 classes, each of 90 images • VisTex DB • 1200 RGB color images • 128 x 128 pixls • 75 classes, each of 16 images • MPEG-7 common color dataset (CCD) • 5420 color images • 332 ground truth sets (GTS) where the number of images in each GTS varies

25. EXPERIMENTAL RESULTS • Corel MR DB • directly from a third of all the images for each class in the Corel DB • Ratio of (1.5:1) • Ratio of (2:1) • VisTex MR DB • Ratio of (1:1),(1.5:1),(1.75:1),(2:1) • MPEG-7 CCD MR DB • Ratio of (1:1),(1.5:1),(2:1) • The sizes and numbers of classes of the three derived DBs are the same as those of the original DBs • Contain images of various resolutions

26. EXPERIMENTAL RESULTS • Performance Measures • For a query q A(q) : a set of retrieved images in a DB B(q) : images relevant to the query q • Precision • Recall • ANMRR (average normalized modified retrieval rank) • measure of retrieval accuracy used in almost all of the MPEG-7 color core experiments • The ANMRR gives just one value for a DB • Lower ANMRR value means more accurate retrieval performance

27. EXPERIMENTAL RESULTS • Specifications of Retrieval Methods • wavelet decomposition level was chosen as Z=2 • total number of quantization levels L for each of the wavelet decomposed H and S component images was also chosen as L=30 • {Lm,1} = {8,4,4,4} , {Lm,2} = {4,2,2,2} • Dimensions of feature vector • NC= 60 • NT = 32 • Z=2 , Proposed progressive retrieval has 3 steps • At the step of j=1,2,3 , the total number of quantization levels for each of the wavelet decomposed H and S component images is given as L = 8,20, and 30 • feature vector dimensions • (N1,N2,N3) =(20,56,92)

28. EXPERIMENTAL RESULTS Proposed retrieval method with progressive scheme is about 1.2 times faster than nonprogressive scheme • The proposed method with nonprogressive scheme and that with progressive scheme (a) Corel DB (b) VisTex DB. former yields the average precision loss of 1.5% and that of 1.1% over the latter for Corel DB and for VisTex DB

29. EXPERIMENTAL RESULTS • Precision versus recall of the proposed method with progressive scheme according to each step (a) Corel DB (b) VisTex DB.

30. EXPERIMENTAL RESULTS • Single features and the proposed progressive retrieval method • Combination of color and texture features and the proposed progressive retrieval method

31. EXPERIMENTAL RESULTS • ANMRR of the retrieval methods The proposed method almost always yields better performance in precision versus recall and in ANMRR over the other methods for the six test DBs

32. EXPERIMENTAL RESULTS • Retrieval ranks of the relevant images for the query image Resolution is identical with the query image Query image The proposed method is more effective for multiresolution image DBs

33. Conclusion • The feature vector is scalable according to the decomposition level Z in the wavelet transform domain • It was found in some experiments that the retrieval accuracies of Z>2 are slightly better than those of Z=2 • Experimental results for six test DBs showed that the proposed method yielded higher retrieval accuracy than the other conventional methods • It was all the more so for multiresolution image Databases