1 / 41

Image Retrieval: Current Techniques, Promising Directions, and Open Issues

Image Retrieval: Current Techniques, Promising Directions, and Open Issues. Yong Rui, Thomas Huang and Shih-Fu Chang Published in the Journal of Visual Communication and Image Representation. Presented by: Deepak Bote. Presentation Outline. History of image retrieval – Issues faced

ulla
Download Presentation

Image Retrieval: Current Techniques, Promising Directions, and Open Issues

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Image Retrieval: Current Techniques, Promising Directions, and Open Issues Yong Rui, Thomas Huang and Shih-Fu Chang Published in the Journal of Visual Communication and Image Representation. Presented by: Deepak Bote

  2. Presentation Outline • History of image retrieval – Issues faced • Solution – Content-based image retrieval • Feature extraction • Multidimensional indexing • Current Systems • Open issues • Conclusion

  3. History of Image Retrieval • Traditional text-based image search engines • Manual annotation of images • Use text-based retrieval methods • E.g. Water lilies Flowers in a pond <Its biological name>

  4. Limitations of text-based approach • Problem of image annotation • Large volumes of databases • Valid only for one language – with image retrieval this limitation should not exist • Problem of human perception • Subjectivity of human perception • Too much responsibility on the end-user • Problem of deeper (abstract) needs • Queries that cannot be described at all, but tap into the visual features of images.

  5. Outline • History of image retrieval – Issues faced • Solution – Content-based image retrieval • Feature extraction • Multidimensional indexing • Current Systems • Open issues • Conclusion

  6. What is CBIR? • Images have rich content. • This content can be extracted as various content features: • Mean color , Color Histogram etc… • Take the responsibility of forming the query away from the user. • Each image will now be described by its own features.

  7. CBIR – A sample search query • User wants to search for, say, many rose images • He submits an existing rose picture as query. • He submits his own sketch of rose as query. • The system will extract image features for this query. • It will compare these features with that of other images in a database. • Relevant results will be displayed to the user.

  8. Sample Query

  9. Sample CBIR architecture

  10. Outline • History of image retrieval – Issues faced • Solution – Content-based image retrieval • Feature extraction • Multidimensional indexing • Current Systems • Open issues • Conclusion

  11. Feature Extraction • What are image features? • Primitive features • Mean color (RGB) • Color Histogram • Semantic features • Color Layout, texture etc… • Domain specific features • Face recognition, fingerprint matching etc… General features

  12. Mean Color • Pixel Color Information: R, G, B • Mean component (R,G or B)= Sum of that component for all pixels Number of pixels Pixel

  13. Histogram • Frequency count of each individual color • Most commonly used color feature representation Corresponding histogram Image

  14. Color Layout • Need for Color Layout • Global color features give too many false positives • How it works: • Divide whole image into sub-blocks • Extract features from each sub-block • Can we go one step further? • Divide into regions based on color feature concentration • This process is called segmentation.

  15. Example: Color layout ** Image adapted from Smith and Chang : Single Color Extraction and Image Query

  16. Texture • Texture – innate property of all surfaces • Clouds, trees, bricks, hair etc… • Refers to visual patterns of homogeneity • Does not result from presence of single color • Most accepted classification of textures based on psychology studies – Tamura representation • Coarseness • Contrast • Directionality • Linelikeness • Regularity • Roughness

  17. Segmentation issues • Considered as a difficult problem • Not reliable • Segments regions, but not objects • Different requirements from segmentation: • Shape extraction: High Accuracy required • Layout features: Coarse segmentation may be enough

  18. Presentation Outline • History of image retrieval – Issues faced • Solution – Content-based image retrieval • Feature extraction • Multidimensional indexing • Current Systems • Open issues • Conclusion

  19. Problem of high dimensions • Mean Color = RGB = 3 dimensional vector • Color Histogram = 256 dimensions • Effective storage and speedy retrieval needed • Traditional data-structures not sufficient • R-trees, SR-Trees etc…

  20. 2-dimensional space Point A D2 D1

  21. 3-dimensional space

  22. Now, imagine… • An N-dimensional box!! • We want to conduct a nearest neighbor query. • R-trees are designed for speedy retrieval of results for such purposes • Designed by Guttmann in 1984

  23. Presentation Outline • History of image retrieval – Issues faced • Solution – Content-based image retrieval • Feature extraction • Multidimensional indexing • Current Systems • Open issues • Conclusion

  24. IBM’s QBIC • QBIC – Query by Image Content • First commercial CBIR system. • Model system – influenced many others. • Uses color, texture, shape features • Text-based search can also be combined. • Uses R*-trees for indexing

  25. QBIC – Search by color ** Images courtesy : Yong Rao

  26. QBIC – Search by shape ** Images courtesy : Yong Rao

  27. QBIC – Query by sketch ** Images courtesy : Yong Rao

  28. Virage • Developed by Virage inc. • Like QBIC, supports queries based on color, layout, texture • Supports arbitrary combinations of these features with weights attached to each • This gives users more control over the search process

  29. VisualSEEk • Research prototype – University of Columbia • Mainly different because it considers spatial relationships between objects. • Global features like mean color, color histogram can give many false positives • Matching spatial relationships between objects and visual features together result in a powerful search.

  30. ISearch – my own system

  31. ISearch – my own system

  32. ISearch – my own system

  33. Feature selection in ISearch

  34. Database Admin facility in ISearch

  35. Presentation Outline • History of image retrieval – Issues faced • Solution – Content-based image retrieval • Feature extraction • Multidimensional indexing • Current Systems • Open issues • Conclusion

  36. Open issues • Gap between low level features and high-level concepts • Human in the loop – interactive systems • Retrieval speed – most research prototypes can handle only a few thousand images. • A reliable test-bed and measurement criterion, please!

  37. Presentation Outline • History of image retrieval – Issues faced • Solution – Content-based image retrieval • Feature extraction • Multidimensional indexing • Current Systems • Open issues • Conclusion

  38. Conclusion • Satisfactory progress, but still… A long way to go…!!

  39. Acknowledgements • Dr. Padma Mundur • Mr. Yong Rao • Mr. Sumit Jain, Software Engineer, KPIT Cummins, India • Mr. Ajay Joglekar, Software Engineer, Veritas India.

  40. References • Y. Rui, T. S. Huang, and S.-F. Chang, “Image retrieval: Current techniques, promising directions, and open issues” • S. Jain, A. Joglekar, and D. Bote, ISearch: A Content-based Image Retrieval (CBIR) Engine, as Bachelor of Computer Engineering final year thesis, Pune University, 2002

  41. THANK YOU!!! Questions?

More Related