1 / 31

Content-based Image Retrieval

Content-based Image Retrieval. Presentation by Charlie Neo. Introduction. Why Digital image database growing rapidly in size Professional needs – Logo Search Difficulty in locating images on the web Example Find a picture of me and jack on the bus. Application CBIR.

Patman
Download Presentation

Content-based Image Retrieval

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Content-based Image Retrieval Presentation by Charlie Neo

  2. Introduction • Why • Digital image database growing rapidly in size • Professional needs – Logo Search • Difficulty in locating images on the web • Example • Find a picture of me and jack on the bus

  3. Application CBIR • Search for one specific image. • General browsing to make an interactive choice. • Search for a picture to go with a broad story or search to illustrate a document. • Search base on the esthetic value of the picture.

  4. Two Classes of CBIRNarrow vs. Broad Domain • Narrow • Medical Imagery Retrieval • Finger Print Retrieval • Satellite Imagery Retrieval • Broad • Photo Collections • Internet

  5. Challenges • Semantic gap • The semantic gap is the lack of coincidence between the information that one can extract from the visual data and the interpretation that the same data have for a user in a given situation. • User seeks semantic similarity, but the database can only provide similarity by data processing. • Huge amount of objects to search among. • Incomplete query specification. • Incomplete image description.

  6. Description Of Content: Image Processing • Color • Local Shape • Texture

  7. Color Image Processing • Problems with color variances • Surface Orientation • Camera Viewpoint • Position of Illumination • Intensity of the Light

  8. Color Image Processing • Approaches • Opponent color axes • Advantage of isolating the brightness information on the third axis. • Invariant to changes in illumination intensity and shadows. • HSV-representation • Invariant under the orientation of the object with respect to the illumination and camera direction. • Search for clusters in a color histogram to identify which pixels in the image originate from one uniformly colored object.

  9. Image Processing for Local Shape • Problems • Occlusion • Different Viewpoint • Approaches • Collect all properties that capture geometric details in the image. • Invariant Descriptors.

  10. Image Texture Processing • Problems • Offer little semantic referent. • Approaches • Markovian analysis • Wavelets • Generated by groups of dilations or dilations and rotations • Some semantic correspondent. • Great For • Satellite images • Images of documents

  11. Description Of Content using Features • Grouping Data • Strong Segmentation • Region T = 0 (object) • Shape and Object features • Weak Segmentation • T subset of 0 • Salient features • Sign Detection • Signs Probabilities • Partitioning • Global feature

  12. Global and Accumulating Features • Accumulation of features using a histogram • The set of features F(m) ordered by histogram index m. • 64-bin histogram, has discriminating power up to 25,000 images

  13. Salient Features • Weak segmentation. (grouping of data into homogeneous regions.) • Salient feature calculations lead to sets of regions or points with known location and feature values. • Innovation of content-based retrieval. • Expected to receive much attention in the further expansion of the field.

  14. Signs • Typical signs are an icon, a character, a traffic light, or a trademark. • Strong semantic interpretation is within grasp • Analysis tends to become application-oriented.

  15. Shape and Object Features • Object segmentation is hard. • Possible for narrow domain. • Fortunately, for our purpose it only requires detection of the object’s presents.

  16. Description of Structure and Lay-Out • Structural feature • Feature values • Relationships between object sets. • captured in a graph • Lay-out descriptions • characterized by locations, size, and features.

  17. Interpretation And Similarity • Semantic Interpretation • Derive interpretation from feature set. • Features generate a probability distribution. • MAVIS2-system: four semantic layers. • Similarity • Similarity measure Sq,d between the images q and d • Sq,d = s(Fq,Fd). • s(Fq,Fd) = g( d(Fq,Fd) ) • e.g. dcan be just the Euclidean distance.

  18. User Interaction • Query Space: Definition and Initialization • Q = {IQ,FQ,SQ,ZQ} • IQ is a selection of images from the large image archive I. • FQ is a selection of features from feature set F. • SQ similarity function. • ZQ is a set of labels to capture goal dependent semantics.

  19. Query Specification

  20. Query Space Display • Besides just showing the images that match the query … • Images are placed in such a way that distances between images in the display reflect SQ. • Highlight parts indicating which parts of the image fulfill the criteria. (exact query)

  21. Interacting with Query Space • The process of query specification and display is iterated, where, in each step, the user revises the query. • user feedback leads to an update of query space: • Both positive and negative examples is used. • Each iteration, the probability of being the target for an image in IQ is increased or decreased

  22. SYSTEM ASPECTS:Storage and Indexing • Standard, Linear File System • O(N) • Three classes of indexing methods • Space partitioning • Data partitioning • Distance-based technique • Varies tree structure • O(log N)

  23. SYSTEM ASPECTS:System Architectures • Separate indexing and retrieval. • Image retrieval as a plug-in module to an existing database system. • Analysis, indexing and training as modules

  24. SYSTEM ASPECTS:System Evaluation • Relevance is subjective. • Human subjects to produce idea ordering for a query.

  25. Cortina: A System for large scale, content based Web Image Retrieval • Built for Web Image Retrieval • 3 million images • Clusters • Data Mining for Semantics

  26. Cortina: A System for large scale, content based Web Image Retrieval

  27. Cortina: A System for large scale, content based Web Image Retrieval • Four global feature descriptor • Homogeneous Texture Descriptor • Edge Histogram Descriptor • Scalable Color Descriptor • Dominant Color Descriptor • Linear combination of the 4 features, as the distance for K-NN search.

  28. Cortina: A System for large scale, content based Web Image Retrieval

  29. Conclusion • There is a need for CBIR • Image retrieval does not entail solving the general image understanding problem. It may be sufficient that a retrieval system present similar images, similar in some user-defined sense. • Interaction • The need for database • The semantic gap

More Related