outline
Download
Skip this Video
Download Presentation
Outline

Loading in 2 Seconds...

play fullscreen
1 / 37

Outline - PowerPoint PPT Presentation


  • 116 Views
  • Uploaded on

Outline. Content-Based Image Retrieval Query-by-Example Query-by-Feature Feature Vector. CBIR and CBR. Content-based Image Retrieval (CBIR) como exemplo de Content-based Retrieval (CBR) concentra em low-level features. Principais id é ias de CBIR:

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Outline' - hija


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
outline
Outline
  • Content-Based Image Retrieval
    • Query-by-Example
    • Query-by-Feature
    • Feature Vector
cbir and cbr
CBIR and CBR
  • Content-based Image Retrieval (CBIR) como exemplo de Content-based Retrieval (CBR)
  • concentra em low-level features.
  • Principais idéias de CBIR:
    • Representar uma imagem como um conjunto de feature descriptors.
    • Definir medidas de similaridade dos descritores
    • Quando um usuário especificar uma query, o sistema retorna imagens, que são ordenadas por similaridade.
slide4
Image Retrieval

Database Images

Query Image

Image Database

Feature

Extraction

Feature

Extraction

Select

Compare

Metadatabase

Feature Vectors

Query Result

cbir de butterflies
CBIR de Butterflies
  • Permitir non-expert users encontrar algumas espécies de butterflies usando informações de aparência de butterflies
  • Aparência:
    • Color, Texture, Shape
problemas
Problemas
  • Como podemos descrever uma butterfly?
  • Como podemos comunicar nossa descrição para uma máquina?
problemas1
Problemas
  • Usuários diferentes têm percepções diferentes.
  • Usuários podem não se lembrar claramente a aparêcia de uma butterfly.
  • Usuários normalmente não têm expertise para descrever butterflies.
  • Usuários normalmente não têm paciência para fazer o browse num grande conjunto resultado.
solu es
Soluções
  • Usar um processo de consulta interativo e direcionado ao usuário: QBF/QBE query process
    • Query By Features e Query By Example
  • Fuzzy feature description para cada butterfly
  • Uma “What You See Is What You Get” query interface
  • Um conjunto representativo de coleção butterflies
qbf qbe query process 1
QBF/QBE query process (1)
  • QBF query:
    • A QBF query is to choose some features of butterflies and expect that the system returns all butterflies with those features.
    • Features of butterflies:
      • Dominant color, texture pattern, shape.
  • QBE query:
    • A QBE query is to point an image and expect that the system returns all butterflies similar to that.
qbf qbe query process 2
QBF/QBE query process (2)
  • Properties of QBF:
    • Rough search
    • When to use:
      • The first query and when users want to enlarge the view in the search space
  • Properties of QBE:
    • Fine search
    • When to use:
      • Usually the last query and when users want to see the neighbors of the query one in the search space.
qbf qbe query process 3
QBF/QBE query process (3)
  • Result page: (Each result page should contain two parts)
    • Result Images:
      • These are the butterfly images satisfy the query conditions.
      • Users can invoke QBE queries from these images.
    • Related Features:
      • These are the features related to the previous query conditions.
      • Users can invoke QBF queries from these features.
feature description 1
Feature Description (1)
  • Feature Description for a butterfly:
    • Like metadata which describe the appearance of this butterfly.
    • This makes QBF queries possible.
    • Feature Description consists of some feature descriptors.
  • Feature descriptor:
    • A ( “feature value” , “match level” ) pair.
feature description 6
Feature Description (6)
  • QBF query:
    • Single feature query:
      • Result images: images with its corresponding degree of match > 0.
      • Ranked by: degree of match.
      • We call this ranked sequence “Feature sequence.”
    • Multiple features query:
      • Merge the corresponding feature sequences.
result presentation
Result Presentation
  • For QBF query:
    • Property: rough search
    • Presentation: representative butterflies only
  • For QBE query:
    • Property: fine search
    • Presentation:
      • For very similar images: present them all
      • For less similar images: representative ones
feature vector indexing
Feature Vector Indexing
  • Goal:
    • To make search efficiently.
  • Problems of Indexing in CBIR:
    • Dimension of feature space is very high.
    • Index structure should support Euclidean and non-Euclidean similarity measures.
  • Solution:
    • Dimension reduction: KLT, DCT, DWT.
    • Similarity indexing: R*-tree, SS-tree, SR-tree.
semi automatic feature extraction
Semi-Automatic Feature Extraction
  • Segmentation:
    • Background segmentation
    • Butterfly object segmentation
  • Feature extraction:
    • Color: color histogram
    • Texture: manual annotation
    • Shape: manual annotation
classic cbir with color feature
Classic CBIR with Color Feature
  • Most of the CBR systems rely on the notion of color, this may differ:
    • Dominant color
    • Scalable color based on color histograms (local for one region, global for the whole image)
    • Color Structure Descriptor (incoporates the spatial structure)
what color is the apple we are so visual
What color is the apple ? We are so visual !!!!

I’d say it is

Bright Red

I really

couldn’t tell you

(I am color blind)

I think it is

“Crimson”

It is

Red!

color histogram representation
Color Histogram: Representation
  • A list of Color-Percentage pairs:
    • Describe the colors and its percentages in an image.
color quantization
Color Quantization
  • Indexed Colors
  • A jpg Image with 256-color components in each RGB channel
  • 256 x 256 x 256 colors in total → n groups, e.g, in 256 groups, that makes a reduction 256x256, I.e., that each group takes 256 colors to count.
similarity measures overview
Similarity Measures - Overview
  • Minkowski Similarity
    • Distance L1 : r = 1
    • Distance L2 : r = 2
  • Quadratic Similarity
  • Intersection Similarity

(Swain et Ballard 1991)

example cont
Example (cont.)
  • Minkowski Similarity
    • Is a L-1 metric

where Ik and Jk is the number of pixels in bin k for image I and J

    • Distance between above three images
      • D(H1, H2) = 1 + 1 + 1 + 1 + 1 + 1 + 1 + 1 = 8
      • D(H1, H3) = 6 + 6 + 2 + 2 + 2 + 2 + 2 + 2 = 24
      • D(H2, H3) = 5 + 5 + 3 + 3 + 1 + 1 + 1 + 1 = 23
example cont1
Example (cont.)
  • Minkowski Similarity
    • Is a L-2 metric
    • Distance between above three images
      • D(H1, H2) = (1 + 1 + 1 + 1 + 1 + 1 + 1 + 1)1/2 = 2.8
      • D(H1, H3) = (36 + 36 + 4 + 4 + 4 + 4 + 4 + 4)1/2 = 9.8
      • D(H2, H3) = (25 + 25 + 9 + 9 + 1 + 1 + 1 + 1) 1/2 = 8.5
qbic distance
QBIC distance
  • Weighted Euclidean distance (QBIC)
    • Is a L-2 metric(?)

distance between histogram H1 and H2:

D = (H1 - H2)T A (H1 - H2)

where A is a symmetric color similarity matrix

A (i, j) = 1 –d (ci, cj) / dmax

where ci and cj are the i-th and j-th color bins,

d (ci , cj) is the color distance in the color space, and dmax is the maximum distance between any two colors in the color space

limitation
Limitation
  • Ignore similarity between colors
    • Example
      • Two color bins
        • Bin-1 color range: 1 – 10
        • Bin-2 color range: 11 – 20
slide30
Three color pixels
      • Pixel 1 is Color 10  Bin-1
      • Pixel 2 is Color 11  Bin-2
      • Pixel 3 is Color 20  Bin-2
  • Pixel 2 is similar to Pixel 3 than Pixel 1  unreasonable !
limitation cont
Limitation (cont.)
  • Ignore spatial relationships among pixels

Different image with same histogram

slide32
Noise-Free Queries (NFQ’s)
  • NFQ is more precise.
  • User can specify semantic constraints:
    • Spatial constraints (relative distances)
    • Scaling constraints (relative sizes)

Rectangular query

Noise-free

query

Similar

Less relevant

slide33
Challenges
  • How do we extract features if we do not know the matching areas beforehand ?
  • How do we index the images ?

Noise-free

query

slide34
One Solution – Local Color Histogram (LCH)
  • Each subimage has a color histogram.
  • Any combination of the histograms can be selected for comparison with the corresponding color histograms of the query image.
slide35
Limitations of LCH
  • Dilemma:
    • Using large partitions is not precise
    • Using small partitions is too expensive
  • Limitation:
    • difficult to handle scaling
demos
DEMOS
  • Hermitage Museum Web Site (QBIC)

http://hermitagemuseum.org/

http://hermitagemuseum.org/fcgi-bin/db2www/qbicColor.mac/qbic?selLang=English

  • http://www.aa-lab.cs.uu.nl/cbirsurvey/cbir-survey/
ad