An unsupervised learning approach to content based image retrieval
Download
1 / 29

An Unsupervised Learning Approach to Content-Based Image Retrieval - PowerPoint PPT Presentation


  • 157 Views
  • Uploaded on

An Unsupervised Learning Approach to Content-Based Image Retrieval. Yixin Chen & James Z. Wang The Pennsylvania State University. Outline. Introduction Cluster-based retrieval of images Experiments Conclusions and future work. Image Retrieval. The driving forces Internet

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' An Unsupervised Learning Approach to Content-Based Image Retrieval' - ull


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
An unsupervised learning approach to content based image retrieval

An Unsupervised Learning Approach to Content-Based Image Retrieval

Yixin Chen & James Z. Wang

The Pennsylvania State University

IEEE Int'l Symposium on Signal Processing and its Applications


Outline
Outline Retrieval

  • Introduction

  • Cluster-based retrieval of images

  • Experiments

  • Conclusions and future work

IEEE Int'l Symposium on Signal Processing and its Applications


Image retrieval
Image Retrieval Retrieval

  • The driving forces

    • Internet

    • Storage devices

    • Computing power

  • Two approaches

    • Text-based approach

    • Content-based approach

IEEE Int'l Symposium on Signal Processing and its Applications


Text based approach
Text-Based Approach Retrieval

  • Input keywords descriptions

Text-Based

Image

Retrieval

System

Elephants

Image

Database

IEEE Int'l Symposium on Signal Processing and its Applications


Text based approach1
Text-Based Approach Retrieval

  • Index images using keywords

    (Google, Lycos, etc.)

    • Easy to implement

    • Fast retrieval

    • Web image search (surrounding text)

    • Manual annotation is not always available

    • A picture is worth a thousand words

    • Surrounding text may not describe the image

IEEE Int'l Symposium on Signal Processing and its Applications


Content based approach
Content-Based Approach Retrieval

  • Index images using low-level features

CBIR

System

Image

Database

Content-based image retrieval (CBIR): search pictures as pictures

IEEE Int'l Symposium on Signal Processing and its Applications


A data flow diagram
A Data-Flow Diagram Retrieval

Histogram, color

layout, sub-images,

regions, etc.

Image

Database

Feature

Extraction

Linear ordering,

Projection to 2-D,

etc.

Euclidean distance,

intersection, shape

comparison, region

matching, etc.

Visualization

Compute

Similarity

Measure

IEEE Int'l Symposium on Signal Processing and its Applications


Open problem
Open Problem Retrieval

  • Nature of digital images: arrays of numbers

  • Descriptions of images: high-level concepts

    • Sunset, mountain, dogs, ……

  • Semantic gap

    • Discrepancy between low-level features and high-level concepts

    • High feature similarity may not always correspond to semantic similarity

IEEE Int'l Symposium on Signal Processing and its Applications


Outline1
Outline Retrieval

  • Introduction

  • Cluster-based retrieval of images

  • Experiments

  • Conclusions and future work

IEEE Int'l Symposium on Signal Processing and its Applications


Motivation
Motivation Retrieval

  • A query image and its top 29 matches returned by a CBIR system

    Horses (11 out of 29), flowers (7 out of 29), golf player (4 out of 29)

IEEE Int'l Symposium on Signal Processing and its Applications


Clue clusters based retrieval of images by unsupervised learning
CLUE: CLUsters-based rEtrieval of images by unsupervised learning

  • Hypothesis

    In the “vicinity” of a query image, images tend to be semantically clustered

  • CLUE attempts to capture high-level semantic concepts by learning the way that images of the same semantics are similar

IEEE Int'l Symposium on Signal Processing and its Applications


System overview
System Overview learning

A general diagram of a CBIR system using CLUE

CLUE

Image

Database

Select

Neighboring

Images

Image

Clustering

Feature

Extraction

Display

And

Feedback

Compute

Similarity

Measure

IEEE Int'l Symposium on Signal Processing and its Applications


Neighboring images selection
Neighboring Images Selection learning

  • Nearest neighbors method

    • Pick k nearest neighbors of the query as seeds

    • Find r nearest neighbors for each seed

    • Take all distinct images as neighboring images

k=3, r=4

IEEE Int'l Symposium on Signal Processing and its Applications


Weighted graph representation
Weighted Graph Representation learning

  • Graph representation

    • Vertices denote images

    • Edges are formed between vertices

    • Nonnegative weight of an edge indicates the similarity between two vertices

IEEE Int'l Symposium on Signal Processing and its Applications


Clustering
Clustering learning

  • Graph partitioning and cut

  • Normalized cut (Ncut) [Shi et al., IEEE Trans. PAMI 22(8)]

  • Recursive Ncut

IEEE Int'l Symposium on Signal Processing and its Applications


Outline2
Outline learning

  • Introduction

  • Cluster-based retrieval of images

  • Experiments

  • Conclusions and future work

IEEE Int'l Symposium on Signal Processing and its Applications


An experimental system
An Experimental System learning

  • Similarity measure

    • UFM[Chen et al. IEEE PAMI 24(9)]

  • Database

    • COREL

    • 60,000

IEEE Int'l Symposium on Signal Processing and its Applications


User interface
User Interface learning

IEEE Int'l Symposium on Signal Processing and its Applications


Query examples
Query Examples learning

  • Query Examples from 60,000-image COREL Database

    Bird, car, food, historical buildings, and soccer game

UFM

CLUE

Bird, 6 out of 11

Bird, 3 out of 11

IEEE Int'l Symposium on Signal Processing and its Applications


Query examples1
Query Examples learning

CLUE

UFM

Car, 8 out of 11

Car, 4 out of 11

Food, 8 out of 11

Food, 4 out of 11

IEEE Int'l Symposium on Signal Processing and its Applications


Query examples2
Query Examples learning

CLUE

UFM

Historical buildings, 10 out of 11

Historical buildings, 8 out of 11

Soccer game, 10 out of 11

Soccer game, 4 out of 11

IEEE Int'l Symposium on Signal Processing and its Applications


Clustering www images
Clustering WWW Images learning

  • Google Image Search

    • Keywords: tiger, Beijing

    • Top 200 returns

    • 4 largest clusters

    • Top 18 images within each cluster

IEEE Int'l Symposium on Signal Processing and its Applications


Clustering www images1
Clustering WWW Images learning

Tiger Cluster 1 (75 images)

Tiger Cluster 2 (64 images)

Tiger Cluster 3 (32 images)

Tiger Cluster 4 (24 images)

IEEE Int'l Symposium on Signal Processing and its Applications


Clustering www images2
Clustering WWW Images learning

Beijing Cluster 1 (61 images)

Beijing Cluster 2 (59 images)

Beijing Cluster 3 (43 images)

Beijing Cluster 4 (31 images)

IEEE Int'l Symposium on Signal Processing and its Applications


Retrieval accuracy
Retrieval Accuracy learning

IEEE Int'l Symposium on Signal Processing and its Applications


Outline3
Outline learning

  • Introduction

  • Cluster-based retrieval of images

  • Experiments

  • Conclusions and future work

IEEE Int'l Symposium on Signal Processing and its Applications


Conclusions
Conclusions learning

  • Retrieving image clusters by unsupervised learning

  • Tested using 60,000 images from COREL and images from WWW

IEEE Int'l Symposium on Signal Processing and its Applications


Future work
Future Work learning

  • Recursive Ncut

  • Representative image

  • Other graph theoretic clustering techniques

  • Nonlinear dimensionality reduction

IEEE Int'l Symposium on Signal Processing and its Applications


Thank you
Thank You! learning

IEEE Int'l Symposium on Signal Processing and its Applications


ad