An unsupervised learning approach to content based image retrieval
This presentation is the property of its rightful owner.
Sponsored Links
1 / 29

An Unsupervised Learning Approach to Content-Based Image Retrieval PowerPoint PPT Presentation


  • 96 Views
  • Uploaded on
  • Presentation posted in: General

An Unsupervised Learning Approach to Content-Based Image Retrieval. Yixin Chen & James Z. Wang The Pennsylvania State University. Outline. Introduction Cluster-based retrieval of images Experiments Conclusions and future work. Image Retrieval. The driving forces Internet

Download Presentation

An Unsupervised Learning Approach to Content-Based Image Retrieval

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


An unsupervised learning approach to content based image retrieval

An Unsupervised Learning Approach to Content-Based Image Retrieval

Yixin Chen & James Z. Wang

The Pennsylvania State University

IEEE Int'l Symposium on Signal Processing and its Applications


Outline

Outline

  • Introduction

  • Cluster-based retrieval of images

  • Experiments

  • Conclusions and future work

IEEE Int'l Symposium on Signal Processing and its Applications


Image retrieval

Image Retrieval

  • The driving forces

    • Internet

    • Storage devices

    • Computing power

  • Two approaches

    • Text-based approach

    • Content-based approach

IEEE Int'l Symposium on Signal Processing and its Applications


Text based approach

Text-Based Approach

  • Input keywords descriptions

Text-Based

Image

Retrieval

System

Elephants

Image

Database

IEEE Int'l Symposium on Signal Processing and its Applications


Text based approach1

Text-Based Approach

  • Index images using keywords

    (Google, Lycos, etc.)

    • Easy to implement

    • Fast retrieval

    • Web image search (surrounding text)

    • Manual annotation is not always available

    • A picture is worth a thousand words

    • Surrounding text may not describe the image

IEEE Int'l Symposium on Signal Processing and its Applications


Content based approach

Content-Based Approach

  • Index images using low-level features

CBIR

System

Image

Database

Content-based image retrieval (CBIR): search pictures as pictures

IEEE Int'l Symposium on Signal Processing and its Applications


A data flow diagram

A Data-Flow Diagram

Histogram, color

layout, sub-images,

regions, etc.

Image

Database

Feature

Extraction

Linear ordering,

Projection to 2-D,

etc.

Euclidean distance,

intersection, shape

comparison, region

matching, etc.

Visualization

Compute

Similarity

Measure

IEEE Int'l Symposium on Signal Processing and its Applications


Open problem

Open Problem

  • Nature of digital images: arrays of numbers

  • Descriptions of images: high-level concepts

    • Sunset, mountain, dogs, ……

  • Semantic gap

    • Discrepancy between low-level features and high-level concepts

    • High feature similarity may not always correspond to semantic similarity

IEEE Int'l Symposium on Signal Processing and its Applications


Outline1

Outline

  • Introduction

  • Cluster-based retrieval of images

  • Experiments

  • Conclusions and future work

IEEE Int'l Symposium on Signal Processing and its Applications


Motivation

Motivation

  • A query image and its top 29 matches returned by a CBIR system

    Horses (11 out of 29), flowers (7 out of 29), golf player (4 out of 29)

IEEE Int'l Symposium on Signal Processing and its Applications


Clue clusters based retrieval of images by unsupervised learning

CLUE: CLUsters-based rEtrieval of images by unsupervised learning

  • Hypothesis

    In the “vicinity” of a query image, images tend to be semantically clustered

  • CLUE attempts to capture high-level semantic concepts by learning the way that images of the same semantics are similar

IEEE Int'l Symposium on Signal Processing and its Applications


System overview

System Overview

A general diagram of a CBIR system using CLUE

CLUE

Image

Database

Select

Neighboring

Images

Image

Clustering

Feature

Extraction

Display

And

Feedback

Compute

Similarity

Measure

IEEE Int'l Symposium on Signal Processing and its Applications


Neighboring images selection

Neighboring Images Selection

  • Nearest neighbors method

    • Pick k nearest neighbors of the query as seeds

    • Find r nearest neighbors for each seed

    • Take all distinct images as neighboring images

k=3, r=4

IEEE Int'l Symposium on Signal Processing and its Applications


Weighted graph representation

Weighted Graph Representation

  • Graph representation

    • Vertices denote images

    • Edges are formed between vertices

    • Nonnegative weight of an edge indicates the similarity between two vertices

IEEE Int'l Symposium on Signal Processing and its Applications


Clustering

Clustering

  • Graph partitioning and cut

  • Normalized cut (Ncut) [Shi et al., IEEE Trans. PAMI 22(8)]

  • Recursive Ncut

IEEE Int'l Symposium on Signal Processing and its Applications


Outline2

Outline

  • Introduction

  • Cluster-based retrieval of images

  • Experiments

  • Conclusions and future work

IEEE Int'l Symposium on Signal Processing and its Applications


An experimental system

An Experimental System

  • Similarity measure

    • UFM[Chen et al. IEEE PAMI 24(9)]

  • Database

    • COREL

    • 60,000

IEEE Int'l Symposium on Signal Processing and its Applications


User interface

User Interface

IEEE Int'l Symposium on Signal Processing and its Applications


Query examples

Query Examples

  • Query Examples from 60,000-image COREL Database

    Bird, car, food, historical buildings, and soccer game

UFM

CLUE

Bird, 6 out of 11

Bird, 3 out of 11

IEEE Int'l Symposium on Signal Processing and its Applications


Query examples1

Query Examples

CLUE

UFM

Car, 8 out of 11

Car, 4 out of 11

Food, 8 out of 11

Food, 4 out of 11

IEEE Int'l Symposium on Signal Processing and its Applications


Query examples2

Query Examples

CLUE

UFM

Historical buildings, 10 out of 11

Historical buildings, 8 out of 11

Soccer game, 10 out of 11

Soccer game, 4 out of 11

IEEE Int'l Symposium on Signal Processing and its Applications


Clustering www images

Clustering WWW Images

  • Google Image Search

    • Keywords: tiger, Beijing

    • Top 200 returns

    • 4 largest clusters

    • Top 18 images within each cluster

IEEE Int'l Symposium on Signal Processing and its Applications


Clustering www images1

Clustering WWW Images

Tiger Cluster 1 (75 images)

Tiger Cluster 2 (64 images)

Tiger Cluster 3 (32 images)

Tiger Cluster 4 (24 images)

IEEE Int'l Symposium on Signal Processing and its Applications


Clustering www images2

Clustering WWW Images

Beijing Cluster 1 (61 images)

Beijing Cluster 2 (59 images)

Beijing Cluster 3 (43 images)

Beijing Cluster 4 (31 images)

IEEE Int'l Symposium on Signal Processing and its Applications


Retrieval accuracy

Retrieval Accuracy

IEEE Int'l Symposium on Signal Processing and its Applications


Outline3

Outline

  • Introduction

  • Cluster-based retrieval of images

  • Experiments

  • Conclusions and future work

IEEE Int'l Symposium on Signal Processing and its Applications


Conclusions

Conclusions

  • Retrieving image clusters by unsupervised learning

  • Tested using 60,000 images from COREL and images from WWW

IEEE Int'l Symposium on Signal Processing and its Applications


Future work

Future Work

  • Recursive Ncut

  • Representative image

  • Other graph theoretic clustering techniques

  • Nonlinear dimensionality reduction

IEEE Int'l Symposium on Signal Processing and its Applications


Thank you

Thank You!

IEEE Int'l Symposium on Signal Processing and its Applications


  • Login