The pyramid match kernel and its improvement
This presentation is the property of its rightful owner.
Sponsored Links
1 / 32

The Pyramid Match Kernel and Its Improvement PowerPoint PPT Presentation


  • 55 Views
  • Uploaded on
  • Presentation posted in: General

The Pyramid Match Kernel and Its Improvement. Guo LiJun. PMK: Kristen Grauman,Trevor Darrell ICCV’05, Oral Projects that use LIBPMK Multiple Kernel Learning from Sets of Partially Matching Image Features, UKACC Control 2008.

Download Presentation

The Pyramid Match Kernel and Its Improvement

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


The pyramid match kernel and its improvement

The Pyramid Match Kernel and Its Improvement

Guo LiJun


The pyramid match kernel and its improvement

  • PMK: Kristen Grauman,Trevor Darrell ICCV’05, Oral

  • Projects that use LIBPMK

    • Multiple Kernel Learning from Sets of Partially Matching Image Features, UKACC Control 2008.

    • Distributed Image Search in Camera Sensor Networks, ACM SenSys 2008.

    • Automated Annotation of Drosophila Gene Expression Patterns Using a Controlled Vocabulary, Bioinformatics 2008.

    • Photo-based Question Answering, ACM Multimedia 2008.

    • Unsupervised Feature Selection via Distributed Coding for Multi-view Object Recognition, CVPR 2008.

    • Combining Brain Computer Interfaces with Vision for Object Categorization, CVPR 2008.

    • Scalable Classifiers for Internet Vision Tasks, CVPR Internet Vision Workshop 2008.

    • Object Category Recognition Using Probabilistic Fusion of Speech and Image Classifiers, MLMI 2007.

    • Envisioning Sketch Recognition: A Local Feature Based Approach to Recognizing Informal Sketches, Ph.D. thesis (2007).

    • Adaptive Vocabulary Forests for Dynamic Indexing and Categry Learning, ICCV 2007.


The pyramid match kernel and its improvement

  • Related Work

    • Local Features

    • Kernel Methods

    • Earth Mover’s Distance

    • Multiresolution Histograms


Sets of features

Sets of features


Sets of features1

invariant region descriptors

local shape features

examples under varying conditions

Sets of features


Problem

Problem

How to build a discriminative classifier using the set representation?

  • Kernel-based methods (e.g. SVM) are appealing for efficiency and generalization power…

  • But what is an appropriate kernel?

  • Each instance is unordered set of vectors

  • Varying number of vectors per instance


Other solution

Other solution

Measuring Similarity Between Sets of Features

1)Voting

2)Bags of Prototypical Features

3)Computing Correspondences


Existing set kernels

  • Compute pair-wise similarity between all vectors in each set

  • Wallraven et al., Lyu, Boughhorbel et al.

  • General family of algebraic functions combining local (vector) kernels

  • Shashua & Hazan

High complexity

Existing set kernels

  • Fit(parametric) model to each set, compare with distance over models

    Kondor & Jebara, Moreno et al., Lafferty & Lebanon, Cuturi & Vert,

    Wolf & Shashua

Restrictive assumptions


The pyramid match kernel and its improvement

Similarity Measures for Unordered Features


Partial matching for sets of features

Partial matching for sets of features

Compare sets by computing a partialmatching between their features.

Robust to clutter, segmentation errors, occlusion…


Pyramid match

optimal partial matching

Pyramid match


Pyramid match overview

Pyramid match overview

Pyramid match kernel measures similarity of a partial matching between two sets:

  • Place multi-dimensional, multi-resolution grid over point sets

  • Consider points matched at finest resolution where they fall into same grid cell

  • Approximate similarity between matched points with worst case similarity at given level

No explicit search for matches!


Pyramid match kernel

Number of newly matched pairs at level i

Measure of difficulty of a match at level i

Pyramid match kernel

Approximate partial match similarity


Feature extraction

,

Histogram pyramid: level i has bins of size 2i

Feature extraction


Counting matches

Counting matches

Histogram intersection


Counting new matches

matches at this level

matches at previous level

Difference in histogram intersections across levels counts number ofnew pairs matched

Counting new matches

Histogram intersection


Pyramid match kernel1

histogram pyramids

number of newly matched pairs at level i

measure of difficulty of a match at level i

Pyramid match kernel

  • Weights inversely proportional to bin size

  • Normalize kernel values to avoid favoring large sets


Efficiency

Efficiency

For sets with m features of dimension d, and pyramids with L levels, computational complexity of

Pyramid match kernel:

Existing set kernel approaches:

or


The pyramid match kernel and its improvement

Example pyramid match

Level 0


The pyramid match kernel and its improvement

Example pyramid match

Level 1


The pyramid match kernel and its improvement

Example pyramid match

Level 2


Example pyramid match

Example pyramid match

pyramid match

optimal match


The pyramid match kernel and its improvement

Approximation of the optimal partial matching

[Indyk & Thaper]

Matching output

Trial number (sorted by optimal distance)

100 sets with 2D points, cardinalities vary between 5 and 100


Building a classifier

Building a classifier

  • Train SVM by computing kernel values between all labeled training examples

  • Classify novel examples by computing kernel values against support vectors

  • One-versus-all for multi-class classification

Convergence is guaranteed since pyramid match kernel is positive-definite.


Object recognition results

Object recognition results

  • ETH-80 database 8 object classes

  • Features:

    • Harris detector

    • PCA-SIFT descriptor, d=10

Eichhorn and Chapelle 2004


Object recognition results1

Object recognition results

  • Caltech objects database 101 object classes

  • Features:

    • SIFT detector

    • PCA-SIFT descriptor, d=10

  • 30 training images / class

  • 43% recognition rate

    (1% chance performance)

  • 0.002 seconds per match


Localization

Localization

  • Inspect intersections to obtain correspondences between features

  • Higher confidence correspondences at finer resolution levels

target

observation


Pyramid match regression

Pyramid match regression

  • Pose estimation from contour features

  • Train SVR with CG data

  • Features: shape context histograms


Summary pyramid match kernel

Summary: Pyramid match kernel

optimal partial matching between sets of features

difficulty of a match at level i

number of new matches at level i


Summary pyramid match kernel1

Summary: Pyramid match kernel

  • A new similarity measure based on implicit correspondences that approximates the optimal partial matching

    • linear time complexity

    • no independence assumption

    • model-free

    • insensitive to clutter

    • positive-definite function

    • fast, effective object recognition


Disadvantage

Disadvantage

Disregard all information about the spatial layout of the features

Improvement

Spatial Pyramid Matching


The pyramid match kernel and its improvement

Spatial Pyramid Matching

Learn Method: PMK or EMD-NN


  • Login