recognition and matching based on local invariant features n.
Download
Skip this Video
Download Presentation
Recognition and Matching based on local invariant features

Loading in 2 Seconds...

play fullscreen
1 / 37

Recognition and Matching based on local invariant features - PowerPoint PPT Presentation


  • 168 Views
  • Uploaded on

Recognition and Matching based on local invariant features. Cordelia Schmid INRIA, Grenoble. David Lowe Univ. of British Columbia. ( ). Introduction. Local invariant photometric descriptors. local descriptor. Local : robust to occlusion/clutter + no segmentation

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Recognition and Matching based on local invariant features' - althea


Download Now An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
recognition and matching based on local invariant features

Recognition and Matching based on local invariant features

Cordelia Schmid

INRIA, Grenoble

David Lowe

Univ. of British Columbia

introduction

( )

Introduction

Local invariant photometric descriptors

local descriptor

Local : robust to occlusion/clutter + no segmentation

Photometric : distinctive

Invariant : to image transformations + illumination changes

history matching
History - Matching

Matching based on line segments

  • Not very discriminant
  • Solution : matching with interest points & correlation

[ A robust technique for matching two uncalibrated images through the recovery of the unknown epipolar geometry,

Z. Zhang, R. Deriche, O. Faugeras and Q. Luong,

Artificial Intelligence 1995 ]

approach
Approach
  • Extraction of interest points with the Harris detector
  • Comparison of points with cross-correlation
  • Verification with the fundamental matrix
harris detector
Harris detector

Interest points extracted with Harris (~ 500 points)

cross correlation matching
Cross-correlation matching

Initial matches (188 pairs)

global constraints
Global constraints

Robust estimation of the fundamental matrix

89 outliers

99 inliers

summary of the approach
Summary of the approach
  • Very good results in the presence of occlusion and clutter
    • local information
    • discriminant greyvalue information
    • robust estimation of the global relation between images
    • for limited view point changes
  • Solution for more general view point changes
    • wide baseline matching (different viewpoint, scale and rotation)
    • local invariant descriptors based on greyvalue information
history recognition
History - Recognition

Color histogram [Swain 91]

Each pixel is described

by a color vector

Distribution of color vectors

is described by a histogram

=> not robust to occlusion, not invariant, not distinctive

history recognition1
History - Recognition

Eigenimages [Turk 91]

  • Each face vector is represented in the eigenimage space
    • eigenvectors with the highest eigenvalues = eigenimages
  • The new image is projected into the eigenimage space
    • determine the closest face

.

.

.

.

  • not robust to occlusion, requires segmentation, not invariant,

discriminant

history recognition2
History - Recognition

Geometric invariants [Rothwell 92]

  • Function with a value independent of the transformation
  • Invariant for image rotation : distance of two points
  • Invariant for planar homography : cross-ratio

where

=> local and invariant, not discriminant, requires sub-pixel extraction of primitives

history recognition3
History - Recognition

Problems : occlusion, clutter, image transformations, distinctiveness

  • Solution : recognition with local photometric invariants

[ Local greyvalue invariants for image retrieval,

C. Schmid and R. Mohr,

PAMI 1997 ]

approach1

( )

Approach

1) Extraction of interest points (characteristic locations)

2) Computation of local descriptors

3) Determining correspondences

4) Selection of similar images

local descriptor

interest points
Interest points

Geometric features

repeatable under transformations

2D characteristics of the signal

high informational content

Comparison of different detectors [Schmid98]

Harris detector

harris detector1
Harris detector

Based on the idea of auto-correlation

Important difference in all directions => interest point

harris detector2
Harris detector

Auto-correlation function for a point and a shift

Discret shifts can be avoided with the auto-correlation matrix

harris detector3
Harris detector

Auto-correlation matrix

harris detection
Harris detection
  • Auto-correlation matrix
    • captures the structure of the local neighborhood
    • measure based on eigenvalues of this matrix
      • 2 strong eigenvalues => interest point
      • 1 strong eigenvalue => contour
      • 0 eigenvalue => uniform region
  • Interest point detection
    • threshold on the eigenvalues
    • local maximum for localization
local descriptors

( )

Local descriptors

local descriptor

Descriptors characterize the local neighborhood of a point

local descriptors1
Local descriptors

Greyvalue derivatives

local descriptors2
Local descriptors

Invariance to image rotation : differential invariants [Koen87]

local descriptors3
Local descriptors

Robustness to illumination changes

In case of an affine transformation

local descriptors4
Local descriptors

Robustness to illumination changes

In case of an affine transformation

or normalization of the image patch with mean and variance

determining correspondences

( )

( )

Determining correspondences

?

=

Vector comparison using the Mahalanobis distance

selection of similar images
Selection of similar images
  • In a large database
    • voting algorithm
    • additional constraints
  • Rapid acces with an indexing mechanism
voting algorithm

( )

vector of

local characteristics

Voting algorithm
voting algorithm1

1

1

0

1

1

2

2

1

1

I is the corresponding model image

1

Voting algorithm

}

}

additional constraints
Additional constraints
  • Semi-local constraints
    • neighboring points should match
    • angles, length ratios should be similar
  • Global constraints
    • robust estimation of the image transformation (homogaphy, epipolar geometry)

1

1

2

2

3

3

results
Results

database with ~1000 images

summary of the approach1
Summary of the approach
  • Very good results in the presence of occlusion and clutter
    • local information
    • discriminant greyvalue information
    • invariance to image rotation and illumination
  • Not invariance to scale and affine changes
  • Solution for more general view point changes
    • local invariant descriptors to scale and rotation
    • extraction of invariant points and regions
approach for matching and recognition
Approach for Matching and Recognition
  • Detection of interest points/regions
    • Harris detector (extension to scale and affine invariance)
    • Blob detector based on Laplacian
  • Computation of descriptors for each point
  • Similarity of descriptors
  • Semi-local constraints
  • Global verification
approach for matching and recognition1
Approach for Matching and Recognition
  • Detection of interest points/regions
  • Computation of descriptors for each point
    • greyvalue patch, diff. invariants, steerable filter, SIFT descriptor
  • Similarity of descriptors
    • correlation, Mahalanobis distance, Euclidean distance
  • Semi-local constraints
  • Global verification
approach for matching and recognition2
Approach for Matching and Recognition
  • Detection of interest points/regions
  • Computation of descriptors for each point
  • Similarity of descriptors
  • Semi-local constraints
    • geometrical or statistical relations between neighborhood points
  • Global verification
    • robust estimation of geometry between images
overview
Overview

8:30-8:45 Scale invariant interest points

8:45-9:00 SIFT descriptors

9:00-9:25 Affine invariance of interest points + applications

9:25-9:45 Evaluation of interest points + descriptors

9:45-10:15 Break

overview1
Overview

10:15-11:15 Object recognition system, demo, applications

11:15-11:45 Recognition of textures and object classes

11:45-12:00 Future directions + discussion