features feature descriptors matching
Skip this Video
Download Presentation
Features, Feature descriptors, Matching

Loading in 2 Seconds...

play fullscreen
1 / 38

Features, Feature descriptors, Matching - PowerPoint PPT Presentation

  • Uploaded on

Features, Feature descriptors, Matching. Jana Kosecka George Mason University. Computer Vision. Visual Sensing. Images I(x,y) – brightness patterns. image appearance depends on structure of the scene material and reflectance properties of the objects

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about ' Features, Feature descriptors, Matching' - hedya

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
features feature descriptors matching

Features, Feature descriptors, Matching

Jana Kosecka

George Mason University


Computer Vision

Visual Sensing

Images I(x,y) – brightness patterns

  • image appearance depends on structure of the scene
  • material and reflectance properties of the objects
  • position and strength of light sources

MSRI Workshop, January 2005

what gives rise to images
What gives rise to images
  • photometric properties of the environment
  • geometric properties of the environment

MSRI Workshop, January 2005

basic ingredients
Basic ingredients

Radiance – amount of energy emitted along certain direction

Iradiance – amount of energy received along certain direction

BRDF – bidirectional reflectance distribution

Lambertian surfaces – the appearance depends only on radiance, not

on the viewing direction

Image intensity for a Lambertian surface

MSRI Workshop, January 2005


MSRI Workshop, January 2005

image primitives and matching
Image Primitives and Matching

Given an image point in left image, what is the (corresponding) point in the right

image, which is the projection of the same 3-D point

MSRI Workshop, January 2005

image primitives and correspondence
Image Primitives and Correspondence

Difficulties – ambiguities, large changes of appearance, due to change

of viewpoint, non-uniquess

MSRI Workshop, January 2005


Matching - Correspondence


Lambertian assumption

Rigid body motion


MSRI Workshop, January 2005

local deformation models
Local Deformation Models
  • Translational model
  • Affine model
  • Transformation of the intensity values taking into account occlusions and noise

MSRI Workshop, January 2005

matching and correspondence
Matching and Correspondence

Motivated by problems

  • Reconstruction of 3D scene from multiple views
  • Object recognition using (constellation of)

features models


  • Small base-line matching
  • Wide base-line matching – large view point changes
  • For now assuming Lambertian assumption – appearance

of a local surface patch is independent of the viewpoint

MSRI Workshop, January 2005

feature tracking and optical flow
Feature Tracking and Optical Flow
  • Translational model
  • Small baseline
  • RHS approximation by the first two terms of Taylor series
  • Brightness constancy constraint

MSRI Workshop, January 2005


Feature Tracking and Optical flow

  • Integrate around over image patch
  • Solve

MSRI Workshop, January 2005


Optical Flow, Feature Tracking


rank(G) = 0 blank wall problem

rank(G) = 1 aperture problem

rank(G) = 2 enough texture – good feature candidates

In reality: choice of threshold is involved

MSRI Workshop, January 2005

affine feature tracking
Affine feature tracking

Intensity offset

Contrast change

MSRI Workshop, January 2005


Optical Flow

  • Previous method - assumption locally constant flow
  • Alternative regularization techniques (locally smooth flow fields,
  • integration along contours)
  • Qualitative properties of the motion fields

MSRI Workshop, January 2005

point feature extraction
Point Feature Extraction
  • Compute eigenvalues of G
  • If smalest eigenvalue  of G is bigger than  - mark pixel as candidate
  • feature point
  • Alternatively feature quality function (Harris Corner Detector)

MSRI Workshop, January 2005


Harris Corner Detector - Example

MSRI Workshop, January 2005

feature selection
Feature Selection
  • Compute Image Gradient
  • Compute Feature Quality measure for each pixel
  • Search for local maxima

MSRI Workshop, January 2005

Feature Quality Function

Local maxima of feature quality function

feature tracking
Feature Tracking
  • Translational motion model
  • Closed form solution
  • Build an image pyramid
  • Start from coarsest level
  • Estimate the displacement at the coarsest level
  • Iterate until finest level

MSRI Workshop, January 2005

coarse to fine feature tracking
Coarse to fine feature tracking




  • compute
  • warp the window in the second image by
  • update the displacement
  • go to finer level
  • At the finest level repeat for several iterations

MSRI Workshop, January 2005

tracked features
Tracked Features

MSRI Workshop, January 2005

wide baseline matching
Wide baseline matching

Point features detected by Harris Corner detector

MSRI Workshop, January 2005

region based similarity metric
Region based Similarity Metric
  • Sum of squared differences
  • Normalize cross-correlation
  • Sum of absolute differences

MSRI Workshop, January 2005

ncc score for two widely separated views
NCC score for two widely separated views

NCC score

MSRI Workshop, January 2005

advanced matching techniques

( )

Advanced matching techniques

NCC - is not invariant with respect to image transformation

1. Selected salient image locations - points, pieces of countours

2. Associate Local photometric descriptors

3. Invariance to image transformations + illumination changes

MSRI Workshop, January 2005

summary of the approach
Summary of the approach
  • Very good results in the presence of occlusion and clutter
    • local information
    • discriminant greyvalue information
    • robust estimation of the global relation between images
    • for limited view point changes
  • Solution for more general view point changes
    • wide baseline matching (different viewpoint, scale and rotation)
    • local invariant descriptors based on greyvalue information

MSRI Workshop, January 2005

local descriptors
Local descriptors

Greyvalue derivatives

Invariance to image rotation :

differential invariants [Koenderink87]

MSRI Workshop, January 2005

feature detection and matching
Feature Detection and Matching
  • Detection of interest points/regions
    • Harris detector (extension to scale and affine invariance)
  • Computation of descriptors for each point (e.g. diff. invariants, steerable filters,

SIFT descriptor)

  • Similarity of descriptors

(Euclidean distance, Mahalanobis Distance)

MSRI Workshop, January 2005

keypoint detector and sift descriptor
Keypoint Detector and SIFT Descriptor
  • Each image is characterized by a set of scale-invariant keypoints and their associated descriptors [D. Lowe,2000]
  • Keypoints - extrema in DOG pyramid
  • Descriptor – 8 bin orientation histograms computed

over 4 x 4 grid overlayed over pixel neighbourhood

and stacked together to form a 128 dim feature vector

MSRI Workshop, January 2005

sift keypoints
SIFT Keypoints

MSRI Workshop, January 2005

  • Scale invariance is not sufficient for large baseline changes
  • State of the art on affine invariant points/regions
  • Affine invariant interest points
  • Application to recognition

MSRI Workshop, January 2005

scale invariant interest points
Scale invariant interest points

Invariant points + associated regions [Mikolajczyk & Schmid’01]

multi-scale Harris points

selection of points

at the characteristic scale

with Laplacian

Courtesy of Schimd’01

MSRI Workshop, January 2005

viewpoint changes
Viewpoint changes
  • Locally approximated by an affine transformation

detected scale invariant region

projected region

Courtesy of Schimd’01

MSRI Workshop, January 2005

affine invariant harris points
Affine invariant Harris points
  • Localization & scale influence affine neighhorbood

=> affine invariant Harris points (Mikolajczyk & Schmid’02)

  • Iterative estimation of these parameters
    • localization– local maximum of the Harris measure
    • scale – automatic scale selection with the Laplacian
    • affine neighborhood – normalization with second moment matrix

Repeat estimation until convergence

  • Initialization with multi-scale interest points

MSRI Workshop, January 2005

alternative features descriptors
Alternative features/descriptors
  • Affine invariant regions (Tuytelaars et al.’00)
    • ellipses fitted to intensity maxima
    • parallelogram formed by interest points and edges
  • Maximally stable regions (Matas et al. BMVC’02)
  • regions stable across large range of thresholds, connected
  • components of thresholded image
  • descriptors – rotationaly and affine invariant and color moments

MSRI Workshop, January 2005

feature matches
Feature Matches

33 correct matches

Courtesy of Schimd’01

MSRI Workshop, January 2005

pieces of countour line descriprors
Pieces of Countour/Line descriprors
  • Select salient pieces using scale invariant

detection techniques

  • Characterize either the intensity profile

along contour/or local neighbourhood with

sideness information – form the descriptor

  • Type of suitable salient regions depends

of the class of objects

  • Computational model of visual attention can

guide the process of selecting salient regions

MSRI Workshop, January 2005

additional changes of the appearance
Additional changes of the appearance

MSRI Workshop, January 2005