Features feature descriptors matching
This presentation is the property of its rightful owner.
Sponsored Links
1 / 38

Features, Feature descriptors, Matching PowerPoint PPT Presentation


  • 95 Views
  • Uploaded on
  • Presentation posted in: General

Features, Feature descriptors, Matching. Jana Kosecka George Mason University. Computer Vision. Visual Sensing. Images I(x,y) – brightness patterns. image appearance depends on structure of the scene material and reflectance properties of the objects

Download Presentation

Features, Feature descriptors, Matching

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Features feature descriptors matching

Features, Feature descriptors, Matching

Jana Kosecka

George Mason University


Features feature descriptors matching

Computer Vision

Visual Sensing

Images I(x,y) – brightness patterns

  • image appearance depends on structure of the scene

  • material and reflectance properties of the objects

  • position and strength of light sources

MSRI Workshop, January 2005


What gives rise to images

What gives rise to images

  • photometric properties of the environment

  • geometric properties of the environment

MSRI Workshop, January 2005


Basic ingredients

Basic ingredients

Radiance – amount of energy emitted along certain direction

Iradiance – amount of energy received along certain direction

BRDF – bidirectional reflectance distribution

Lambertian surfaces – the appearance depends only on radiance, not

on the viewing direction

Image intensity for a Lambertian surface

MSRI Workshop, January 2005


Challenges

Challenges

MSRI Workshop, January 2005


Image primitives and matching

Image Primitives and Matching

Given an image point in left image, what is the (corresponding) point in the right

image, which is the projection of the same 3-D point

MSRI Workshop, January 2005


Image primitives and correspondence

Image Primitives and Correspondence

Difficulties – ambiguities, large changes of appearance, due to change

of viewpoint, non-uniquess

MSRI Workshop, January 2005


Features feature descriptors matching

Matching - Correspondence

radiance

Lambertian assumption

Rigid body motion

Correspondence

MSRI Workshop, January 2005


Local deformation models

Local Deformation Models

  • Translational model

  • Affine model

  • Transformation of the intensity values taking into account occlusions and noise

MSRI Workshop, January 2005


Matching and correspondence

Matching and Correspondence

Motivated by problems

  • Reconstruction of 3D scene from multiple views

  • Object recognition using (constellation of)

    features models

    Varieties

  • Small base-line matching

  • Wide base-line matching – large view point changes

  • For now assuming Lambertian assumption – appearance

    of a local surface patch is independent of the viewpoint

MSRI Workshop, January 2005


Feature tracking and optical flow

Feature Tracking and Optical Flow

  • Translational model

  • Small baseline

  • RHS approximation by the first two terms of Taylor series

  • Brightness constancy constraint

MSRI Workshop, January 2005


Features feature descriptors matching

Feature Tracking and Optical flow

  • Integrate around over image patch

  • Solve

MSRI Workshop, January 2005


Features feature descriptors matching

Optical Flow, Feature Tracking

Conceptually:

rank(G) = 0 blank wall problem

rank(G) = 1 aperture problem

rank(G) = 2 enough texture – good feature candidates

In reality: choice of threshold is involved

MSRI Workshop, January 2005


Affine feature tracking

Affine feature tracking

Intensity offset

Contrast change

MSRI Workshop, January 2005


Features feature descriptors matching

Optical Flow

  • Previous method - assumption locally constant flow

  • Alternative regularization techniques (locally smooth flow fields,

  • integration along contours)

  • Qualitative properties of the motion fields

MSRI Workshop, January 2005


Point feature extraction

Point Feature Extraction

  • Compute eigenvalues of G

  • If smalest eigenvalue  of G is bigger than  - mark pixel as candidate

  • feature point

  • Alternatively feature quality function (Harris Corner Detector)

MSRI Workshop, January 2005


Features feature descriptors matching

Harris Corner Detector - Example

MSRI Workshop, January 2005


Feature selection

Feature Selection

  • Compute Image Gradient

  • Compute Feature Quality measure for each pixel

  • Search for local maxima

MSRI Workshop, January 2005

Feature Quality Function

Local maxima of feature quality function


Feature tracking

Feature Tracking

  • Translational motion model

  • Closed form solution

  • Build an image pyramid

  • Start from coarsest level

  • Estimate the displacement at the coarsest level

  • Iterate until finest level

MSRI Workshop, January 2005


Coarse to fine feature tracking

Coarse to fine feature tracking

0

1

2

  • compute

  • warp the window in the second image by

  • update the displacement

  • go to finer level

  • At the finest level repeat for several iterations

MSRI Workshop, January 2005


Tracked features

Tracked Features

MSRI Workshop, January 2005


Wide baseline matching

Wide baseline matching

Point features detected by Harris Corner detector

MSRI Workshop, January 2005


Region based similarity metric

Region based Similarity Metric

  • Sum of squared differences

  • Normalize cross-correlation

  • Sum of absolute differences

MSRI Workshop, January 2005


Ncc score for two widely separated views

NCC score for two widely separated views

NCC score

MSRI Workshop, January 2005


Advanced matching techniques

( )

Advanced matching techniques

NCC - is not invariant with respect to image transformation

1. Selected salient image locations - points, pieces of countours

2. Associate Local photometric descriptors

3. Invariance to image transformations + illumination changes

MSRI Workshop, January 2005


Summary of the approach

Summary of the approach

  • Very good results in the presence of occlusion and clutter

    • local information

    • discriminant greyvalue information

    • robust estimation of the global relation between images

    • for limited view point changes

  • Solution for more general view point changes

    • wide baseline matching (different viewpoint, scale and rotation)

    • local invariant descriptors based on greyvalue information

MSRI Workshop, January 2005


Local descriptors

Local descriptors

Greyvalue derivatives

Invariance to image rotation :

differential invariants [Koenderink87]

MSRI Workshop, January 2005


Feature detection and matching

Feature Detection and Matching

  • Detection of interest points/regions

    • Harris detector (extension to scale and affine invariance)

  • Computation of descriptors for each point (e.g. diff. invariants, steerable filters,

    SIFT descriptor)

  • Similarity of descriptors

    (Euclidean distance, Mahalanobis Distance)

MSRI Workshop, January 2005


Keypoint detector and sift descriptor

Keypoint Detector and SIFT Descriptor

  • Each image is characterized by a set of scale-invariant keypoints and their associated descriptors [D. Lowe,2000]

  • Keypoints - extrema in DOG pyramid

  • Descriptor – 8 bin orientation histograms computed

    over 4 x 4 grid overlayed over pixel neighbourhood

    and stacked together to form a 128 dim feature vector

MSRI Workshop, January 2005


Sift keypoints

SIFT Keypoints

MSRI Workshop, January 2005


Overview

Overview

  • Scale invariance is not sufficient for large baseline changes

  • State of the art on affine invariant points/regions

  • Affine invariant interest points

  • Application to recognition

MSRI Workshop, January 2005


Scale invariant interest points

Scale invariant interest points

Invariant points + associated regions [Mikolajczyk & Schmid’01]

multi-scale Harris points

selection of points

at the characteristic scale

with Laplacian

Courtesy of Schimd’01

MSRI Workshop, January 2005


Viewpoint changes

Viewpoint changes

  • Locally approximated by an affine transformation

detected scale invariant region

projected region

Courtesy of Schimd’01

MSRI Workshop, January 2005


Affine invariant harris points

Affine invariant Harris points

  • Localization & scale influence affine neighhorbood

    => affine invariant Harris points (Mikolajczyk & Schmid’02)

  • Iterative estimation of these parameters

    • localization– local maximum of the Harris measure

    • scale – automatic scale selection with the Laplacian

    • affine neighborhood – normalization with second moment matrix

      Repeat estimation until convergence

  • Initialization with multi-scale interest points

MSRI Workshop, January 2005


Alternative features descriptors

Alternative features/descriptors

  • Affine invariant regions (Tuytelaars et al.’00)

    • ellipses fitted to intensity maxima

    • parallelogram formed by interest points and edges

  • Maximally stable regions (Matas et al. BMVC’02)

  • regions stable across large range of thresholds, connected

  • components of thresholded image

  • descriptors – rotationaly and affine invariant and color moments

MSRI Workshop, January 2005


Feature matches

Feature Matches

33 correct matches

Courtesy of Schimd’01

MSRI Workshop, January 2005


Pieces of countour line descriprors

Pieces of Countour/Line descriprors

  • Select salient pieces using scale invariant

    detection techniques

  • Characterize either the intensity profile

    along contour/or local neighbourhood with

    sideness information – form the descriptor

  • Type of suitable salient regions depends

    of the class of objects

  • Computational model of visual attention can

    guide the process of selecting salient regions

MSRI Workshop, January 2005


Additional changes of the appearance

Additional changes of the appearance

MSRI Workshop, January 2005


  • Login