Feature detection and matching in images with radial distortion
This presentation is the property of its rightful owner.
Sponsored Links
1 / 22

Feature Detection and Matching in Images with Radial Distortion PowerPoint PPT Presentation


  • 99 Views
  • Uploaded on
  • Presentation posted in: General

Feature Detection and Matching in Images with Radial Distortion. Miguel Lourenço, João P. Barreto, Abed Malti Institute for Systems and Robotics, Faculty of Science and Technology University of Coimbra, Portugal. Presentation Outline. SIFT Features – brief overview

Download Presentation

Feature Detection and Matching in Images with Radial Distortion

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Feature detection and matching in images with radial distortion

Feature Detection and Matching in Images with Radial Distortion

Miguel Lourenço, João P. Barreto, Abed Malti

Institute for Systems and Robotics,

Faculty of Science and Technology

University of Coimbra, Portugal


Presentation outline

Presentation Outline

  • SIFT Features – brief overview

  • RD problems in keypoint detection and matching

    • Theoretical reasoning

    • Experimental validation

  • Improvement to the SIFT algorithm to enhance it with RD

  • Real experiments – a comparison study

  • Motion estimation and 3D reconstruction in endoscopic images


Motivation for keypoint detection and matching

Motivation for keypoint detection and matching

  • Point correspondence across multiple views

    • Camera calibration

    • Sparse 3D reconstruction

    • Recover camera/robot motion

    • Visual Slam

  • Representation of image content

    • Image retrieval applications

    • Recognition tasks (e.g. Voc-tree)

    • Image compression

Partioning of the descriptor space


Sift features lowe ijcv 2004 6725 citations on google scholar

SIFT Features (Lowe, IJCV 2004 – 6725 citations on google scholar)

  • SIFT is probably the most broadly used algorithm for keypoint detection and matching

  • How does SIFT work ?

    • Image salient points detected in a scale space framework

(x,y,sigma)

Increase scale

Gaussian pyramid

DoG pyramid


Sift features lowe ijcv 2004 6725 citations on scholar google

SIFT Features (Lowe, IJCV 2004 – 6725 citations on Scholar google)

  • SIFT is probably the most broadly used algorithm for keypoint detection and matching

  • How does SIFT work ?

    • Image salient points detected in a scale space framework

    • SIFT descriptor is computed based on local image gradient on a scale and rotation normalized patch


Problem statement 1 2

Problem statement (1/2)

Q

  • What is radial distortion?

    • Bending of the light rays pulling image points towards the center along radial direction

X

Z

Mini-lens

Boroscope

Fish-eye lens

  • Cameras with radial lens distortion are often used in computer and robotic vision applications

O

O'


Problem statement 2 2

Problem statement (2/2)

  • SIFT is invariant to rotation and scale but it is not invariant to RD

421 correct matches

336 correct matches

  • Our Contribution: Modifications to the original SIFT for invariance to image RD

  • Assumptions:

    • RD can be fairly described by the division model (Fitzgibbon, CVPR 2001)

    • RD is roughly known ( e.g. line stretching ) (Barreto, CVIU 2006)


Tracking rd effects in sift

Tracking RD effects in SIFT

  • How does RD affect the SIFT algorithm?

    • Study using images with artificially added distortion

      • Isolate the RD effect in SIFT detection and matching

      • Reliable ground truth

RD = 55%

RD = 35%

RD = 0%

RD = 15%

  • Improvement to the SIFT algorithm to handle RD issues

  • Results on real imagery


How does rd affect keypoint detection

How does RD affect keypoint detection?

  • Repeatability of keypoint detection decreases with increasing distortion

  • Small features (fine scale) tend to disappear during the blurring process

  • Coarse features tend to be detected at finer levels of scale

  • Flat regions (e.g. edges) start gain to strong gradient variations

Filtering bounds

Regular DoG pyramid

‘Distorted’ DoG pyramid


Proposed solution adaptive smoothing

Proposed Solution: Adaptive smoothing

  • Radial distortion must be removed before the Gaussian smoothing

  • We can avoid the reconstruction artifacts by using an adaptive filter

Rectification (~ 1.5 seconds in Matlab)


Standard vs adaptive gaussian smoothing

Standard vs Adaptive Gaussian smoothing

  • Inherent properties of the standard Gaussian filter

    • Decouple the convolution mask in X and Y directions

  • Simplification of the adaptive filter

  • Advantages of the Simplified Adaptive Filter

    • Shape only depends on the radius of the convolution window

    • Isotropic filter that can be decoupled for each image radius


Detection repeatability synthetic adding of rd

Detection repeatability (synthetic adding of RD)

Repeatability

Computational time

Error in calibration

  • Better Repeatability results for keypoint detection

  • Lower computational time than image rectification

  • More robust to calibration errors


How does rd affect matching

How does RD affect matching?

  • RD modifies the local structures in the image and by consequence the gradients are affected

  • Changes in local gradients of the image deteriorates SIFT descriptor performance

  • Proposed solution: Compute gradients in the distorted image and perform implicit correction using the jacobian matrix of the distortion function


Matching evaluation synthetic adding of rd

Matching evaluation (synthetic adding of RD)

Implicit gradient correction

SIFT in Rectified Images

SIFT in RD Images

  • Compressive effect adds new contributions to the descriptor that do not occur in undistorted views

  • The matching performance can be improved by correcting image gradients before building the descriptor

  • Implicit gradient correction outperforms explicit image rectification for distortion amounts up to 25%.


Experiments with real images

Experiments with Real Images

  • Planar scenes for repeatability test and scenes with depth variation for motion estimation

    • Firewire camera with regular lens (~ 10% ) of distortion

    • Dragonfly camera with mini lens (~ 25% ) of distortion

    • Firewire camera with fish-eye lens (~ 45% ) of distortion


Planar images

Planar images

SIFT in RD Images

SIFT in Rectified Images

Our method

10% distortion

176 matches

294 matches

364 matches

25% distortion

201 matches

310 matches

401matches

45% distortion

326 matches

112 matches

253 matches


Motion recovery sparse 3d reconstruction

Motion recovery / Sparse 3D reconstruction

  • Scenes with depth variation where wrong matches are discarded using epipolar geometric constraints

Main Scene

10% distortion

25% distortion

45% distortion


Experimental evaluation

Experimental evaluation

Main Scene

Number of Inliers

Inliers Distribution

RMS rotation angle

3D reconstruction


Conclusions future work

Conclusions / Future Work

  • We proposed a set of modifications to the original SIFT algorithm (RD-SIFT) for achieving invariance to radial distortion. The additional computational overhead is minimum

  • RD – SIFT proved to be superior to explicit image correction

    • Better repeatability and retrieval performance

    • Less computational overhead

    • Increased robustness to calibration errors

  • Future Work

    • Extend the approach to other keypoint detectors (e.g. MSER and SURF)

    • Real-time implementation using GPGPU (to make available to the community)

    • Get rid of calibration dependence


Thanks for coming

THANKS FOR COMING 


Detection after explicit rd correction

Detection after explicit RD correction

  • Correct the radial distortion via image rectification

Rectification (~ 1.5 seconds in Matlab)

  • Drawbacks of this approach

    • Signal reconstruction introduces artifacts affecting SIFT performance

Image

1.5x Image (Bilinear)

1.5x Image (Bicubic)


  • Login