Feature detection and matching in images with radial distortion
Download
1 / 22

Feature Detection and Matching in Images with Radial Distortion - PowerPoint PPT Presentation


  • 133 Views
  • Uploaded on

Feature Detection and Matching in Images with Radial Distortion. Miguel Lourenço, João P. Barreto, Abed Malti Institute for Systems and Robotics, Faculty of Science and Technology University of Coimbra, Portugal. Presentation Outline. SIFT Features – brief overview

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' Feature Detection and Matching in Images with Radial Distortion' - forrest-lloyd


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Feature detection and matching in images with radial distortion

Feature Detection and Matching in Images with Radial Distortion

Miguel Lourenço, João P. Barreto, Abed Malti

Institute for Systems and Robotics,

Faculty of Science and Technology

University of Coimbra, Portugal


Presentation outline
Presentation Outline Distortion

  • SIFT Features – brief overview

  • RD problems in keypoint detection and matching

    • Theoretical reasoning

    • Experimental validation

  • Improvement to the SIFT algorithm to enhance it with RD

  • Real experiments – a comparison study

  • Motion estimation and 3D reconstruction in endoscopic images


Motivation for keypoint detection and matching
Motivation for keypoint detection and matching Distortion

  • Point correspondence across multiple views

    • Camera calibration

    • Sparse 3D reconstruction

    • Recover camera/robot motion

    • Visual Slam

  • Representation of image content

    • Image retrieval applications

    • Recognition tasks (e.g. Voc-tree)

    • Image compression

Partioning of the descriptor space


Sift features lowe ijcv 2004 6725 citations on google scholar
SIFT Features Distortion(Lowe, IJCV 2004 – 6725 citations on google scholar)

  • SIFT is probably the most broadly used algorithm for keypoint detection and matching

  • How does SIFT work ?

    • Image salient points detected in a scale space framework

(x,y,sigma)

Increase scale

Gaussian pyramid

DoG pyramid


Sift features lowe ijcv 2004 6725 citations on scholar google
SIFT Features Distortion(Lowe, IJCV 2004 – 6725 citations on Scholar google)

  • SIFT is probably the most broadly used algorithm for keypoint detection and matching

  • How does SIFT work ?

    • Image salient points detected in a scale space framework

    • SIFT descriptor is computed based on local image gradient on a scale and rotation normalized patch


Problem statement 1 2
Problem statement (1/2) Distortion

Q

  • What is radial distortion?

    • Bending of the light rays pulling image points towards the center along radial direction

X

Z

Mini-lens

Boroscope

Fish-eye lens

  • Cameras with radial lens distortion are often used in computer and robotic vision applications

O

O'


Problem statement 2 2
Problem statement (2/2) Distortion

  • SIFT is invariant to rotation and scale but it is not invariant to RD

421 correct matches

336 correct matches

  • Our Contribution: Modifications to the original SIFT for invariance to image RD

  • Assumptions:

    • RD can be fairly described by the division model (Fitzgibbon, CVPR 2001)

    • RD is roughly known ( e.g. line stretching ) (Barreto, CVIU 2006)


Tracking rd effects in sift
Tracking RD effects in SIFT Distortion

  • How does RD affect the SIFT algorithm?

    • Study using images with artificially added distortion

      • Isolate the RD effect in SIFT detection and matching

      • Reliable ground truth

RD = 55%

RD = 35%

RD = 0%

RD = 15%

  • Improvement to the SIFT algorithm to handle RD issues

  • Results on real imagery


How does rd affect keypoint detection
How does RD affect keypoint detection? Distortion

  • Repeatability of keypoint detection decreases with increasing distortion

  • Small features (fine scale) tend to disappear during the blurring process

  • Coarse features tend to be detected at finer levels of scale

  • Flat regions (e.g. edges) start gain to strong gradient variations

Filtering bounds

Regular DoG pyramid

‘Distorted’ DoG pyramid


Proposed solution adaptive smoothing
Proposed Solution: Adaptive smoothing Distortion

  • Radial distortion must be removed before the Gaussian smoothing

  • We can avoid the reconstruction artifacts by using an adaptive filter

Rectification (~ 1.5 seconds in Matlab)


Standard vs adaptive gaussian smoothing
Standard Distortionvs Adaptive Gaussian smoothing

  • Inherent properties of the standard Gaussian filter

    • Decouple the convolution mask in X and Y directions

  • Simplification of the adaptive filter

  • Advantages of the Simplified Adaptive Filter

    • Shape only depends on the radius of the convolution window

    • Isotropic filter that can be decoupled for each image radius


Detection repeatability synthetic adding of rd
Detection repeatability (synthetic adding of RD) Distortion

Repeatability

Computational time

Error in calibration

  • Better Repeatability results for keypoint detection

  • Lower computational time than image rectification

  • More robust to calibration errors


How does rd affect matching
How does RD affect matching? Distortion

  • RD modifies the local structures in the image and by consequence the gradients are affected

  • Changes in local gradients of the image deteriorates SIFT descriptor performance

  • Proposed solution: Compute gradients in the distorted image and perform implicit correction using the jacobian matrix of the distortion function


Matching evaluation synthetic adding of rd
Matching evaluation (synthetic adding of RD) Distortion

Implicit gradient correction

SIFT in Rectified Images

SIFT in RD Images

  • Compressive effect adds new contributions to the descriptor that do not occur in undistorted views

  • The matching performance can be improved by correcting image gradients before building the descriptor

  • Implicit gradient correction outperforms explicit image rectification for distortion amounts up to 25%.


Experiments with real images
Experiments with Real Images Distortion

  • Planar scenes for repeatability test and scenes with depth variation for motion estimation

    • Firewire camera with regular lens (~ 10% ) of distortion

    • Dragonfly camera with mini lens (~ 25% ) of distortion

    • Firewire camera with fish-eye lens (~ 45% ) of distortion


Planar images
Planar images Distortion

SIFT in RD Images

SIFT in Rectified Images

Our method

10% distortion

176 matches

294 matches

364 matches

25% distortion

201 matches

310 matches

401matches

45% distortion

326 matches

112 matches

253 matches


Motion recovery sparse 3d reconstruction
Motion recovery / Sparse 3D reconstruction Distortion

  • Scenes with depth variation where wrong matches are discarded using epipolar geometric constraints

Main Scene

10% distortion

25% distortion

45% distortion


Experimental evaluation
Experimental evaluation Distortion

Main Scene

Number of Inliers

Inliers Distribution

RMS rotation angle

3D reconstruction


Conclusions future work
Conclusions / Future Work Distortion

  • We proposed a set of modifications to the original SIFT algorithm (RD-SIFT) for achieving invariance to radial distortion. The additional computational overhead is minimum

  • RD – SIFT proved to be superior to explicit image correction

    • Better repeatability and retrieval performance

    • Less computational overhead

    • Increased robustness to calibration errors

  • Future Work

    • Extend the approach to other keypoint detectors (e.g. MSER and SURF)

    • Real-time implementation using GPGPU (to make available to the community)

    • Get rid of calibration dependence


Thanks for coming
THANKS FOR COMING Distortion


Detection after explicit rd correction
Detection after explicit RD correction Distortion

  • Correct the radial distortion via image rectification

Rectification (~ 1.5 seconds in Matlab)

  • Drawbacks of this approach

    • Signal reconstruction introduces artifacts affecting SIFT performance

Image

1.5x Image (Bilinear)

1.5x Image (Bicubic)


ad