Image stitching
Download
1 / 63

Image Stitching - PowerPoint PPT Presentation


  • 154 Views
  • Uploaded on
  • Presentation posted in: General

Image Stitching. Shangliang Jiang Kate Harrison. What is image stitching?. What is image stitching?. Introduction. Are you getting the whole picture? Compact Camera FOV = 50 x 35°. Introduction. Are you getting the whole picture? Compact Camera FOV = 50 x 35°

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha

Download Presentation

Image Stitching

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Image Stitching

Shangliang Jiang

Kate Harrison


What is image stitching?


What is image stitching?


Introduction

  • Are you getting the whole picture?

    • Compact Camera FOV = 50 x 35°


Introduction

  • Are you getting the whole picture?

    • Compact Camera FOV = 50 x 35°

    • Human FOV = 200 x 135°


Introduction

  • Are you getting the whole picture?

    • Compact Camera FOV = 50 x 35°

    • Human FOV = 200 x 135°

    • Panoramic Mosaic = 360 x 180°


Recognizing Panoramas

  • 1D Rotations (q)

    • Ordering  matching images


Recognizing Panoramas

  • 1D Rotations (q)

    • Ordering  matching images


Recognizing Panoramas

  • 1D Rotations (q)

    • Ordering  matching images


  • 2D Rotations (q, f)

    • Ordering  matching images

Recognizing Panoramas

  • 1D Rotations (q)

    • Ordering  matching images


  • 2D Rotations (q, f)

    • Ordering  matching images

Recognizing Panoramas

  • 1D Rotations (q)

    • Ordering  matching images


  • 2D Rotations (q, f)

    • Ordering  matching images

Recognizing Panoramas

  • 1D Rotations (q)

    • Ordering  matching images


Recognizing Panoramas


Overview

  • Feature Matching

  • Image Matching

  • Bundle Adjustment

  • Image Compositing

  • Conclusions


Overview

  • Feature Matching

  • Image Matching

  • Bundle Adjustment

  • Image Compositing

  • Conclusions


Overview

  • Feature Matching

    • SIFT Features

    • Nearest Neighbor Matching

  • Image Matching

  • Bundle Adjustment

  • Image Compositing

  • Conclusions


SIFT Features

  • SIFT features are…

    • Geometrically invariant to similarity transforms,

      • some robustness to affine change

    • Photometrically invariant to affine changes in intensity


Overview

  • Feature Matching

    • SIFT Features

    • Nearest Neighbor Matching

  • Image Matching

  • Bundle Adjustment

  • Image Compositing

  • Conclusions


Nearest Neighbor Matching

  • Find k nearest neighbors for each feature

    • k  number of overlapping images (we use k = 4)

  • Use k-d tree

    • k-d tree recursively bi-partitions data at mean in the dimension of maximum variance

    • Approximate nearest neighbors found in O(nlogn)


Overview

  • Feature Matching

    • SIFT Features

    • Nearest Neighbor Matching

  • Image Matching

  • Bundle Adjustment

  • Image Compositing

  • Conclusions


Overview

  • Feature Matching

  • Image Matching

  • Bundle Adjustment

  • Image Compositing

  • Conclusions


Overview

  • Feature Matching

  • Image Matching

  • Bundle Adjustment

  • Image Compositing

  • Conclusions


Overview

  • Feature Matching

  • Image Matching

    • Random Sample Consensus (RANSAC) for Homography

    • Probabilistic model for verification

  • Bundle Adjustment

  • Image Compositing

  • Conclusions


Overview

  • Feature Matching

  • Image Matching

    • Random Sample Consensus (RANSAC) for Homography

    • Probabilistic model for verification

  • Bundle Adjustment

  • Image Compositing

  • Conclusions


RANSAC for Homography


RANSAC for Homography

  • SIFT features of two similar images


RANSAC for Homography

  • SIFT features common to both images


RANSAC for Homography

  • Select random subset of features (e.g. 6)

  • Compute motion estimate

  • Apply motion estimate to all SIFT features

    • Compute error: feature pairs not described by motion estimate

  • Repeat many times (e.g. 500)

    • Keep estimate with best error


RANSAC for Homography


Overview

  • Feature Matching

  • Image Matching

    • Random Sample Consensus (RANSAC) for Homography

    • Probabilistic model for verification

  • Bundle Adjustment

  • Image Compositing

  • Conclusions


Probabilistic model for verification

  • Potential problem:

    • Two images don’t match…

    • … but RANSAC found a motion estimate

  • Do a quick check to make sure the images do match

    • MAP for inliers vs. outliers


Finding the panoramas


Finding the panoramas


Finding the panoramas


Overview

  • Feature Matching

  • Image Matching

    • RANSAC for Homography

    • Probabilistic model for verification

  • Bundle Adjustment

  • Image Compositing

  • Conclusions


Overview

  • Feature Matching

  • Image Matching

  • Bundle Adjustment

  • Image Compositing

  • Conclusions


Overview

  • Feature Matching

  • Image Matching

  • Bundle Adjustment

  • Image Compositing

  • Conclusions


Overview

  • Feature Matching

  • Image Matching

  • Bundle Adjustment

    • Error function

  • Image Compositing

  • Conclusions


Overview

  • Feature Matching

  • Image Matching

  • Bundle Adjustment

    • Error function

  • Image Compositing

  • Conclusions


Bundle Adjustment

  • New images initialised with rotation, focal length of best matching image


Bundle Adjustment

  • New images initialised with rotation, focal length of best matching image


Error function

  • Sum of squared projection errors

    • n = #images

    • I(i) = set of image matches to image i

    • F(i, j) = set of feature matches between images i,j

    • rijk = residual of kth feature match between images i,j

  • Robust error function


Overview

  • Feature Matching

  • Image Matching

  • Bundle Adjustment

    • Error function

  • Image Compositing

  • Conclusions


Overview

  • Feature Matching

  • Image Matching

  • Bundle Adjustment

  • Image Compositing

  • Conclusions


Overview

  • Feature Matching

  • Image Matching

  • Bundle Adjustment

  • Image Compositing

  • Conclusions


Blending


Gain compensation


How do we blend?

Linear blending

Multi-band blending


Multi-band Blending

  • Burt & Adelson 1983

    • Blend frequency bands over range l


2-band Blending

Low frequency (l > 2 pixels)

High frequency (l < 2 pixels)


3-band blendingBand 1: high frequencies


3-band blendingBand 2: mid-range frequencies


3-band blendingBand 3: low frequencies


Panorama straightening

Heuristic: people tend to shoot pictures in a certain way


Overview

  • Feature Matching

  • Image Matching

  • Bundle Adjustment

  • Image Compositing

  • Conclusions


Overview

  • Feature Matching

  • Image Matching

  • Bundle Adjustment

  • Image Compositing

  • Conclusions


Conclusion


Algorithm


AutoStitch program

AutoStitch.net


Open questions

  • Advanced camera modeling

    • radial distortion, camera motion, scene motion, vignetting, exposure, high dynamic range, flash …

  • Full 3D case – recognizing 3D objects/scenes in unordered datasets


Credits

  • Automatic Panoramic Image Stitching Using Invariant Features, 2007

    • Matthew Brown and David G. Lowe (Uni. of British Columbia)

  • Recognising Panoramas, 2003

    • Matthew Brown and David G. Lowe (Uni. of British Columbia)

    • 2003

    • Thanks for the slides!

  • Image Alignment and Stitching: A Tutorial, 2006

    • Richard Szeliski (Microsoft)


Questions?


ad
  • Login