Image stitching
This presentation is the property of its rightful owner.
Sponsored Links
1 / 63

Image Stitching PowerPoint PPT Presentation


  • 120 Views
  • Uploaded on
  • Presentation posted in: General

Image Stitching. Shangliang Jiang Kate Harrison. What is image stitching?. What is image stitching?. Introduction. Are you getting the whole picture? Compact Camera FOV = 50 x 35°. Introduction. Are you getting the whole picture? Compact Camera FOV = 50 x 35°

Download Presentation

Image Stitching

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Image stitching

Image Stitching

Shangliang Jiang

Kate Harrison


What is image stitching

What is image stitching?


What is image stitching1

What is image stitching?


Introduction

Introduction

  • Are you getting the whole picture?

    • Compact Camera FOV = 50 x 35°


Introduction1

Introduction

  • Are you getting the whole picture?

    • Compact Camera FOV = 50 x 35°

    • Human FOV = 200 x 135°


Introduction2

Introduction

  • Are you getting the whole picture?

    • Compact Camera FOV = 50 x 35°

    • Human FOV = 200 x 135°

    • Panoramic Mosaic = 360 x 180°


Recognizing panoramas

Recognizing Panoramas

  • 1D Rotations (q)

    • Ordering  matching images


Recognizing panoramas1

Recognizing Panoramas

  • 1D Rotations (q)

    • Ordering  matching images


Recognizing panoramas2

Recognizing Panoramas

  • 1D Rotations (q)

    • Ordering  matching images


Recognizing panoramas3

  • 2D Rotations (q, f)

    • Ordering  matching images

Recognizing Panoramas

  • 1D Rotations (q)

    • Ordering  matching images


Recognizing panoramas4

  • 2D Rotations (q, f)

    • Ordering  matching images

Recognizing Panoramas

  • 1D Rotations (q)

    • Ordering  matching images


Recognizing panoramas5

  • 2D Rotations (q, f)

    • Ordering  matching images

Recognizing Panoramas

  • 1D Rotations (q)

    • Ordering  matching images


Recognizing panoramas6

Recognizing Panoramas


Overview

Overview

  • Feature Matching

  • Image Matching

  • Bundle Adjustment

  • Image Compositing

  • Conclusions


Overview1

Overview

  • Feature Matching

  • Image Matching

  • Bundle Adjustment

  • Image Compositing

  • Conclusions


Overview2

Overview

  • Feature Matching

    • SIFT Features

    • Nearest Neighbor Matching

  • Image Matching

  • Bundle Adjustment

  • Image Compositing

  • Conclusions


Sift features

SIFT Features

  • SIFT features are…

    • Geometrically invariant to similarity transforms,

      • some robustness to affine change

    • Photometrically invariant to affine changes in intensity


Overview3

Overview

  • Feature Matching

    • SIFT Features

    • Nearest Neighbor Matching

  • Image Matching

  • Bundle Adjustment

  • Image Compositing

  • Conclusions


Nearest neighbor matching

Nearest Neighbor Matching

  • Find k nearest neighbors for each feature

    • k  number of overlapping images (we use k = 4)

  • Use k-d tree

    • k-d tree recursively bi-partitions data at mean in the dimension of maximum variance

    • Approximate nearest neighbors found in O(nlogn)


Overview4

Overview

  • Feature Matching

    • SIFT Features

    • Nearest Neighbor Matching

  • Image Matching

  • Bundle Adjustment

  • Image Compositing

  • Conclusions


Overview5

Overview

  • Feature Matching

  • Image Matching

  • Bundle Adjustment

  • Image Compositing

  • Conclusions


Overview6

Overview

  • Feature Matching

  • Image Matching

  • Bundle Adjustment

  • Image Compositing

  • Conclusions


Overview7

Overview

  • Feature Matching

  • Image Matching

    • Random Sample Consensus (RANSAC) for Homography

    • Probabilistic model for verification

  • Bundle Adjustment

  • Image Compositing

  • Conclusions


Overview8

Overview

  • Feature Matching

  • Image Matching

    • Random Sample Consensus (RANSAC) for Homography

    • Probabilistic model for verification

  • Bundle Adjustment

  • Image Compositing

  • Conclusions


Ransac for homography

RANSAC for Homography


Ransac for homography1

RANSAC for Homography

  • SIFT features of two similar images


Ransac for homography2

RANSAC for Homography

  • SIFT features common to both images


Ransac for homography3

RANSAC for Homography

  • Select random subset of features (e.g. 6)

  • Compute motion estimate

  • Apply motion estimate to all SIFT features

    • Compute error: feature pairs not described by motion estimate

  • Repeat many times (e.g. 500)

    • Keep estimate with best error


Ransac for homography4

RANSAC for Homography


Overview9

Overview

  • Feature Matching

  • Image Matching

    • Random Sample Consensus (RANSAC) for Homography

    • Probabilistic model for verification

  • Bundle Adjustment

  • Image Compositing

  • Conclusions


Probabilistic model for verification

Probabilistic model for verification

  • Potential problem:

    • Two images don’t match…

    • … but RANSAC found a motion estimate

  • Do a quick check to make sure the images do match

    • MAP for inliers vs. outliers


Finding the panoramas

Finding the panoramas


Finding the panoramas1

Finding the panoramas


Finding the panoramas2

Finding the panoramas


Overview10

Overview

  • Feature Matching

  • Image Matching

    • RANSAC for Homography

    • Probabilistic model for verification

  • Bundle Adjustment

  • Image Compositing

  • Conclusions


Overview11

Overview

  • Feature Matching

  • Image Matching

  • Bundle Adjustment

  • Image Compositing

  • Conclusions


Overview12

Overview

  • Feature Matching

  • Image Matching

  • Bundle Adjustment

  • Image Compositing

  • Conclusions


Overview13

Overview

  • Feature Matching

  • Image Matching

  • Bundle Adjustment

    • Error function

  • Image Compositing

  • Conclusions


Overview14

Overview

  • Feature Matching

  • Image Matching

  • Bundle Adjustment

    • Error function

  • Image Compositing

  • Conclusions


Bundle adjustment

Bundle Adjustment

  • New images initialised with rotation, focal length of best matching image


Bundle adjustment1

Bundle Adjustment

  • New images initialised with rotation, focal length of best matching image


Error function

Error function

  • Sum of squared projection errors

    • n = #images

    • I(i) = set of image matches to image i

    • F(i, j) = set of feature matches between images i,j

    • rijk = residual of kth feature match between images i,j

  • Robust error function


Overview15

Overview

  • Feature Matching

  • Image Matching

  • Bundle Adjustment

    • Error function

  • Image Compositing

  • Conclusions


Overview16

Overview

  • Feature Matching

  • Image Matching

  • Bundle Adjustment

  • Image Compositing

  • Conclusions


Overview17

Overview

  • Feature Matching

  • Image Matching

  • Bundle Adjustment

  • Image Compositing

  • Conclusions


Blending

Blending


Gain compensation

Gain compensation


How do we blend

How do we blend?

Linear blending

Multi-band blending


Multi band blending

Multi-band Blending

  • Burt & Adelson 1983

    • Blend frequency bands over range l


2 band blending

2-band Blending

Low frequency (l > 2 pixels)

High frequency (l < 2 pixels)


3 band blending band 1 high frequencies

3-band blendingBand 1: high frequencies


3 band blending band 2 mid range frequencies

3-band blendingBand 2: mid-range frequencies


3 band blending band 3 low frequencies

3-band blendingBand 3: low frequencies


Panorama straightening

Panorama straightening

Heuristic: people tend to shoot pictures in a certain way


Overview18

Overview

  • Feature Matching

  • Image Matching

  • Bundle Adjustment

  • Image Compositing

  • Conclusions


Overview19

Overview

  • Feature Matching

  • Image Matching

  • Bundle Adjustment

  • Image Compositing

  • Conclusions


Conclusion

Conclusion


Algorithm

Algorithm


Autostitch program

AutoStitch program

AutoStitch.net


Open questions

Open questions

  • Advanced camera modeling

    • radial distortion, camera motion, scene motion, vignetting, exposure, high dynamic range, flash …

  • Full 3D case – recognizing 3D objects/scenes in unordered datasets


Credits

Credits

  • Automatic Panoramic Image Stitching Using Invariant Features, 2007

    • Matthew Brown and David G. Lowe (Uni. of British Columbia)

  • Recognising Panoramas, 2003

    • Matthew Brown and David G. Lowe (Uni. of British Columbia)

    • 2003

    • Thanks for the slides!

  • Image Alignment and Stitching: A Tutorial, 2006

    • Richard Szeliski (Microsoft)


Questions

Questions?


  • Login