Image stitching
Download
1 / 63

Image Stitching - PowerPoint PPT Presentation


  • 167 Views
  • Uploaded on

Image Stitching. Shangliang Jiang Kate Harrison. What is image stitching?. What is image stitching?. Introduction. Are you getting the whole picture? Compact Camera FOV = 50 x 35°. Introduction. Are you getting the whole picture? Compact Camera FOV = 50 x 35°

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' Image Stitching' - adin


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Image stitching

Image Stitching

Shangliang Jiang

Kate Harrison




Introduction
Introduction

  • Are you getting the whole picture?

    • Compact Camera FOV = 50 x 35°


Introduction1
Introduction

  • Are you getting the whole picture?

    • Compact Camera FOV = 50 x 35°

    • Human FOV = 200 x 135°


Introduction2
Introduction

  • Are you getting the whole picture?

    • Compact Camera FOV = 50 x 35°

    • Human FOV = 200 x 135°

    • Panoramic Mosaic = 360 x 180°


Recognizing panoramas
Recognizing Panoramas

  • 1D Rotations (q)

    • Ordering  matching images


Recognizing panoramas1
Recognizing Panoramas

  • 1D Rotations (q)

    • Ordering  matching images


Recognizing panoramas2
Recognizing Panoramas

  • 1D Rotations (q)

    • Ordering  matching images


Recognizing panoramas3

Recognizing Panoramas

  • 1D Rotations (q)

    • Ordering  matching images


Recognizing panoramas4

Recognizing Panoramas

  • 1D Rotations (q)

    • Ordering  matching images


Recognizing panoramas5

Recognizing Panoramas

  • 1D Rotations (q)

    • Ordering  matching images



Overview
Overview

  • Feature Matching

  • Image Matching

  • Bundle Adjustment

  • Image Compositing

  • Conclusions


Overview1
Overview

  • Feature Matching

  • Image Matching

  • Bundle Adjustment

  • Image Compositing

  • Conclusions


Overview2
Overview

  • Feature Matching

    • SIFT Features

    • Nearest Neighbor Matching

  • Image Matching

  • Bundle Adjustment

  • Image Compositing

  • Conclusions


Sift features
SIFT Features

  • SIFT features are…

    • Geometrically invariant to similarity transforms,

      • some robustness to affine change

    • Photometrically invariant to affine changes in intensity


Overview3
Overview

  • Feature Matching

    • SIFT Features

    • Nearest Neighbor Matching

  • Image Matching

  • Bundle Adjustment

  • Image Compositing

  • Conclusions


Nearest neighbor matching
Nearest Neighbor Matching

  • Find k nearest neighbors for each feature

    • k  number of overlapping images (we use k = 4)

  • Use k-d tree

    • k-d tree recursively bi-partitions data at mean in the dimension of maximum variance

    • Approximate nearest neighbors found in O(nlogn)


Overview4
Overview

  • Feature Matching

    • SIFT Features

    • Nearest Neighbor Matching

  • Image Matching

  • Bundle Adjustment

  • Image Compositing

  • Conclusions


Overview5
Overview

  • Feature Matching

  • Image Matching

  • Bundle Adjustment

  • Image Compositing

  • Conclusions


Overview6
Overview

  • Feature Matching

  • Image Matching

  • Bundle Adjustment

  • Image Compositing

  • Conclusions


Overview7
Overview

  • Feature Matching

  • Image Matching

    • Random Sample Consensus (RANSAC) for Homography

    • Probabilistic model for verification

  • Bundle Adjustment

  • Image Compositing

  • Conclusions


Overview8
Overview

  • Feature Matching

  • Image Matching

    • Random Sample Consensus (RANSAC) for Homography

    • Probabilistic model for verification

  • Bundle Adjustment

  • Image Compositing

  • Conclusions



Ransac for homography1
RANSAC for Homography

  • SIFT features of two similar images


Ransac for homography2
RANSAC for Homography

  • SIFT features common to both images


Ransac for homography3
RANSAC for Homography

  • Select random subset of features (e.g. 6)

  • Compute motion estimate

  • Apply motion estimate to all SIFT features

    • Compute error: feature pairs not described by motion estimate

  • Repeat many times (e.g. 500)

    • Keep estimate with best error



Overview9
Overview

  • Feature Matching

  • Image Matching

    • Random Sample Consensus (RANSAC) for Homography

    • Probabilistic model for verification

  • Bundle Adjustment

  • Image Compositing

  • Conclusions


Probabilistic model for verification
Probabilistic model for verification

  • Potential problem:

    • Two images don’t match…

    • … but RANSAC found a motion estimate

  • Do a quick check to make sure the images do match

    • MAP for inliers vs. outliers





Overview10
Overview

  • Feature Matching

  • Image Matching

    • RANSAC for Homography

    • Probabilistic model for verification

  • Bundle Adjustment

  • Image Compositing

  • Conclusions


Overview11
Overview

  • Feature Matching

  • Image Matching

  • Bundle Adjustment

  • Image Compositing

  • Conclusions


Overview12
Overview

  • Feature Matching

  • Image Matching

  • Bundle Adjustment

  • Image Compositing

  • Conclusions


Overview13
Overview

  • Feature Matching

  • Image Matching

  • Bundle Adjustment

    • Error function

  • Image Compositing

  • Conclusions


Overview14
Overview

  • Feature Matching

  • Image Matching

  • Bundle Adjustment

    • Error function

  • Image Compositing

  • Conclusions


Bundle adjustment
Bundle Adjustment

  • New images initialised with rotation, focal length of best matching image


Bundle adjustment1
Bundle Adjustment

  • New images initialised with rotation, focal length of best matching image


Error function
Error function

  • Sum of squared projection errors

    • n = #images

    • I(i) = set of image matches to image i

    • F(i, j) = set of feature matches between images i,j

    • rijk = residual of kth feature match between images i,j

  • Robust error function


Overview15
Overview

  • Feature Matching

  • Image Matching

  • Bundle Adjustment

    • Error function

  • Image Compositing

  • Conclusions


Overview16
Overview

  • Feature Matching

  • Image Matching

  • Bundle Adjustment

  • Image Compositing

  • Conclusions


Overview17
Overview

  • Feature Matching

  • Image Matching

  • Bundle Adjustment

  • Image Compositing

  • Conclusions




How do we blend
How do we blend?

Linear blending

Multi-band blending


Multi band blending
Multi-band Blending

  • Burt & Adelson 1983

    • Blend frequency bands over range l


2 band blending
2-band Blending

Low frequency (l > 2 pixels)

High frequency (l < 2 pixels)


3 band blending band 1 high frequencies
3-band blendingBand 1: high frequencies


3 band blending band 2 mid range frequencies
3-band blendingBand 2: mid-range frequencies


3 band blending band 3 low frequencies
3-band blendingBand 3: low frequencies


Panorama straightening
Panorama straightening

Heuristic: people tend to shoot pictures in a certain way


Overview18
Overview

  • Feature Matching

  • Image Matching

  • Bundle Adjustment

  • Image Compositing

  • Conclusions


Overview19
Overview

  • Feature Matching

  • Image Matching

  • Bundle Adjustment

  • Image Compositing

  • Conclusions




Autostitch program
AutoStitch program

AutoStitch.net


Open questions
Open questions

  • Advanced camera modeling

    • radial distortion, camera motion, scene motion, vignetting, exposure, high dynamic range, flash …

  • Full 3D case – recognizing 3D objects/scenes in unordered datasets


Credits
Credits

  • Automatic Panoramic Image Stitching Using Invariant Features, 2007

    • Matthew Brown and David G. Lowe (Uni. of British Columbia)

  • Recognising Panoramas, 2003

    • Matthew Brown and David G. Lowe (Uni. of British Columbia)

    • 2003

    • Thanks for the slides!

  • Image Alignment and Stitching: A Tutorial, 2006

    • Richard Szeliski (Microsoft)



ad