stanford cs223b computer vision winter 2006 lecture 8 structure from motion l.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Stanford CS223B Computer Vision, Winter 2006 Lecture 8 Structure From Motion PowerPoint Presentation
Download Presentation
Stanford CS223B Computer Vision, Winter 2006 Lecture 8 Structure From Motion

Loading in 2 Seconds...

play fullscreen
1 / 44

Stanford CS223B Computer Vision, Winter 2006 Lecture 8 Structure From Motion - PowerPoint PPT Presentation


  • 197 Views
  • Uploaded on

Stanford CS223B Computer Vision, Winter 2006 Lecture 8 Structure From Motion. Professor Sebastian Thrun CAs: Dan Maynes-Aminzade, Mitul Saha, Greg Corrado Slides by: Gary Bradski, Intel Research and Stanford SAIL. Structure From Motion. features. camera.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

Stanford CS223B Computer Vision, Winter 2006 Lecture 8 Structure From Motion


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
stanford cs223b computer vision winter 2006 lecture 8 structure from motion

Stanford CS223B Computer Vision, Winter 2006Lecture 8 Structure From Motion

Professor Sebastian Thrun

CAs: Dan Maynes-Aminzade, Mitul Saha, Greg Corrado

Slides by: Gary Bradski, Intel Research and Stanford SAIL

structure from motion
Structure From Motion

features

camera

Recover: structure (feature locations), motion (camera extrinsics)

structure from motion 1
Structure From Motion (1)

[Tomasi & Kanade 92]

structure from motion 2
Structure From Motion (2)

[Tomasi & Kanade 92]

structure from motion 3
Structure From Motion (3)

[Tomasi & Kanade 92]

structure from motion8
Structure From Motion
  • Problem 1:
    • Given n points pij =(xij, yij) in m images
    • Reconstruct structure: 3-D locations Pj =(xj, yj, zj)
    • Reconstruct camera positions (extrinsics) Mi=(Aj, bj)
  • Problem 2:
    • Establish correspondence: c(pij)
bundle adjustment
Bundle Adjustment
  • SFM = Nonlinear Least Squares problem
  • Minimize through
    • Gradient Descent
    • Conjugate Gradient
    • Gauss-Newton
    • Levenberg Marquardt (!)
  • Prone to local minima
count constraints vs unknowns
Count # Constraints vs #Unknowns
  • m camera poses
  • n points
  • 2mn point constraints
  • 6m+3n unknowns
  • Suggests: need 2mn  6m + 3n
  • But: Can we really recover all parameters???
how many parameters can t we recover
How Many Parameters Can’t We Recover?

We can recover all but…

Place Your Bet!

count constraints vs unknowns14
Count # Constraints vs #Unknowns
  • m camera poses
  • n points
  • 2mn point constraints
  • 6m+3n unknowns
  • Suggests: need 2mn  6m + 3n
  • But: Can we really recover all parameters???
    • Can’t recover origin, orientation (6 params)
    • Can’t recover scale (1 param)
  • Thus, we need 2mn  6m + 3n -7
are done
Are done?
  • No, bundle adjustment has many local minima.
the trick of the day
The “Trick Of The Day”
  • Replace Perspective by Orthographic Geometry
  • Replace Euclidean Geometry by Affine Geometry
  • Solve SFM linearly (“closed” form, globally optimal)
  • Post-Process to make solution Euclidean
  • Post-Process to make solution perspective

By Tomasi and Kanade, 1992

orthographic camera model
Orthographic Camera Model

Extrinsic Parameters

Rotation

Orthographic Projection

Limit of Pinhole Model:

orthographic projection
Orthographic Projection

Limit of Pinhole Model:

Orthographic Projection

the affine sfm problem
The Affine SFM Problem

drop the

constraints

subject to

count constraints vs unknowns21
Count # Constraints vs #Unknowns
  • m camera poses
  • n points
  • 2mn point constraints
  • 8m+3n unknowns
  • Suggests: need 2mn  8m + 3n
  • But: Can we really recover all parameters???
how many parameters can t we recover22
How Many Parameters Can’t We Recover?

We can recover all but…

Place Your Bet!

points for solving affine sfm problem
Points for Solving Affine SFM Problem
  • m camera poses
  • n points
  • Need to have: 2mn  8m + 3n-12
affine sfm
Affine SFM

Fix coordinate system

by making p0=origin

Rank Theorem:

Q has rank 3

Proof:

the rank theorem
The Rank Theorem

2m elements

n elements

affine solution to orthographic sfm
Affine Solution to Orthographic SFM

Gives also the optimal affine reconstruction under noise

back to orthographic projection
Back To Orthographic Projection

Find C and d for which constraints are met

Search in 12-dim space (instead of 8m + 3n-12)

back to projective geometry
Back To Projective Geometry

Orthographic (in the limit)

Projective

back to projective geometry31
Back To Projective Geometry

O

X

-x

Z

f

Optimize

Using orthographic solution as starting point

the trick of the day32
The “Trick Of The Day”
  • Replace Perspective by Orthographic Geometry
  • Replace Euclidean Geometry by Affine Geometry
  • Solve SFM linearly (“closed” form, globally optimal)
  • Post-Process to make solution Euclidean
  • Post-Process to make solution perspective

By Tomasi and Kanade, 1992

structure from motion33
Structure From Motion
  • Problem 1:
    • Given n points pij =(xij, yij) in m images
    • Reconstruct structure: 3-D locations Pj =(xj, yj, zj)
    • Reconstruct camera positions (extrinsics) Mi=(Aj, bj)
  • Problem 2:
    • Establish correspondence: c(pij)
the correspondence problem
The Correspondence Problem

View 1

View 2

View 3

correspondence solution 1
Correspondence: Solution 1
  • Track features (e.g., optical flow)
  • …but fails when images taken from widely different poses
correspondence solution 2
Correspondence: Solution 2
  • Start with random solution A, b, P
  • Compute soft correspondence: p(c|A,b,P)
  • Plug soft correspondence into SFM
  • Reiterate

See Dellaert/Seitz/Thorpe/Thrun, Machine Learning Journal, 2003

correspondence alternative approach
Correspondence: Alternative Approach
  • Ransac [Fisher/Bolles]

= Random sampling and consensus

summary sfm
Summary SFM
  • Problem
    • Determine feature locations (=structure)
    • Determine camera extrinsic (=motion)
  • Two Principal Solutions
    • Bundle adjustment (nonlinear least squares, local minima)
    • SVD (through orthographic approximation, affine geometry)
  • Correspondence
    • (RANSAC)
    • Expectation Maximization