stanford cs223b computer vision winter 2006 lecture 8 structure from motion l.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Stanford CS223B Computer Vision, Winter 2006 Lecture 8 Structure From Motion PowerPoint Presentation
Download Presentation
Stanford CS223B Computer Vision, Winter 2006 Lecture 8 Structure From Motion

Loading in 2 Seconds...

play fullscreen
1 / 44

Stanford CS223B Computer Vision, Winter 2006 Lecture 8 Structure From Motion - PowerPoint PPT Presentation


  • 193 Views
  • Uploaded on

Stanford CS223B Computer Vision, Winter 2006 Lecture 8 Structure From Motion. Professor Sebastian Thrun CAs: Dan Maynes-Aminzade, Mitul Saha, Greg Corrado Slides by: Gary Bradski, Intel Research and Stanford SAIL. Structure From Motion. features. camera.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Stanford CS223B Computer Vision, Winter 2006 Lecture 8 Structure From Motion' - brone


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
stanford cs223b computer vision winter 2006 lecture 8 structure from motion

Stanford CS223B Computer Vision, Winter 2006Lecture 8 Structure From Motion

Professor Sebastian Thrun

CAs: Dan Maynes-Aminzade, Mitul Saha, Greg Corrado

Slides by: Gary Bradski, Intel Research and Stanford SAIL

structure from motion
Structure From Motion

features

camera

Recover: structure (feature locations), motion (camera extrinsics)

structure from motion 1
Structure From Motion (1)

[Tomasi & Kanade 92]

structure from motion 2
Structure From Motion (2)

[Tomasi & Kanade 92]

structure from motion 3
Structure From Motion (3)

[Tomasi & Kanade 92]

structure from motion8
Structure From Motion
  • Problem 1:
    • Given n points pij =(xij, yij) in m images
    • Reconstruct structure: 3-D locations Pj =(xj, yj, zj)
    • Reconstruct camera positions (extrinsics) Mi=(Aj, bj)
  • Problem 2:
    • Establish correspondence: c(pij)
bundle adjustment
Bundle Adjustment
  • SFM = Nonlinear Least Squares problem
  • Minimize through
    • Gradient Descent
    • Conjugate Gradient
    • Gauss-Newton
    • Levenberg Marquardt (!)
  • Prone to local minima
count constraints vs unknowns
Count # Constraints vs #Unknowns
  • m camera poses
  • n points
  • 2mn point constraints
  • 6m+3n unknowns
  • Suggests: need 2mn  6m + 3n
  • But: Can we really recover all parameters???
how many parameters can t we recover
How Many Parameters Can’t We Recover?

We can recover all but…

Place Your Bet!

count constraints vs unknowns14
Count # Constraints vs #Unknowns
  • m camera poses
  • n points
  • 2mn point constraints
  • 6m+3n unknowns
  • Suggests: need 2mn  6m + 3n
  • But: Can we really recover all parameters???
    • Can’t recover origin, orientation (6 params)
    • Can’t recover scale (1 param)
  • Thus, we need 2mn  6m + 3n -7
are done
Are done?
  • No, bundle adjustment has many local minima.
the trick of the day
The “Trick Of The Day”
  • Replace Perspective by Orthographic Geometry
  • Replace Euclidean Geometry by Affine Geometry
  • Solve SFM linearly (“closed” form, globally optimal)
  • Post-Process to make solution Euclidean
  • Post-Process to make solution perspective

By Tomasi and Kanade, 1992

orthographic camera model
Orthographic Camera Model

Extrinsic Parameters

Rotation

Orthographic Projection

Limit of Pinhole Model:

orthographic projection
Orthographic Projection

Limit of Pinhole Model:

Orthographic Projection

the affine sfm problem
The Affine SFM Problem

drop the

constraints

subject to

count constraints vs unknowns21
Count # Constraints vs #Unknowns
  • m camera poses
  • n points
  • 2mn point constraints
  • 8m+3n unknowns
  • Suggests: need 2mn  8m + 3n
  • But: Can we really recover all parameters???
how many parameters can t we recover22
How Many Parameters Can’t We Recover?

We can recover all but…

Place Your Bet!

points for solving affine sfm problem
Points for Solving Affine SFM Problem
  • m camera poses
  • n points
  • Need to have: 2mn  8m + 3n-12
affine sfm
Affine SFM

Fix coordinate system

by making p0=origin

Rank Theorem:

Q has rank 3

Proof:

the rank theorem
The Rank Theorem

2m elements

n elements

affine solution to orthographic sfm
Affine Solution to Orthographic SFM

Gives also the optimal affine reconstruction under noise

back to orthographic projection
Back To Orthographic Projection

Find C and d for which constraints are met

Search in 12-dim space (instead of 8m + 3n-12)

back to projective geometry
Back To Projective Geometry

Orthographic (in the limit)

Projective

back to projective geometry31
Back To Projective Geometry

O

X

-x

Z

f

Optimize

Using orthographic solution as starting point

the trick of the day32
The “Trick Of The Day”
  • Replace Perspective by Orthographic Geometry
  • Replace Euclidean Geometry by Affine Geometry
  • Solve SFM linearly (“closed” form, globally optimal)
  • Post-Process to make solution Euclidean
  • Post-Process to make solution perspective

By Tomasi and Kanade, 1992

structure from motion33
Structure From Motion
  • Problem 1:
    • Given n points pij =(xij, yij) in m images
    • Reconstruct structure: 3-D locations Pj =(xj, yj, zj)
    • Reconstruct camera positions (extrinsics) Mi=(Aj, bj)
  • Problem 2:
    • Establish correspondence: c(pij)
the correspondence problem
The Correspondence Problem

View 1

View 2

View 3

correspondence solution 1
Correspondence: Solution 1
  • Track features (e.g., optical flow)
  • …but fails when images taken from widely different poses
correspondence solution 2
Correspondence: Solution 2
  • Start with random solution A, b, P
  • Compute soft correspondence: p(c|A,b,P)
  • Plug soft correspondence into SFM
  • Reiterate

See Dellaert/Seitz/Thorpe/Thrun, Machine Learning Journal, 2003

correspondence alternative approach
Correspondence: Alternative Approach
  • Ransac [Fisher/Bolles]

= Random sampling and consensus

summary sfm
Summary SFM
  • Problem
    • Determine feature locations (=structure)
    • Determine camera extrinsic (=motion)
  • Two Principal Solutions
    • Bundle adjustment (nonlinear least squares, local minima)
    • SVD (through orthographic approximation, affine geometry)
  • Correspondence
    • (RANSAC)
    • Expectation Maximization