Multiple view geometry
This presentation is the property of its rightful owner.
Sponsored Links
1 / 38

Multiple View Geometry PowerPoint PPT Presentation


  • 141 Views
  • Uploaded on
  • Presentation posted in: General

Multiple View Geometry. Marc Pollefeys University of North Carolina at Chapel Hill. Modified by Philippos Mordohai. Outline. 3-D Reconstruction Fundamental matrix estimation Chapters 9 and 10 of “Multiple View Geometry in Computer Vision” by Hartley and Zisserman. Three questions:.

Download Presentation

Multiple View Geometry

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Multiple view geometry

Multiple View Geometry

Marc Pollefeys

University of North Carolina at Chapel Hill

Modified by Philippos Mordohai


Outline

Outline

  • 3-D Reconstruction

  • Fundamental matrix estimation

  • Chapters 9 and 10 of “Multiple View Geometry in Computer Vision” by Hartley and Zisserman


Three questions

Three questions:

  • Correspondence geometry: Given an image point x in the first image, how does this constrain the position of the corresponding point x’ in the second image?

  • (ii)Camera geometry (motion): Given a set of corresponding image points {xi ↔x’i}, i=1,…,n, what are the cameras P and P’ for the two views?

  • (iii)Scene geometry (structure): Given corresponding image points xi ↔x’i and cameras P, P’, what is the position of (their pre-image) X in space?


Epipolar geometry

p

p

L2

L2

m1

m1

m1

C1

C1

C1

M

M

L1

L1

l1

l1

e1

e1

lT1

l2

e2

e2

Canonical representation:

l2

m2

m2

m2

l2

l2

Fundamental matrix (3x3 rank 2 matrix)

C2

C2

C2

Epipolar Geometry

Underlying structure in set of matches for rigid scenes

  • Computable from corresponding points

  • Simplifies matching

  • Allows to detect wrong matches

  • Related to calibration


3d reconstruction of cameras and structure

3D reconstruction of cameras and structure

reconstruction problem:

given xi↔x‘i , compute P,P‘ and Xi

for all i

without additional information possible up to projective ambiguity


Outline of reconstruction

Outline of reconstruction

  • Compute F from correspondences

  • Compute camera matrices from F

  • Compute 3D point for each pair of corresponding points

computation of F

use x‘iFxi=0 equations, linear in coeff. F

8 points (linear), 7 points (non-linear), 8+ (least-squares)

(more on this next class)

computation of camera matrices

use

triangulation

compute intersection of two backprojected rays


Reconstruction ambiguity similarity

Reconstruction ambiguity: similarity


Multiple view geometry

Reconstruction ambiguity: projective


Terminology

Terminology

xi↔x‘i

Original scene Xi

Projective, affine, similarity reconstruction

= reconstruction that is identical to original up to

projective, affine, similarity transformation

Literature: Metric and Euclidean reconstruction

= similarity reconstruction


The projective reconstruction theorem

The projective reconstruction theorem

If a set of point correspondences in two views determine thefundamental matrix uniquely, then the scene and cameras may be reconstructed from these correspondences alone, and any two such reconstructions from these correspondences are projectively equivalent

  • along same ray ofP2, idem for P‘2

two possibilities: X2i=HX1i, or points along baseline

key result:

allows reconstruction from pair of uncalibrated images


Stratified reconstruction

Stratified reconstruction

  • Projective reconstruction

  • Affine reconstruction

  • Metric reconstruction


Multiple view geometry

Projective to affine

(if D≠0)

theorem says up to projective transformation,

but projective with fixed p∞ is affine transformation

can be sufficient depending on application,

e.g. mid-point, centroid, parallellism


Translational motion

Translational motion

points at infinity (not necessarily visible) are fixed for a pure translation

 reconstruction of xi↔xi is on p∞


Scene constraints

Scene constraints

Parallel lines

parallel lines intersect at infinity

reconstruction of corresponding vanishing point yields

point on plane at infinity

3 sets of parallel lines allow to uniquely determine p∞

remark: in presence of noise determining the intersection of parallel lines is a delicate problem

remark: obtaining vanishing point in one image can be sufficient


Multiple view geometry

Scene constraints


Multiple view geometry

Scene constraints


Affine to metric

*

*

projection

constraints

Affine to metric

identify absolute conic

transform so that

then projective transformation relating original and reconstruction is a similarity transformation

in practice, find image of W∞

image w∞back-projects to cone that intersects p∞ in W∞

note that image is independent of particular reconstruction


Multiple view geometry

Affine to metric

given

possible transformation from affine to metric is

proof:

(Cholesky factorization to obtain A)


Orthogonality

Orthogonality

vanishing points corresponding to orthogonal directions

vanishing line and vanishing point corresponding

to plane and normal direction


Known internal parameters

rectangular pixels

square pixels

Known internal parameters


Same camera for all images

Same camera for all images

same intrinsics  same image of the absolute conic

e.g. moving camera

given sufficient images there is in general only one

conic that projects to the same image in all images,

i.e. the absolute conic

This approach is called self-calibration

transfer of IAC:

provides 4 constraints, one more needed


Direct metric reconstruction using

(in general two solutions)

Direct metric reconstruction using ω

approach 1

calibrated reconstruction

approach 2

compute projective reconstruction

back-project w from both images

intersection defines W∞ and its support plane p∞


Direct reconstruction using ground truth

(2 lin. eq. in H-1per view,

3 for two views)

Direct reconstruction using ground truth

use control points XEi with know coordinates

to go from projective to metric

(3 lin. eq. in H per point,

H has 15 d.o.f.)


Reconstruction summary

Reconstruction summary

  • Given two uncalibrated images compute (PM,P‘M,{XMi})

  • (i.e. within similarity of original scene and cameras)

  • Algorithm

  • Compute projective reconstruction (P,P‘,{Xi})

    • Compute F from xi↔x‘i

    • Compute P,P‘ from F

    • Triangulate Xi from xi↔x‘i

  • Rectify reconstruction from projective to metric

    • Direct method: compute H from control points

    • Stratified method:

    • Affine reconstruction: compute p∞

    • Metric reconstruction: compute IAC w


Outline1

Outline

  • 3-D Reconstruction

  • Fundamental matrix estimation

  • Chapters 9 and 10 of “Multiple View Geometry in Computer Vision” by Hartley and Zisserman


Epipolar geometry basic equation

Epipolar geometry: basic equation

separate known from unknown

(data)

(unknowns)

(linear)


The singularity constraint

The singularity constraint

SVD from linearly computed F matrix (rank 3)

Compute closest rank-2 approximation


The minimum case 7 point correspondences

The minimum case – 7 point correspondences

one parameter family of solutions

but F1+lF2 not automatically rank 2


The minimum case impose rank 2

3

F7pts

F

F2

F1

The minimum case – impose rank 2

(obtain 1 or 3 solutions)

(cubic equation)

Compute possible l as eigenvalues of

(only real solutions are potential solutions)


The not normalized 8 point algorithm

~10000

~100

~10000

~100

~10000

~10000

~100

~100

1

Orders of magnitude difference

between column of data matrix

 least-squares yields poor results

!

The NOT normalized 8-point algorithm


The normalized 8 point algorithm

Transform image to ~[-1,1]x[-1,1]

(-1,1)

(1,1)

(0,0)

(-1,-1)

(1,-1)

normalized least squares yields good results(Hartley, PAMI´97)

The normalized 8-point algorithm

(0,500)

(700,500)

(0,0)

(700,0)


Geometric distance

Geometric distance

Gold standard

Sampson error

Symmetric epipolar distance


Gold standard

Gold standard

Maximum Likelihood Estimation

(= least-squares for Gaussian noise)

Initialize: normalized 8-point, (P,P‘) from F, reconstruct Xi

Parameterize:

(overparametrized F=[t]xM)

Minimize cost using Levenberg-Marquardt

(preferably sparse LM, see book)


Multiple view geometry

Gold standard

Alternative, minimal parametrization (with a=1)

(note (x,y,1) and (x‘,y‘,1) are epipoles)

  • problems:

  • a=0

 pick largest of a,b,c,d to fix to 1

  • epipole at infinity

 pick largest of x,y,w and of x’,y’,w’

4x3x3=36 parametrizations!

reparametrize at every iteration, to be sure


  • Login