epipolar geometry class 5 l.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Epipolar geometry Class 5 PowerPoint Presentation
Download Presentation
Epipolar geometry Class 5

Loading in 2 Seconds...

play fullscreen
1 / 56

Epipolar geometry Class 5 - PowerPoint PPT Presentation


  • 214 Views
  • Uploaded on

Epipolar geometry Class 5. 3D photography course schedule (tentative). Optical flow. Brightness constancy assumption. (small motion). 1D example. possibility for iterative refinement. Optical flow. Brightness constancy assumption. (small motion). 2D example. the “aperture” problem.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

Epipolar geometry Class 5


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
optical flow
Optical flow
  • Brightness constancy assumption

(small motion)

  • 1D example

possibility for iterative refinement

optical flow4
Optical flow
  • Brightness constancy assumption

(small motion)

  • 2D example

the “aperture” problem

(1 constraint)

?

(2 unknowns)

isophote I(t+1)=I

isophote I(t)=I

optical flow5
Optical flow
  • How to deal with aperture problem?

(3 constraints if color gradients are different)

Assume neighbors have same displacement

lucas kanade
Lucas-Kanade

Assume neighbors have same displacement

least-squares:

revisiting the small motion assumption
Revisiting the small motion assumption
  • Is this motion small enough?
    • Probably not—it’s much larger than one pixel (2nd order terms dominate)
    • How might we solve this problem?

* From Khurram Hassan-Shafique CAP5415 Computer Vision 2003

reduce the resolution
Reduce the resolution!

* From Khurram Hassan-Shafique CAP5415 Computer Vision 2003

coarse to fine optical flow estimation

u=1.25 pixels

u=2.5 pixels

u=5 pixels

u=10 pixels

image It-1

image It-1

image I

image I

Gaussian pyramid of image It-1

Gaussian pyramid of image I

Coarse-to-fine optical flow estimation

slides from

Bradsky and Thrun

coarse to fine optical flow estimation10

warp & upsample

run iterative L-K

.

.

.

image J

image It-1

image I

image I

Gaussian pyramid of image It-1

Gaussian pyramid of image I

Coarse-to-fine optical flow estimation

slides from

Bradsky and Thrun

run iterative L-K

feature tracking
Feature tracking
  • Identify features and track them over video
    • Small difference between frames
    • potential large difference overall
  • Standard approach:

KLT (Kanade-Lukas-Tomasi)

good features to track
Good features to track
  • Use same window in feature selection as for tracking itself
  • Compute motion assuming it is small

Affine is also possible, but a bit harder (6x6 in stead of 2x2)

differentiate:

example
Example

Simple displacement is sufficient between consecutive frames, but not to compare to reference template

good features to keep tracking
Good features to keep tracking

Perform affine alignment between first and last frame

Stop tracking features with too large errors

slide19

Two-view geometry

Three questions:

  • Correspondence geometry: Given an image point x in the first image, how does this constrain the position of the corresponding point x’ in the second image?
  • (ii) Camera geometry (motion): Given a set of corresponding image points {xi ↔x’i}, i=1,…,n, what are the cameras P and P’ for the two views?
  • (iii) Scene geometry (structure): Given corresponding image points xi ↔x’i and cameras P, P’, what is the position of (their pre-image) X in space?
slide20

The epipolar geometry

C,C’,x,x’ and X are coplanar

slide21

The epipolar geometry

What if only C,C’,x are known?

slide22

The epipolar geometry

All points on p project on l and l’

slide23

The epipolar geometry

Family of planes p and lines l and l’

Intersection in e and e’

slide24

The epipolar geometry

epipoles e,e’

= intersection of baseline with image plane

= projection of projection center in other image

= vanishing point of camera motion direction

an epipolar plane = plane containing baseline (1-D family)

an epipolar line = intersection of epipolar plane with image

(always come in corresponding pairs)

slide26

Example: motion parallel with image plane

(simple for stereo  rectification)

slide28

The fundamental matrix F

algebraic representation of epipolar geometry

we will see that mapping is (singular) correlation (i.e. projective mapping from points to lines) represented by the fundamental matrix F

slide29

The fundamental matrix F

geometric derivation

mapping from 2-D to 1-D family (rank 2)

slide30

The fundamental matrix F

algebraic derivation

(note: doesn’t work for C=C’  F=0)

slide31

The fundamental matrix F

correspondence condition

The fundamental matrix satisfies the condition that for any pair of corresponding points x↔x’ in the two images

slide32

The fundamental matrix F

F is the unique 3x3 rank 2 matrix that satisfies x’TFx=0 for all x↔x’

  • Transpose: if F is fundamental matrix for (P,P’), then FT is fundamental matrix for (P’,P)
  • Epipolar lines: l’=Fx & l=FTx’
  • Epipoles: on all epipolar lines, thus e’TFx=0, x e’TF=0, similarly Fe=0
  • F has 7 d.o.f. , i.e. 3x3-1(homogeneous)-1(rank2)
  • F is a correlation, projective mapping from a point x to a line l’=Fx (not a proper correlation, i.e. not invertible)
slide35

Fundamental matrix for pure translation

General motion

Pure translation

for pure translation F only has 2 degrees of freedom

slide36

The fundamental matrix F

relation to homographies

valid for all plane homographies

slide37

The fundamental matrix F

relation to homographies

requires

slide38

Projective transformation and invariance

Derivation based purely on projective concepts

F invariant to transformations of projective 3-space

unique

not unique

canonical form

slide39

~ ~

Show that if F is same for (P,P’) and (P,P’),

there exists a projective transformation H so that

P=HP and P’=HP’

~

~

Projective ambiguity of cameras given F

previous slide: at least projective ambiguity

this slide: not more!

lemma:

(22-15=7, ok)

slide40

The projective reconstruction theorem

If a set of point correspondences in two views determine thefundamental matrix uniquely, then the scene and cameras may be reconstructed from these correspondences alone, and any two such reconstructions from these correspondences are projectively equivalent

allows reconstruction from pair of uncalibrated images!

slide41

p

p

L2

L2

m1

m1

m1

C1

C1

C1

M

M

L1

L1

l1

l1

e1

e1

lT1

l2

e2

e2

Canonical representation:

l2

m2

m2

m2

l2

l2

Fundamental matrix (3x3 rank 2 matrix)

C2

C2

C2

Epipolar geometry

Underlying structure in set of matches for rigid scenes

  • Computable from corresponding points
  • Simplifies matching
  • Allows to detect wrong matches
  • Related to calibration
epipolar geometry
Epipolar geometry?

courtesy Frank Dellaert

slide43

Other entities besides points?

Lines give no constraint for two view geometry

(but will for three and more views)

Curves and surfaces yield some constraints related to tangency

(e.g. Sinha et al. CVPR’04)

computation of f
Computation of F
  • Linear (8-point)
  • Minimal (7-point)
  • Robust (RANSAC)
  • Non-linear refinement (MLE, …)
  • Practical approach
slide45

Epipolar geometry: basic equation

separate known from unknown

(data)

(unknowns)

(linear)

slide46

~10000

~100

~10000

~100

~10000

~10000

~100

~100

1

Orders of magnitude difference

between column of data matrix

 least-squares yields poor results

!

the NOT normalized 8-point algorithm

slide47

(0,500)

(700,500)

(-1,1)

(1,1)

(0,0)

(0,0)

(700,0)

(-1,-1)

(1,-1)

the normalized 8-point algorithm

Transform image to ~[-1,1]x[-1,1]

normalized least squares yields good results(Hartley, PAMI´97)

slide48

the singularity constraint

SVD from linearly computed F matrix (rank 3)

Compute closest rank-2 approximation

slide50

the minimum case – 7 point correspondences

one parameter family of solutions

but F1+lF2 not automatically rank 2

slide51

3

F7pts

F

F2

F1

the minimum case – impose rank 2

(obtain 1 or 3 solutions)

(cubic equation)

Compute possible l as eigenvalues of

(only real solutions are potential solutions)

slide52

(generate

hypothesis)

(verify hypothesis)

Automatic computation of F

Step 1. Extract features

Step 2. Compute a set of potential matches

Step 3. do

Step 3.1 select minimal sample (i.e. 7 matches)

Step 3.2 compute solution(s) for F

Step 3.3 determine inliers

until (#inliers,#samples)<95%

RANSAC

Step 4. Compute F based on all inliers

Step 5. Look for additional matches

Step 6. Refine F based on all correct matches

slide53

Finding more matches

restrict search range to neighborhood of epipolar line

(e.g. 1.5 pixels)

relax disparity restriction (along epipolar line)

slide54

Issues:

  • (Mostly) planar scene (see next slide)
  • Absence of sufficient features (no texture)
  • Repeated structure ambiguity
  • Robust matcher also finds
  • support for wrong hypothesis
  • solution: detect repetition

(Schaffalitzky and Zisserman,

BMVC‘98)

computing f for quasi planar scenes qdegsac
Computing F for quasi-planar scenes QDEGSAC

337 matches on plane, 11 off plane

#inliers

%inclusion of

out-of-plane inliers

17% success for RANSAC

100% for QDEGSAC

data rank

slide56
geometric relations between two views is fully

described by recovered 3x3 matrix F

two-view geometry