CS 395/495-26: Spring 2003

1 / 25

CS 395/495-26: Spring 2003 - PowerPoint PPT Presentation

CS 395/495-26: Spring 2003. IBMR: Week 4A from Chapter 3: Point Correspondence and Direct Linear Transform (DLT) Jack Tumblin jet@cs.northwestern.edu. Proj_A Workaround. Programming ‘Pick &amp; Drag’ is messy! KeyDown, KeyUp is easy (see ProjA_06) We have only a few points to drag;

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.

PowerPoint Slideshow about 'CS 395/495-26: Spring 2003' - cybele

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
CS 395/495-26: Spring 2003

IBMR: Week 4A

from Chapter 3:

Point Correspondence and Direct Linear Transform (DLT)

Jack Tumblin

jet@cs.northwestern.edu

Proj_A Workaround
• Programming ‘Pick & Drag’ is messy!
• KeyDown, KeyUp is easy (see ProjA_06)

We have only a few points to drag;

• Use number keys to select points:
• ‘1’ for 1st point, ‘2’ for 2nd point, etc.
• ‘SHIFT 1’ for 1st line, ‘SHIFT 2’ for 2nd line...
• Use arrow keys to change point / line coords:
• left/right for x1 value,
• up/down for x2 value,
• SHIFT up/down for x3 value, etc.
Most Important Ideas

SVD solves Null-Space problems Ax = 0(‘flatten, stack, find null space’ to solve)

• MANY problems can be written this way
• Expandable! --more data points tall, ‘overconstrained’ A matrix --SVD finds optimal ‘least squares’ solution.

Robustness: bane of all inverse methodsAccuracy vs. # of measurements (more isn’t always better! Outliers...) Data-rich images: quantity easier than qualitymore is easier than better)

?How can more measurements reduce error?

Recall: Rectification Undo parts of H

x’ = H x where H = HS HA HP

GOAL: Put world plane x’ into view plane x

• Affine Rect.; (find only HP(2DOF));
• Metric Rect.; (find HA and HP (6DOF));
• Full Rect.; (find all: HSHAHP (8DOF));

METHODS:

• Affine: ‘Vanishing Point’, Horizon line methods
• Metric: Conics & Circular Points (!?!?)
• Full: 4-point correspondence
Finding H from Image Space

L

M

• Affine Rect; (find only HP(2DOF)); 2 parallel line pairs2 points on horizon line
• Metric Rect. (find HA and HP (4DOF));

5 perpendicular line pairs C*’

• Full Rect.; (find all: HSHAHP (8DOF)); 4 (or more point pairs)  all H coeffs

L

m

HS HA HP

2D image space

2D world space

Recall: Projective Transform H

x

y

Apply the 3x3 matrix Hx’ = Hx

2D image (x,y)Homog. coords [x,y,w]T = x w=1

Homog. coordsx’ =[x’,y’,w’]Tw’=1 2D image (x’,y’)

x

x

x2

(x/w,y/w)

y

x

y

y’

y

x3

x

x’

x’

y

x1

(x’/w’,y’/w’)

x

Recall: Projective Transform: Hx = x’

h11 x + h12 y + h13

h31 x + h32 y + h33

h21 x + h22 y + h21

h31 x + h32 y + h33

x’2

x’3

x’1

x’3

H defines point pairs (x,x’) (correspondences)

h11 h12 h13

h21 h22 h21

h31 h32 h33

x1

x2

x3

x’1

x’2

x’3

x1

x2

x3

x

y

w

Input (or output) image is on central plane

=

=

let w=1

x’ = =

y’ = =

Write R2 expressions;(for each x,x’ pair)

x’ (h31 x + h32 y + h33) = (h11 x + h12 y + h13)

Rearrange, solve as a matrix problem…

y’ (h31 x + h32 y + h33) = (h11 x + h12 y + h13)

Naïve Method: (pg 13-14)

h11

h12

h13

h21

h22

h23

h31

h32

h33

4 point correspondence:

Book shows:

Rearrange: known vector (dot) unknown vector

stack, solve for null space…

But this assumes x1=x, x2=y, x3=w=1,

There is a better way (DLT)…

x’ (h31 x + h32 y + h33) = (h11 x + h12 y + h13)

y’ (h31 x + h32 y + h33) = (h11 x + h12 y + h13)

[x y 1 0 0 0 -x’x -x’y -x’] = 0

[0 0 0 x y 1 -y’x -y’y -y’]

Direct Linear Tranform (DLT)
• Naïve method: Solves Hx – x’ = 0
• Requires constant scale (x3=1 in measurement)
• compare:
• include the w, w’ terms:
• look what happens when w’0!

h11

h12

h13

h21

h22

h23

h31

h32

h33

[ 0 0 0 x y 1 -y’x -y’y -y’ ]

[ x y 1 0 0 0 -x’x -x’y -x’ ]

[ 0 0 0 -w’x -w’y -w’w y’x y’y y’w] = 0

[w’x w’y w’w 0 0 0 -x’x -x’y -x’w]

Direct Linear Tranform (DLT)
• Naïve method: Solves Hx – x’ = 0
• Requires constant scale (x3=1 in measurement)
• compare:
• DLT method: Solve Hx  x’ = 0
• (a different identity: x’  x’ = 0)
• Accepts any scale (x3=w1 OK, xx3)

h11

h12

h13

h21

h22

h23

h31

h32

h33

[ 0 0 0 x y 1 -y’x -y’y -y’ ]

[ x y 1 0 0 0 -x’x -x’y -x’ ]

[ 0 0 0 -w’x -w’y -w’w y’x y’y y’w] = 0

[w’x w’y w’w 0 0 0 -x’x -x’y -x’w]

Direct Linear Tranform (DLT)
• Naïve method: Solve Hx – x’ = 0
• Requires constant scale (x3=1 always)
• compare:
• DLT method: Solve Hx  x’ = 0
• Accepts any scale (x3=w1, x3’=w’1)

h1

h2

h3

[ 0 0 0 x y 1 -y’x -y’y -y’ ]

[ x y 1 0 0 0 -x’x -x’y -x’ ]

[ 0 0 0 -w’x -w’y -w’w y’x y’y y’w] = 0

[w’x w’y w’w 0 0 0 -x’x -x’y -x’w]

[ 0T -w’i xiT y’I xiT ]

[w’i xiT 0T -x’i xiT ]

Direct Linear Tranform (DLT)
• DLT method: Solve H  x = 0
• Accepts any scale, any point (x3=w1 is OK)
• ‘Pure’ and Compatible -- P2terms only, no R2
• Much better suited to error measurements.
• Subtlety:
• Won’t divide-by-zero if w=0 or h33=0(it happens!)
• has a 3rd row; it is not degenerate if w’=0
• OK to use it…(Solve 8x12)

0T -w’i xiTy’I xiT

w’i xiT0T -x’i xiT =0

-y’i xiT -x’i xiT 0T

h1

h2

h3

Deceptive ‘Robustness’
• Suppose we have 4 pt-correspondences:
• Use DLT to write 8x9 matrix A: Ah=0
• Solve for h null space. ALWAYS gives H matrix
• But what if points are bad / fictional?
• 3 collinear input pts, crooked out: IMPOSSIBLE!
• Yet we get an H solution! Why?
• A matrix rank is 7 or 6 rank 1 H result(s)
• ‘Null Space’ of A may contain >1 H solution!
• ‘Degenerate’ H solution of form aH1 + bH2…
Actual ‘Robustness’
• Vectorizing (‘Flatten, Stack, Null Space’ method) works for almost ANY input! (Points, lines, planes, ||, , conics, quadrics, cross-ratios, vanishing points, twisted cubics…
• Use DLT formulation: (Hx  x’) = 0
• Rearrange as dot product: (known)·(unknown) = 0
• Be careful to have ENOUGH constraints

tricky when you mix types: points, lines,… (pg 75)

= 0

measured output

measured inputtransformed

How can we use >4 point correspondences?

• Easy:
• Add more rows to our 8x9 matrix A: A·h = 0
• Use SVD to find Null space (Always gives an answer!)
• Result: ‘Least squares’ solution; error=A·h = 
• minimizes || A h ||2 = i 2i where,iis error for i-th pt. correspondence:
• i = || Hxi xi’ ||2 = || (2 rows of A)*h ||2 = ‘algebraic distance’
• ‘Algebraic Distance’ ? No geometric meaning!

( - )2+ ( - )2

d(a,b)2 =

b2

b3

b1

b3

a1

a3

a2

a3

• 2D‘Algebraic Distance’ ?No geometric meaning!
• 2D ‘Geometric Distance’ d(a,b)2is Better: measurable length in input or output space

if a = (a1 a2 a3) and b = (b1 b2 b3), then define

Turns out that:

d(a,b)2 = dalgebraic(a,b)

a3·b3

(Not very surprising)

Overall Strategy:

• Collect extra measurements (>4 point pairs, etc. …)
• expect errors; call them ‘estimates’ x
• Compute a 1st solution (probably by SVD)
• Compute error d(Hx, x’)2, and use this to…
• ‘Tweak’ answer Hand estimates x
• Stop when error < useful threshold

^

^

^

^

Using Estimates
• Simplest: ‘one image’ transfer method:
• Assume inputs are a perfect test pattern:

only output pts are in error

• Adjust output estimates x’ until d(Hx, x’)20

(note we re-compute H as x’ changes)

• Better: ‘Symmetric’ transfer method:
• Assume BOTH inputs and outputs have error.
• Adjust BOTH input and output ests x x’(note we re-compute H as x, x’ change)
• Stop when d(Hx,x’)2 + d(H-1x’, x)20

^

^

^

^

^ ^ ^ ^

DLT: Four-Point Correspondence
• 4 point correspondence:
• Book shows:
• Rearrange: known vector (dot) unknown vector
• stack, solve for null space…

x’ (h31 x + h32 y + h33) = (h11 x + h12 y + h13)

y’ (h31 x + h32 y + h33) = (h11 x + h12 y + h13)

[x y 1 0 0 0 -x’x -x’y -x’] = 0

[0 0 0 x y 1 -y’x -y’y -y’]

h11

h12

h13

h21

h22

h23

h31

h32

h33

Direct Linear Tranform (DLT)
• Project 2 method: Solve Hx – x’ = 0
• Requires constant scale (x3=1 in measurement)
• compare:
• DLT method: Solve Hx  x’ = 0
• Accepts any scale (x3=w1 OK, xx3)

h11

h12

h13

h21

h22

h23

h31

h32

h33

[ 0 0 0 x y 1 -y’x -y’y -y’ ]

[ x y 1 0 0 0 -x’x -x’y -x’ ]

[ 0 0 0 -w’x -w’y -w’w y’x y’y y’w] = 0

[w’x w’y w’w 0 0 0 -x’x -x’y -x’w]

Direct Linear Tranform (DLT)
• Project 2 method: Solve Hx – x’ = 0
• Requires constant scale (x3=1 always)
• compare:
• DLT method: Solve Hx  x’ = 0
• Accepts any scale (x3=w1, x3’=w’1)

h1

h2

h3

[ 0 0 0 x y 1 -y’x -y’y -y’ ]

[ x y 1 0 0 0 -x’x -x’y -x’ ]

[ 0 0 0 -w’x -w’y -w’w y’x y’y y’w] = 0

[w’x w’y w’w 0 0 0 -x’x -x’y -x’w]

[ 0T -w’i xiT y’I xiT ]

[w’i xiT 0T -x’i xiT ]

Direct Linear Tranform (DLT)
• DLT method: Solve H  x = 0
• Accepts any scale, any point (x3=w1 is OK)
• ‘Pure’, Compatible -- P2terms only
• Much better suited to error measurements.
• Subtlety:
• Won’t divide-by-zero if w=0 or h33=0(it happens!)
• has a 3rd row; it is not degenerate if w’=0
• OK to use it…(Solve 8x12)

0T -w’i xiTy’I xiT

w’i xiT0T -x’i xiT =0

-y’i xiT -x’i xiT 0T

h1

h2

h3

Deceptive ‘Robustness’
• Suppose we have 4 pt-correspondences:
• Use DLT to write 8x9 (or 12x9) matrix A: Ah=0
• Solve for h null space. ALWAYS gives H matrix
• But what if points are bad / fictional?
• 3 collinear input pts, crooked out: IMPOSSIBLE!
• Yet we get an H solution! Why?
• A matrix rank is 7 or 6 rank 1 H result(s)
• ‘Null Space’ of A may contain >1 H solution!
• ‘Degenerate’ H solution of form aH1 + bH2…