Announcements

1. Announcements • Email list – let me know if you have NOT been getting mail • Assignment hand in procedure • web page • at least 3 panoramas: • (1) sample sequence, (2) w/Kaidan, (3) handheld • hand procedure will be posted online • Readings for Monday (via web site) • Seminar on Thursday, Jan 18 Ramin Zabih: “Energy Minimization for Computer Vision via Graph Cuts “ • Correction to radial distortion term

2. Projective geometry • Readings • Mundy, J.L. and Zisserman, A., Geometric Invariance in Computer Vision, Chapter 23: Appendix: Projective Geometry for Machine Vision, MIT Press, Cambridge, MA, 1992, pp. 463-534  (for this week, read  23.1 - 23.5, 23.10) • available online: http://www.cs.cmu.edu/~ph/869/papers/zisser-mundy.pdf Ames Room

3. Projective geometry—what’s it good for? • Uses of projective geometry • Drawing • Measurements • Mathematics for projection • Undistorting images • Focus of expansion • Camera pose estimation, match move • Object recognition via invariants • Today: single-view projective geometry • Projective representation • Point-line duality • Vanishing points/lines • Homographies • The Cross-Ratio • Later: multi-view geometry

4. Applications of projective geometry • Criminisi et al., “Single View Metrology”, ICCV 1999 • Other methods • Horry et al., “Tour Into the Picture”, SIGGRAPH 96 • Shum et al., CVPR 98 • ... Vermeer’s Music Lesson

5. 4 3 2 1 1 2 3 4 Measurements on planes Approach: unwarp then measure What kind of warp is this? • A Homography

6. Image rectification p’ p • To unwarp (rectify) an image • solve for homography H given p and p’ • solve equations of the form: wp’’ = Hp • linear in unknowns: w and coefficients of H • H is defined up to an arbitrary scale factor • can assume H[2,2] = 1 • how many points are necessary to solve for H? work out on board

7. (x,y,1) image plane The projective plane • Why do we need homogeneous coordinates? • represent points at infinity, homographies, perspective projection, multi-view relationships • What is the geometric intuition? • a point in the image is a ray in projective space y (sx,sy,s) (0,0,0) x z • Each point(x,y) on the plane is represented by a ray(sx,sy,s) • all points on the ray are equivalent: (x, y, 1)  (sx, sy, s)

8. A line is a plane of rays through origin • all rays (x,y,z) satisfying: ax + by + cz = 0 l p • A line is also represented as a homogeneous 3-vector l Projective lines • What is a line in projective space?

9. l1 p l l2 Point and line duality • A line l is a homogeneous 3-vector (a ray) • It is  to every point (ray) p on the line: lp=0 p2 p1 • What is the line l spanned by rays p1 and p2 ? • l is  to p1 and p2  l = p1p2 • l is the plane normal • What is the intersection of two lines l1 and l2 ? • p is  to l1 and l2  p = l1l2 • Points and lines are dual in projective space • every property of points also applies to lines

10. (a,b,0) y y (sx,sy,0) z z image plane image plane x x • Ideal line • l  (a, b, 0) – parallel to image plane • Corresponds to a line in the image (finite coordinates) Ideal points and lines • Ideal point (“point at infinity”) • p  (x, y, 0) – parallel to image plane • It has infinite image coordinates

11. Homographies of points and lines • Computed by 3x3 matrix multiplication • To transform a point: p’ = Hp • To transform a line: lp=0  l’p’=0 • 0 = lp = lH-1Hp = lH-1p’  l’ = lH-1 • lines are transformed bypremultiplication of H-1

12. 3D projective geometry • These concepts generalize naturally to 3D • Homogeneous coordinates • Projective 3D points have four coords: P = (X,Y,Z,W) • Duality • A plane N is also represented by a 4-vector • Points and planes are dual in 3D: N P=0 • Projective transformations • Represented by 4x4 matrices T: P’ = TP, N’ = NT-1 • However • Can’t use cross-products in 4D. We need new tools • Grassman-Cayley Algebra • generalization of cross product, allows interactions between points, lines, and planes via “meet” and “join” operators • Won’t get into this stuff today

13. 3D to 2D: “Perspective” Projection • Matrix Projection: • Preserves • Lines • Incidence • Does not preserve • Lengths • Angles • Parallelism

14. vanishing point Vanishing Points • Vanishing point • projection of a point at infinity image plane camera center ground plane

15. vanishing point Vanishing Points (2D) image plane camera center line on ground plane

16. line on ground plane Vanishing Points • Properties • Any two parallel lines have the same vanishing point • The ray from C through v point is parallel to the lines • An image may have more than one vanishing point image plane vanishing point V camera center C line on ground plane

17. v1 v2 Vanishing Lines • Multiple Vanishing Points • Any set of parallel lines on the plane define a vanishing point • The union of all of these vanishing points is the horizon line • also called vanishing line • Note that different planes define different vanishing lines

18. Vanishing Lines • Multiple Vanishing Points • Any set of parallel lines on the plane define a vanishing point • The union of all of these vanishing points is the horizon line • also called vanishing line • Note that different planes define different vanishing lines

19. Computing Vanishing Points • Properties • Pis a point at infinity, v is its projection • They depend only on line direction • Parallel lines P0 + tD, P1 + tD intersect at X V P0 D

20. Computing Vanishing Lines • Properties • l is intersection of horizontal plane through C with image plane • Compute l from two sets of parallel lines on ground plane • All points at same height as C project to l • Provides way of comparing height of objects in the scene C l ground plane

21. Fun With Vanishing Points

22. Perspective Cues

23. Perspective Cues

24. Perspective Cues

25. Comparing Heights Vanishing Point

26. Measuring Height 5.4 5 Camera height 4 3.3 3 2.8 2 1

27. Measuring Height Without a Ruler Y C ground plane • Compute Y from image measurements • Need more than vanishing points to do this

28. The Cross-Ratio of 4 Collinear Points P4 P3 P2 P1 The Cross Ratio • A Projective Invariant • Something that does not change under projective transformations (including perspective projection) • Can permute the point ordering • 4! = 24 different orders (but only 6 distinct values) • This is the fundamental invariant of projective geometry

29. Scene Cross Ratio t vL C Image Cross Ratio b vZ Measuring Height  T (top of object) L (height of camera on the this line) Y B (bottom of object) ground plane

30. Algebraic Derivation • Eliminating  and  yields • Can calculate  given a known height in scene Measuring Height t Y C b reference plane

31. So Far • Can measure • Positions within a plane • Height • More generally—distance between any two parallel planes • These capabilities provide sufficient tools for single-view modeling • To do: • Compute camera projection matrix

32. Vanishing Points and Projection Matrix = vx (X vanishing point) • Not So Fast! We only know v’s up to a scale factor • Can fully specify by providing 3 reference lengths

33. 3D Modeling from a Photograph

34. 3D Modeling from a Photograph

35. Conics • Conic (= quadratic curve) • Defined by matrix equation • Can also be defined using the cross-ratio of lines • “Preserved” under projective transformations • Homography of a circle is a… • 3D version: quadric surfaces are preserved • Other interesting properties (Sec. 23.6 in Mundy/Zisserman) • Tangency • Bipolar lines • Silhouettes