Multi-linear Systems and Invariant Theory in the Context of Computer Vision and Graphics CS329

1. Multi-linear Systems and Invariant Theory in the Context of Computer Vision and Graphics CS329 Amnon Shashua

2. Material We Will Cover Today • The structure of 3D->2D projection matrix • The homography matrix • A primer on projective geometry of the plane

3. The Structure of a Projection Matrix

4. The Structure of a Projection Matrix

5. The Structure of a Projection Matrix

6. The Structure of a Projection Matrix Generally, is called the “principle point” is “aspect ratio” is called the “skew”

7. The Camera Center such that has rank=3, thus Why is the camera center?

8. The Camera Center Why is the camera center? Consider the “optical ray” All points along the line are mapped to the same point is a ray through the camera center

9. The Epipolar Points

10. Choice of Canonical Frame is the new world coordinate frame We have 15 degrees of freedom (16 upto scale) Choose W such that

11. Choice of Canonical Frame Let We are left with 4 degrees of freedom (upto scale):

12. Choice of Canonical Frame

13. Choice of Canonical Frame where are free variables

14. Projection Matrices Let be the image of point at frame number j where are free variables are known

15. Family of Homography Matrices Stands for the family of 2D projective transformations between two fixed images induced by a plane in space The remainder of this class is about making the above statement intelligible

16. Family of Homography Matrices Recall, 3D->2D from Euclidean world frame to image world frame to first camera frame Let K,K’ be the internal parameters of camera 1,2 and choose canonical frame in which R=I and T=0 for first camera.

17. Family of Homography Matrices Recall that 3rd row of K is

18. Family of Homography Matrices Assume are on a planar surface

19. Family of Homography Matrices and Let where the matching pair Image-to-image mapping are induced by a planar surface.

20. Family of Homography Matrices when first camera frame is the homography matrix induced by the plane at infinity

21. Family of Homography Matrices is the epipole in the second image

22. Relationship Between two Homography Matrices is the projection of onto the first image

23. Estimating the Homography Matrix  How many matching points? p’ p 4 points make a basis for the projective plane

24. Projective Geometry of the Plane Equation of a line in the 2D plane The line is represented by the vector and Correspondence between lines and vectors are not 1-1 because represents the same line The vector does represent any line. Two vectors differing by a scale factor are equivalent. This equivalence class is called homogenous vector. Any vector is a representation of the equivalence class.

25. Projective Geometry of the Plane A point lies on the line (coincident with) which is represented by iff But also represents the point represents the point The vector does represent any point. Points and lines are dual to each other (only in the 2D case!).

26. Projective Geometry of the Plane note:

27. Lines and Points at Infinity Consider lines with infinitely large coordinates which represents the point All meet at the same point

28. Lines and Points at Infinity The points lie on a line is called the line at infinity The line The points are called ideal points. A line meets at (which is the direction of the line)

29. A Model of the Projective Plane ideal point is the plane Points are represented as lines (rays) through the origin Lines are represented as planes through the origin

30. A Model of the Projective Plane ={lines through the origin in } } ={1-dim subspaces of

31. Projective Transformations in The study of properties of the projective plane that are invariant under a group of transformations. Projectivity: that maps lines to lines (i.e. preserves colinearity) Any invertible 3x3 matrix is a Projectivity: Let Colinear points, i.e. the points lie on the line is the dual. is called homography, colineation

32. Projective Transformations in perspectivity A composition of perspectivities from a plane to other planes and back to is a projectivity. Every projectivity can be represented in this way.

33. Projective Transformations in Example, a prespectivity in 1D: Lines adjoining matching points are concurrent Lines adjoining matching points (a,a’),(b,b’),(c,c’) are not concurrent

34. Projective Transformations in is not invariant under H: Points on are is not necessarily 0 Parallel lines do not remain parallel ! is mapped to

35. Projective Basis A Simplex in is a set of n+2 points such that no subset Of n+1 of them lie on a hyperplane (linearly dependent). a Simplex is 4 points In Theorem: there is a unique colineation between any two Simplexes

36. Why do we need 4 points Invariants are measurements that remain fixed under colineations # of independent invariants = # d.o.f of configuration - # d.o.f of trans. H has 3 d.o.f Ex: 1D case A point in 1D is represented by 1 parameter. 4 points we have: 4-3=1 invariant (cross ratio) 2D case: H has 8 d.o.f, a point has 2 d.o.f thus 5 points induce 2 invariants

37. Why do we need 4 points The cross-ratio of 4 points: 24 permutations of the 4 points forming 6 groups:

38. Why do we need 4 points 5 points gives us 10 d.o.f, thus 10-8=2 invariants which represent 2D are the 4 basis points (simplex) are determined uniquely by Point of intersection is preserved under projectivity (exercise) uniquely determined