Introduction à la vision artificielle III

1. Introduction à la vision artificielle III Jean Ponce Email: ponce@di.ens.fr Lecture given by Josef Sivic<Josef.Sivic@ens.fr> Planches après les courssur : http://www.di.ens.fr/~ponce/introvis/lect3.pptx http://www.di.ens.fr/~ponce/introvis/lect3.pdf http://www.di.ens.fr/~ponce/introvis/sbook.pdf Premier exo, du le 10 octobre http://www.di.ens.fr/willow/teaching/introvis13/assignment1/

2. Camera geometryand calibration II Intrinsic and extrinsic parameters Strong (Euclidean) calibration Degenerate configurations What about affine cameras?

3. The Intrinsic Parameters of a Camera Calibration Matrix The Perspective Projection Equation

4. The Extrinsic Parameters of a Camera

5. Explicit Form of the Projection Matrix Note: M is only defined up to scale in this setting!!

6. Theorem (Faugeras, 1993)

7. Projection equation: Geometric Interpretation • Observations: • is the equation of a plane of normal direction a1 • From the projection equation, it is also • the plane defined by: u = 0 • Similarly: • (a2,b2) describes the plane defined by: v = 0 • (a3,b3) describes the plane defined by: • That is the plane passing through the pinhole (z = 0)

8. a1 a2 Geometric Interpretation: The rows of the projection matrix describe the three planes defined by the image coordinate system a3 v C u

9. Other useful geometric properties P p Q: Given an image point p, what is the direction of the corresponding ray in space? A: Q: Can we compute the position of the camera center W? A:

10. Linear Camera Calibration

11. Linear Systems Square system: A x b • unique solution • Gaussian elimination = Rectangular system ?? • underconstrained: • infinity of solutions A x b = • overconstrained: • no solution Minimize |Ax-b| 2

12. How do you solve overconstrained linear equations ??

13. Homogeneous Linear Systems Square system: A x 0 • unique solution: 0 • unless Det(A)=0 = Rectangular system ?? • 0 is always a solution A x 0 = 2 Minimize |Ax| under the constraint |x| =1 2

14. E(x)-E(e1) = xT(UTU)x-e1T(UTU)e1 = 112+ … +qq2-1 > 1(12+ … +q2-1)=0 How do you solve overconstrained homogeneous linear equations ?? The solution is e . 1

15. Example: Line Fitting Problem: minimize with respect to (a,b,d). • Minimize E with respect to d: n • Minimize E with respect to a,b: where • Done !!

16. Note: • Matrix of second moments of inertia • Axis of least inertia

17. Linear Camera Calibration

18. Once M is known, you still got to recover the intrinsic and extrinsic parameters !!! This is a decomposition problem, not an estimation problem. r • Intrinsic parameters • Extrinsic parameters

19. Degenerate Point Configurations Are there other solutions besides M ?? • Coplanar points: (l,m,n)=(P,0,0) or (0,P,0) or (0,0,P ) • Points lying on the intersection curve of two quadric • surfaces = straight line + twisted cubic Does not happen for 6 or more random points!

20. Analytical Photogrammetry Non-Linear Least-Squares Methods • Newton • Gauss-Newton • Levenberg-Marquardt Iterative, quadratically convergent in favorable situations

21. What about Affine Cameras? Weak-Perspective Projection Paraperspective Projection

22. More Affine Cameras Orthographic Projection Parallel Projection

23. r Weak-Perspective Projection Model (p and P are in homogeneous coordinates) p = M P (P is in homogeneous coordinates) p = A P + b (neither p nor P is in hom. coordinates)

24. Weak-Perspective Projection

25. Definition: A 2x4 matrix M = [Ab], where A is a rank-2 2x3 matrix, is called an affine projection matrix. Theorem: All affine projection models can be represented by affine projection matrices.

26. General form of the weak-perspective projection equation: (1) Theorem: An affine projection matrix can be written uniquely (up to a sign amibguity) as a weak perspective projection matrix as defined by (1).

27. Camera calibration - example applications

28. Mobile Robot Localization (Devy et al., 1997)

29. (Rothganger, Sudsang, & Ponce, 2002)