Camera calibration

1. Camera calibration Digital Visual Effects, Spring 2005 Yung-Yu Chuang 2005/4/6 with slides by Richard Szeliski, Steve Seitz, and Marc Pollefyes

2. Announcements • Project #1 artifacts voting. • Project #2 camera.

3. Outline • Nonlinear least square methods • Camera projection models • Camera calibration • Bundle adjustment

4. Nonlinear least square methods

5. It is widely seen in data fitting. Least square

6. is linear, too. Linear least square y t prediction residual

7. Nonlinear least square

8. It is very hard to solve in general. Here, we only consider a simpler problem of finding local minimum. Function minimization Least square is related to function minimization.

9. Function minimization

10. Quadratic functions

11. Quadratic functions isocontour gradient

12. Quadratic functions

13. Descent methods

14. Descent direction

15. It has good performance in the initial stage of the iterative process. Steepest descent method

16. Steepest descent method

17. It has good performance in the final stage of the iterative process. Newton’s method

18. This needs to calculate second-order derivative which might not be available. Hybrid method

19. Line search

20. Levenberg-Marquardt method • LM can be thought of as a combination of steepest descent and the Newton method. When the current solution is far from the correct one, the algorithm behaves like a steepest descent method: slow, but guaranteed to converge. When the current solution is close to the correct solution, it becomes a Newton method.

21. Nonlinear least square

22. Levenberg-Marquardt method

23. Levenberg-Marquardt method

25. Camera projection models

26. Pinhole camera

27. Pinhole camera model origin principal point (optical center)

28. Pinhole camera model

29. Pinhole camera model

30. Principal point offset intrinsic matrix

31. Intrinsic matrix • non-square pixels (digital video) • skew • radial distortion Is this form of K good enough?

32. Camera rotation and translation extrinsic matrix

33. Two kinds of parameters • internal or intrinsic parameters such as focal length, optical center, aspect ratio:what kind of camera? • external or extrinsic (pose) parameters including rotation and translation:where is the camera?

34. Other projection models

35. Orthographic projection • Special case of perspective projection • Distance from the COP to the PP is infinite • Also called “parallel projection”: (x, y, z) → (x, y) Image World

36. Other types of projection • Scaled orthographic • Also called “weak perspective” • Affine projection • Also called “paraperspective”

37. Fun with perspective

38. Perspective cues

39. Perspective cues

40. Fun with perspective Ames room

41. Forced perspective in LOTR

42. Camera calibration

43. Camera calibration • Estimate both intrinsic and extrinsic parameters • Mainly, two categories: • Photometric calibration: use reference objects with known geometry • Self calibration: only assume static scene, e.g. structure from motion

44. Camera calibration approaches • linear regression (least squares) • nonlinear optinization • multiple planar patterns

45. Chromaglyphs (HP research)

46. Linear regression

47. Linear regression • Directly estimate 11 unknowns in the M matrix using known 3D points (Xi,Yi,Zi) and measured feature positions (ui,vi)

48. Linear regression Solve for Projection Matrix M using least-square techniques

49. Normal equation Given an overdetermined system the normal equation is that which minimizes the sum of the square differences between left and right sides

50. Linear regression • Advantages: • All specifics of the camera summarized in one matrix • Can predict where any world point will map to in the image • Disadvantages: • Doesn’t tell us about particular parameters • Mixes up internal and external parameters • pose specific: move the camera and everything breaks