Stanford CS223B Computer Vision, Winter 2005 Lecture 2 Lenses and Camera Calibration

1. Stanford CS223B Computer Vision, Winter 2005Lecture 2 Lenses and Camera Calibration Sebastian Thrun, Stanford Rick Szeliski, Microsoft Hendrik Dahlkamp, Stanford

2. News of the Day • Homework assignment 1 is up(?) • Reading list on the Web • 14 projects on the Web

3. Today’s Goals • Thin Lens • Aberrations • Calibration

4. Pinhole Camera -- Brunelleschi, XVth Century Marc Pollefeys comp256, Lect 2

5. Snell’s Law Snell’s law n1 sina1 = n2 sin a2

6. Thin Lens: Definition focus optical axis f Spherical lense surface: Parallel rays are refracted to single point

7. Thin Lens: Projection optical axis Image plane f z Spherical lense surface: Parallel rays are refracted to single point

8. Thin Lens: Projection optical axis Image plane f f z Spherical lense surface: Parallel rays are refracted to single point

9. Thin Lens: Properties • Any ray entering a thin lens parallel to the optical axis must go through the focus on other side • Any ray entering through the focus on one side will be parallel to the optical axis on the other side

10. Thin Lens: Model Q P O Fr Fl p R Z f f z

11. The Thin Lens Law Q P O Fr Fl p R Z f f z

12. The Thin Lens Law

13. Limits of the Thin Lens Model 3 assumptions : • all rays from a point are focused onto 1 image point • Remember thin lens small angle assumption 2. all image points in a single plane 3. magnification is constant Deviations from this ideal are aberrations

14. Today’s Goals • Thin Lens • Aberrations • Calibration

15. Aberrations 2 types : geometrical : geometry of the lense, small for paraxial rays chromatic : refractive index function of wavelength Marc Pollefeys

16. Geometrical Aberrations • spherical aberration • astigmatism • distortion • coma aberrations are reduced by combining lenses

17. Spherical Aberration rays parallel to the axis do not converge outer portions of the lens yield smaller focal lenghts

18. Astigmatism Different focal length for inclined rays Marc Pollefeys

19. Distortion magnification/focal length different for different angles of inclination pincushion (tele-photo) barrel (wide-angle) Can be corrected! (if parameters are know) Marc Pollefeys

20. Coma point off the axis depicted as comet shaped blob Marc Pollefeys

21. Chromatic Aberration rays of different wavelengths focused in different planes cannot be removed completely Marc Pollefeys

22. Vignetting Effect: Darkens pixels near the image boundary

23. CCD vs. CMOS Mature technology Specific technology High production cost High power consumption Higher fill rate Blooming Sequential readout Recent technology Standard IC technology Cheap Low power Less sensitive Per pixel amplification Random pixel access Smart pixels On chip integration with other components Marc Pollefeys

24. Today’s Goals • Thin Lens • Aberrations • Calibration • Problem definition • Solution with Homogeneous Parameters • Solution by nonlinear Least Squares method • Distortion

25. Intrinsic Camera Parameters • Determine the intrinsic parameters of a camera (with lens) • What are Intrinsic Parameters?

26. Intrinsic Camera Parameters • Determine the intrinsic parameters of a camera (with lens) • Intrinsic Parameters: • Focal Length f • Pixel size sx ,sy • Distortion coefficients k1 ,k2… • Image center ox ,oy

27. A Quiz • Can we determine all intrinsic parameters by … exposing the camera to many known objects?

28. Example Calibration Pattern

29. Another Quiz (the last today) • How Many Flat Calibration Targets are Needed for Calibration? 1: 2: 3: 4: 5: 10 • How Many Corner Points do we need in Total? 1: 2: 3: 4: 10: 20

30. Experiment 1: Parallel Board

31. Projective Perspective of Parallel Board 10cm 20cm 30cm

32. Experiment 2: Tilted Board

33. Projective Perspective of Tilted Board 10cm 20cm 30cm 50cm 100cm 500cm

34. Perspective Camera Model Object Space

35. Calibration: 2 steps • Step 1: Transform into camera coordinates • Step 2: Transform into image coordinates

36. Calibration Model (extrinsic) Homogeneous Coordinates

37. Homogeneous Coordinates • Idea: Most Operations Become Linear! • Extract Image Coordinates by Z-normalization

38. Advantage of Homogeneous C’s i-th data point

39. Calibration Model (intrinsic) Focal length Pixel size Image center

40. Intrinsic Transformation

41. Plugging the Model Together!

42. Summary Parameters • Extrinsic • Rotation • Translation • Intrinsic • Focal length • Pixel size • Image center coordinates • (Distortion coefficients)

43. Q: Can We recover all Intrinsic Params? • No

44. Summary Parameters, Revisited • Focal length, in pixel units • Aspect ratio • Extrinsic • Rotation • Translation • Intrinsic • Focal length • Pixel size • Image center coordinates • (Distortion coefficients)

45. Today’s Goals • Thin Lens • Aberrations • Calibration • Problem definition • Solution with Homogeneous Parameters • Solution by nonlinear Least Squares method • Distortion

46. Calibration a la Trucco • Substitute • Advantage: Equations are linear in params • If over-constrained, minimize Least Mean Square fct • One possible solution: • Enforce constraint that R is rotation matrix • Lots of considerations to recover individual params…

47. Today’s Goals • Thin Lens • Aberrations • Calibration • Problem definition • Solution with Homogeneous Parameters • Solution by nonlinear Least Squares method • Distortion

48. Calibration by nonlinear Least Squares • Calibration Examples: …

49. Calibration by nonlinear Least Squares • Least Squares

50. Calibration by nonlinear Least Squares • Least Mean Square • Gradient descent: