Calibration

1. Calibration ECE 847:Digital Image Processing Stan Birchfield Clemson University

2. Image projection • Project world point (x,y,z) to image point (u,v) • Assume pinhole camera model

3. Image projection (u,v) image point (x,y,z) world point internal calibration external calibration (pose) Total parameters: 11 (5 internal, 6 external)

4. Internal calibration parameters • fx: horizontal focal length • fy: vertical focal length • q: skew (angle b/w horizontal and vertical axes – p/2 for most real cameras • (u0, v0): principal point (intersection of optical axis with image plane) where

5. Two approaches to calibration • Calibrate to meaningful fixed world coord system • Solve for P directly • Good for fixed camera, specific application (e.g., tracking vehicles on highway, mobile robot on ground plane) • But gives no insight into internal calibration parameters • If camera-world relationship changes, calibration must start from scratch • Compute internal and external parameters separately • P = A[R t] • Internal parameters turn camera into a metric device • Can now be used for computing 3D rays (and points) in Euclidean space • Necessary for SFM

6. Homography mapping from world plane

7. Bird’s eye view • Use normalized DLT to find H • Now can unwarp image to provide bird’s eye (top-down) view of scene • This upgrades projective to Euclidean

8. Stratification of geometries • Euclidean • Similarity • Affine • Projective single known length identify circular points identify l∞ more transformations, fewer invariants

9. Calibration algorithms • Tsai 1987 • Uses some known parameters from camera specs • Requires 3D calibration target with orthogonal planes • Zhang 2000 • Uses projective geometry • Only requires planar calibration target • Much simpler

10. Zhang’s algorithm • Based on the image of the absolute conic (IAC): • Algorithm at a glance: • Find w∞ • Decompose w∞ to get K • What is the IAC?

11. Celestial sphere • Celestial sphere is imaginary sphere with infinite radius • Points on sphere(stars) unaffectedby translation http://www.oneminuteastronomer.com/wp-content/uploads/2009/06/celestial-sphere.jpg

12. Plane at infinity • p∞ is idealized celestial sphere • points at infinity lie on p∞ =(0,0,0,1)T • last coordinate is zero • invariant to translation

13. Line at infinity • l∞ is analogous quantity in 2D • points at infinity lie on l∞ , capture direction Euclidean plane l∞ =(0,0,1)T

14. Absolute points (2D) • Two special points on l∞ : (1, ±i, 0) • These are the absolute points, or circular points • All circles intersect at absolute points • absolute points satisfy x2+y2=0, w=0 • absolute points  Euclidean

15. Ellipses and circles • Q1: Two ellipses intersect at 4 points, two circles at only 2. Why? • Q2: An ellipse is defined by 5 points, circle by only 3. Why?

16. All circles intersect at absolute points Answer: if   Therefore, the absolute points lie on every circle

17. Absolute points are invariant to similarity transformations

18. Absolute points encode Euclidean (similarity) geometry in a single compact representation

19. Absolute conic (3D) • All spheres intersect at absolute conic • plane at infinity: P∞=(0,0,0,1) • points at infinity: (x,y,z,0) • absolute conic satisfies x2+y2+z2=0, w=0 • Therefore, W∞=I3x3 (identity matrix) in a metric frame • W∞ contains purely imaginary points • absolute conic  Euclidean

20. IAC • Projection of the absolute conic on the image plane • Image of the absolute conic (IAC) • IAC  calibrated camera (internal parameters) • Projection of P∞ onto image plane is homography • Translate camera, no change to this projection (cf. seeing stars on celestial sphere) • Rotate camera, projection does change • But IAC, which is the projection of W∞ , does not change with translation or rotation • H = KR for plane at infinity, but w∞ = P W∞ = K-T K-1

21. Algorithm overview • Take images of multiple planes • Each plane contains absolute points • Projection of absolute points lie on IAC • Use multiple planes to perform least squares fit to IAC • Decompose IAC to get K (internal parameters)

22. Zhang’s algorithm • Recall: • Euclidean constraints on rotation matrix: • Substitute to yield constraints on K:

23. 0 0 Cholesky decomposition • Recall: • Cholesky factorizes any symmetric and positive definite matrix into product of lower and upper triangular matrices: w∞ = LU 0 K: K-1: KT: K-T: 0

24. Zhang’s algorithm (bundle adjustment)

25. Geometric interpretation • Draw projection of IAC

26. Image of absolute points yields same constraints • Projection of plane is homography H • Leads to constraint: • Splitting into real and imaginary:

27. Tsai’s algorithm

28. What can you do with a calibrated camera

29. Radial lens distortion

30. Radial lens distortion http://toothwalker.org/optics/distortion.html

31. Radial lens distortion ru = rd + k1 rd3  where k1 > 0 for barrel distortion and k1 < 0 for pincushion. http://www.imatest.com/docs/distortion.html

32. Lens distortion It is frequently asserted that negative values of D correspond to barrel distortion, and positive values to pincushion distortion. For simple distortion curves this is true, but a few lenses with a more complex distortion curve do not follow this simple rule. As a case in point I consider the above Distagon 2.8/21. Fig. 4 shows a grid distorted according to the data in Fig. 3. Although curve A is negative everywhere, the grid reveals significant pincushion distortion toward the corners. Indeed, it is not the sign of D which determines the type of distortion, but the slope of the curve. A negative slope (yellow part) of D implies barrel distortion, a positive slope (orange part) pincushion distortion. The steeper the slope, the more pronounced the distortion. Barrel distortion typically will have a positive term for K1 where as pincushion distortion will have a negative value. -- http://en.wikipedia.org/wiki/Distortion_(optics) http://toothwalker.org/optics/distortion.html