CS 691 Computational Photography

1. CS 691 Computational Photography Instructor: Gianfranco Doretto 3D to 2D Projections

2. Pinhole camera model • Pinhole model: • Captures pencil of rays – all rays through a single point • The point is called Center of Projection (COP) • The image is formed on the Image Plane • Effective focal length f is distance from COP to Image Plane

3. Dimensionality Reduction Machine (3D to 2D) 3D world 2D image • What have we lost? • Angles • Distances (lengths)

4. Slide source: Seitz Projection can be tricky…

5. Slide source: Seitz Projection can be tricky…

6. Projective Geometry What is lost? • Length Who is taller? Which is closer?

7. Lengths can’t be trusted... A’ C’ B’

8. …but humans adopt! Müller-Lyer Illusion We don’t make measurements in the image plane http://www.michaelbach.de/ot/sze_muelue/index.html

9. Projective Geometry What is lost? • Length • Angles Parallel? Perpendicular?

10. Parallel lines aren’t…

11. Projective Geometry What is preserved? • Straight lines are still straight

12. Vanishing points and lines Parallel lines in the world intersect in the image at a “vanishing point”

13. Vanishing points and lines Vanishing Point Vanishing Point o o Vanishing Line

14. Vertical vanishing point (at infinity) Vanishing line Vanishing point Vanishing point Vanishing points and lines Credit: Criminisi

15. Vanishing points and lines Photo from online Tate collection

16. Note on estimating vanishing points Use multiple lines for better accuracy … but lines will not intersect at exactly the same point in practice One solution: take mean of intersecting pairs … bad idea! Instead, minimize angular differences

17. Vanishing objects

18. Modeling projection • The coordinate system • We will use the pin-hole model as an approximation • Put the optical center (Center Of Projection) at the origin • Put the image plane (Projection Plane) in front of the COP • Why? • The camera looks down the negative z axis • we need this if we want right-handed-coordinates

19. We get the projection by throwing out the last coordinate: Modeling projection • Projection equations • Compute intersection with PP of ray from (x,y,z) to COP • Derived using similar triangles (on board)

20. Homogeneous coordinates • Trick: add one more coordinate: • no—division by z is nonlinear • Is this a linear transformation? homogeneous scene coordinates homogeneous image coordinates Converting from homogeneous coordinates

21. Homogeneous coordinates Invariant to scaling Point in Cartesian is ray in Homogeneous Homogeneous Coordinates Cartesian Coordinates

22. divide by third coordinate divide by fourth coordinate Perspective Projection • Projection is a matrix multiply using homogeneous coordinates: • This is known as perspective projection • The matrix is the projection matrix • Can also formulate as a 4x4

23. Orthographic Projection • Special case of perspective projection • Distance from the COP to the PP is infinite • Also called “parallel projection” • What’s the projection matrix? Image World

24. Scaled Orthographic Projection • Special case of perspective projection • Object dimensions are small compared to distance to camera • Also called “weak perspective” • What’s the projection matrix?

25. Spherical Projection • What if PP is spherical with center at COP? • In spherical coordinates, projection is trivial: • (θ,Φ) = (θ,Φ,d) • Note: doesn’t depend on focal length!

26. Projection matrix R,T jw kw Ow iw x: Image Coordinates: w(u,v,1) K: Intrinsic Matrix (3x3) R: Rotation (3x3) t: Translation (3x1) X:World Coordinates: (X,Y,Z,1)

27. Projection matrix • Intrinsic Assumptions • Unit aspect ratio • Principal point at (0,0) • No skew • Extrinsic Assumptions • No rotation • Camera at (0,0,0) K

28. Remove assumption: known optical center • Intrinsic Assumptions • Unit aspect ratio • No skew • Extrinsic Assumptions • No rotation • Camera at (0,0,0)

29. Remove assumption: square pixels • Intrinsic Assumptions • No skew • Extrinsic Assumptions • No rotation • Camera at (0,0,0)

30. Remove assumption: non-skewed pixels Intrinsic Assumptions • Extrinsic Assumptions • No rotation • Camera at (0,0,0) Note: different books use different notation for parameters

31. Oriented and Translated Camera R jw t kw Ow iw

32. Allow camera translation Intrinsic Assumptions • Extrinsic Assumptions • No rotation

33. Slide Credit: Saverese p’ γ p y 3D Rotation of Points Rotation around the coordinate axes, counter-clockwise: z

34. Allow camera rotation

35. Degrees of freedom 5 6

36. Vanishing Point = Projection from Infinity

37. Lens Flaws

38. Lens Flaws: Chromatic Aberration • Dispersion: wavelength-dependent refractive index • (enables prism to spread white light beam into rainbow) • Modifies ray-bending and lens focal length: f(λ) • color fringes near edges of image • Corrections: add ‘doublet’lens of flint glass, etc.

39. Chromatic Aberration Near Lens Center Near Lens Outer Edge

40. Radial Distortion (e.g. ‘Barrel’ and ‘pin-cushion’) straight lines curve around the image center

41. Radial Distortion • Radial distortion of the image • Caused by imperfect lenses • Deviations are most noticeable for rays that pass through the edge of the lens No distortion Pin cushion Barrel

42. Radial Distortion

43. Slide Credits • This set of sides also contains contributionskindly made available by the following authors • Alexei Efros • Stephen E. Palmer • Steve Seitz • Derek Hoiem • David Forsyth • George Bebis • SilvioSavarese