Warping

1. Warping CSE 590 Computational Photography Tamara Berg

2. Picture Time!

3. Image Warping http://www.jeffrey-martin.com Slides from: Alexei Efros & Steve Seitz

4. Image Warping in Biology • D'Arcy Thompson • http://www-groups.dcs.st-and.ac.uk/~history/Miscellaneous/darcy.html • http://en.wikipedia.org/wiki/D'Arcy_Thompson • Importance of shape and structure in evolution Slide by Durand and Freeman

5. f f f T T x x x f x Image Transformations • image filtering: change range of image • g(x) = T(f(x)) image warping: change domain of image g(x) = f(T(x))

6. T T Image Transformations • image filtering: change range of image • g(x) = T(f(x)) f g image warping: change domain of image g(x) = f(T(x)) f g

7. Parametric (global) warping • Examples of parametric warps: aspect rotation translation perspective cylindrical affine

8. T Parametric (global) warping • Transformation T is a coordinate-changing machine: • p’ = T(p) • What does it mean that T is global? • Is the same for any point p • can be described by just a few numbers (parameters) • Let’s represent T as a matrix: • p’ = Mp p = (x,y) p’ = (x’,y’)

9. Scaling • Scaling a coordinate means multiplying each of its components by a scalar • Uniform scaling means this scalar is the same for all components: 2

10. X  2,Y  0.5 Scaling • Non-uniform scaling: different scalars per component:

11. Scaling • Scaling operation: • Or, in matrix form: scaling matrix S What’s inverse of S?

12. (x’, y’) (x, y)  2-D Rotation x’ = x cos() - y sin() y’ = x sin() + y cos()

13. (x’, y’) (x, y)  2-D Rotation f

14. (x’, y’) (x, y)  2-D Rotation x = r cos (f) y = r sin (f) x’ = r cos (f + ) y’ = r sin (f + ) f

15. (x’, y’) (x, y)  2-D Rotation x = r cos (f) y = r sin (f) x’ = r cos (f + ) y’ = r sin (f + ) Trig Identity… x’ = r cos(f) cos() – r sin(f) sin() y’ = r sin(f) cos() + r cos(f) sin() f

16. (x’, y’) (x, y)  2-D Rotation x = r cos (f) y = r sin (f) x’ = r cos (f + ) y’ = r sin (f + ) Trig Identity… x’ = r cos(f) cos() – r sin(f) sin() y’ = r sin(f) cos() + r cos(f) sin() Substitute… x’ = x cos() - y sin() y’ = x sin() + y cos() f

17. (x’, y’) (x, y)  2-D Rotation • This is easy to capture in matrix form: • What is the inverse transformation? • Rotation by –q • For rotation matrices R f

18. 2x2 Matrices • What types of transformations can be represented with a 2x2 matrix? 2D Identity?

19. 2x2 Matrices • What types of transformations can be represented with a 2x2 matrix? 2D Identity?

20. 2x2 Matrices • What types of transformations can be represented with a 2x2 matrix? 2D Identity? 2D Scale?

21. 2x2 Matrices • What types of transformations can be represented with a 2x2 matrix? 2D Identity? 2D Scale?

22. 2x2 Matrices • What types of transformations can be represented with a 2x2 matrix? 2D Rotate around (0,0)? 2D Shear?

23. 2x2 Matrices • What types of transformations can be represented with a 2x2 matrix? 2D Mirror about Y axis?

24. 2x2 Matrices • What types of transformations can be represented with a 2x2 matrix? 2D Mirror about Y axis?

25. 2x2 Matrices • What types of transformations can be represented with a 2x2 matrix? 2D Mirror about Y axis? 2D Mirror over (0,0)?

26. 2x2 Matrices • What types of transformations can be represented with a 2x2 matrix? 2D Mirror about Y axis? 2D Mirror over (0,0)?

27. 2x2 Matrices • What types of transformations can be represented with a 2x2 matrix? 2D Translation?

28. 2x2 Matrices • What types of transformations can be represented with a 2x2 matrix? 2D Translation? NO!

29. All 2D Linear Transformations • Linear transformations are combinations of … • Scale, • Rotation, • Shear, and • Mirror • Properties of linear transformations: • Origin maps to origin • Lines map to lines • Parallel lines remain parallel • Ratios are preserved • Closed under composition

30. Homogeneous Coordinates • Q: How can we represent translation as a 3x3 matrix?

31. Homogeneous Coordinates • Homogeneous coordinates • represent coordinates in 2 dimensions with a 3-vector

32. y 2 (2,1,1) or (4,2,2) or (6,3,3) 1 x 2 1 Homogeneous Coordinates 2D Points  Homogeneous Coordinates • Append 1 to every 2D point: (x y)  (x y 1) Homogeneous coordinates  2D Points • Divide by third coordinate (x y w)  (x/w y/w) Special properties • Scale invariant: (x y w) = k * (x y w) • (x, y, 0) represents a point at infinity • (0, 0, 0) is not allowed Scale Invariance

33. Homogeneous Coordinates • Q: How can we represent translation as a 3x3 matrix?

34. Homogeneous Coordinates • Q: How can we represent translation as a 3x3 matrix? • A: Using the rightmost column:

35. Translation • Example of translation Homogeneous Coordinates tx = 2ty= 1

36. Translation • Example of translation Homogeneous Coordinates tx = 2ty= 1

37. Basic 2D Transformations • Basic 2D transformations as 3x3 matrices Translate Scale Rotate Shear

38. Matrix Composition • Transformations can be combined by matrix multiplication p’ = T(tx,ty) R(Q) S(sx,sy) p

39. Affine Transformations • Affine transformations are combinations of … • Linear transformations, and • Translations • Properties of affine transformations: • Origin does not necessarily map to origin • Lines map to lines • Parallel lines remain parallel • Ratios are preserved • Closed under composition • Models change of basis

40. Projective Transformations • Projective transformations … • Affine transformations, and • Projective warps • Properties of projective transformations: • Origin does not necessarily map to origin • Lines map to lines • Parallel lines do not necessarily remain parallel • Ratios are not preserved • Closed under composition • Models change of basis

41. Recovering Transformations • What if we know f and g and want to recover the transform T? • willing to let user provide correspondences • How many do we need? ? T(x,y) y y’ x x’ f(x,y) g(x’,y’)

42. Translation: # correspondences? • How many correspondences needed for translation? ? T(x,y) y y’ x x’

43. Translation: # correspondences? • How many correspondences needed for translation? • How many Degrees of Freedom? ? T(x,y) y y’ x x’

44. Translation: # correspondences? • How many correspondences needed for translation? • How many Degrees of Freedom? • What is the transformation matrix? ? T(x,y) y y’ x x’

45. Euclidian: # correspondences? • How many correspondences needed for translation+rotation? • How many DOF? ? T(x,y) y y’ x x’

46. Affine: # correspondences? • How many correspondences needed for affine? • How many DOF? ? T(x,y) y y’ x x’

47. Affine: # correspondences? • How many correspondences needed for affine? • How many DOF? ? T(x,y) y y’ x x’

48. Projective: # correspondences? • How many correspondences needed for projective? • How many DOF? ? T(x,y) y y’ x x’

49. Example: warping triangles B’ • Given two triangles: ABC and A’B’C’ in 2D (12 numbers) • Need to find transform T to transfer all pixels from one to the other. • How can we compute the transformation matrix: B ? T(x,y) C’ A C A’ Source Destination

50. Example: warping triangles B’ • Given two triangles: ABC and A’B’C’ in 2D (12 numbers) • Need to find transform T to transfer all pixels from one to the other. • How can we compute the transformation matrix: B ? T(x,y) C’ A C A’ Source Destination cp2transform in matlab! input – correspondences. output – transformation.