Image Processing in SIGGRAPH 06

1. Image Processing in SIGGRAPH 06 Speaker: Qianqian Hu Date: March 31, 2006

2. Outlines • Fast Median and Bilateral Filtering • Ben Weiss (Shell & Slate Software) • Hybrid Images • Aude Oliva (Massachusetts Institute of Technology, Department of Brain and Cognitive Sciences), Antonio Torralba (Massachusetts Institute of Technology, Computer Science and Artificial Intelligence Laboratory), Philippe G. Schyns (University of Glasgow) • Image Deformation Using Moving Least Squares • Scott Schaefer (Texas A&M University), Travis McPhail, Joe Warren (Rice University) • Appearance-Space Texture Synthesis • Sylvain Lefebvre, Hugues Hoppe (Microsoft Research)

3. Image Deformation using Moving Least Squares Scott Schaefer Travis McPhail Joe Warren Texas A&M University Rice University Rice Univeristy

4. Example

5. Previous Work • Grid-based techniques: • Bivariate cubic splines[Sederberg and Parry, 1986, Lee et al, 1995] • Shepard’s interpolant[Beier and Neely, 1992] • Transformation-based technique: • Radial Basis Function[Bookstein, 1989] • Triangulation-based technique[Igarashi et al, 2005]

6. Overview of SIG’05

7. Characters of the Deformation Function • Given a set of handles p, and corresponding new positions q. The deformation function f satisfies • Interpolation: f(pi)=qi • Smoothness: smooth deformations • Identity: q=p f(v) = v

8. Moving Least Squares • Given a point v in the image, the best affine transformation lv(x) minimizes where DF: f(v)= lv(v)

9. Affine Transformation lv(x)=xM +T, • M :a linear transformation matrix (rotation and scaling) • T :a translation Best affine transformation where

10. Least Squares Problem where

11. Affine Deformations • Solution for matrix M • Solution for deformation function

12. Result Non-uniform scaling and shear

13. Similarity Deformations • Constraints: uniform scaling, i.e, • Define , where • Least squares problem where

14. Similarity Deformations • Solution for matrix M • Solution for deformation function where

15. Result uniformscaling

16. Rigid Deformations • Constraint: no uniform scaling, i.e., • Theoretical base

17. Rigid Deformations • Solution for matrix M • Solution for deformation function where , and Aiis as in similarity deformations.

18. Example

19. Examples for Rigid MLS

20. Examples for Rigid MLS

21. Deformation with Line Segments • Least squares problem

22. Affine Lines • Line segments are expressed in matrix form • Least squares problem

23. Solution • The deformation function and

24. Similarity Lines • The deformation function

25. Rigid Lines • The deformation function where

26. Examples for Rigid Lines

27. Implementation • Every pixel is replaced by a grid • Every resulting pixel is calculated using bilinear interpolation

28. Contributions • A simple closed-form solution • a linear system (2X2) at each point • No use of linear solver • Simple, and realtime • Handles: • points, • line segments. • As-rigid-as possible

29. Shortcoming • Lack of topological information

30. Future Work • Adding topological information • Generalizing to 3D to deform surfaces • Handles can be any curves

31. Fast Median and Bilateral Filtering Ben Weiss Shell & Slate Software Corp.

32. Example

33. Contributions • Improving Runtime from O(r) to O(logr) • Scalable to arbitrary radius • Realtime • Fitting for any bit-depth

34. Related Work • A variety of O(r) algorithms Huang, T.S., 1981. Two-Dimensional Signal Processing II: Transforms and Median Filters. No good performance for large filtering kernels. • A tree-based O(log2r) algorithm Gil, J. and Werman, M., 1993. Computing 2-D Min, Median, and Max Filters. Ill-suited for deep-pipelined, vector-capable modern processors. • A parallel algorithm with time complexity of O(log4r) Ranka, S., and Sahni, S., 1989. Efficient Serial and Parallel Algorithms for Median Filtering. even worser than linear for r<55.

35. Median Filtering • A pixel value is replaced by the median of its neighbours. [Tukey, 1977]

36. Advantages • Reducing image noise • Preserving edges • Basic algorithm of many image-processing techniques • Rank-order filtering • Morphological processing slowness!!!

37. Basic O(r) Algorithm • Consider a r-radius median filter to an 8-bit image.

38. Histogram and Mean Value • Use a 256-element histogram • Mean value = the index v*such that Integral =2r2+2r+1

39. Huang’s O(r) Algorithm

40. Fundamental idea • If multiple columns are processed at once, the aforementioned redundant calculations become sequential.

41. Distributive Histograms • For disjoint image regions A and B:

42. Adapted Huang’s Algorithm

43. Three Tiers and Beyond

44. O(logr) Algorithm

45. Higher-Depth Median Filtering • 16-bit and HDR(High dynamic range) images • Histogram exponentially with bit-depth

46. The Ordinal Transform cardinal values consecutive ordinal values

47. Comparison(1)

48. Comparison(2) • For 8-bit data • 50 times faster than Photoshop • For 16-bit data • 20 times as fast as Photoshop MedianDemo

49. Bilateral Filtering • A normalized convolution[Tomasi, 1998] • Spatial distance • Relative difference in intensity

50. Linear-Intensity Bilateral • A box spatial and triangular intensity filter