Image Processing and Reconstructions Tools

1. Image Processing and Reconstructions Tools Ramesh Raskar Mitsubishi Electric Research Labs Cambridge, MA

2. Image Tools • Gradient domain operations, • Tone mapping, fusion and matting • Graph cuts, • Segmentation and mosaicing • Bilateral and Trilateral filters, • Denoising, image enhancement

3. Intensity Gradient in 1D Intensity Gradient 105 105 1 1 I(x) G(x) Gradient at x,G(x) = I(x+1)- I(x)Forward Difference

4. Reconstruction from Gradients Gradient Intensity 105 105 ? ? 1 1 G(x) I(x) For n intensity values, about n gradients

5. Reconstruction from Gradients Gradient Intensity 105 105 ? 1 1 G(x) I(x) 1D Integration I(x) = I(x-1) + G(x) Cumulative sum

6. Intensity Gradient in 2D Gradient at x,y as Forward Differences Gx(x,y) = I(x+1 , y)- I(x,y) Gy(x,y) = I(x , y+1)- I(x,y) G(x,y) = (Gx , Gy) Grad X Grad Y

7. Intensity Gradient Vectors in Images Gradient Vector

8. Image Intensity Gradients in 2D Sanity Check: Recovering Original Image 2D Integration Grad X Solve Poisson Equation, 2D linear system Grad Y

9. Intensity Gradient Manipulation A Common Pipeline 2D Integration Grad X New Grad X Gradient Processing New Grad Y Grad Y Modify Gradients

10. Graph and Images Credits: Jianbo Shi

11. Agrawala et al, Digital Photomontage, Siggraph 2004

13. Source images Brush strokes Computed labeling Composite

14. i Wij j Wij Segmentation = Graph partition V: graph node E: edges connection nodes Wij: Edge weight Image pixel Link to neighboring pixels Pixel similarity

15. Minimum Cost Cuts in a graph Cut: Set of edges whose removal makes a graph disconnected Si,j : Similarity between pixel i and pixel j Cost of a cut, A A

16. Graph Cuts for Segmentation and Mosaicing Cut ~ String on a height field Brush strokes Computed labeling

17. Bilateral Filtering Gaussian Smoothing Input Log(Intensity) Bilateral Smoothing [Ben Weiss, Siggraph 2006]

18. Start with Gaussian filtering • Here, input is a step function + noise output input

19. Start with Gaussian filtering • Spatial Gaussian f output input

20. Start with Gaussian filtering • Output is blurred output input

21. Gaussian filter as weighted average • Weight of x depends on distance to x output input

22. The problem of edges • Here, “pollutes” our estimate J(x) • It is too different output input

23. Principle of Bilateral filtering [Tomasi and Manduchi 1998] • Penalty g on the intensity difference output input

24. Bilateral filtering [Tomasi and Manduchi 1998] • Spatial Gaussian f output input

25. Bilateral filtering [Tomasi and Manduchi 1998] • Spatial Gaussian f • Gaussian g on the intensity difference output input

26. [Tomasi and Manduchi 1998] • The weights are different for each output pixel output input

27. Bilateral filtering is non-linear [Tomasi and Manduchi 1998] • The weights are different for each output pixel output input

28. Bilateral Filtering • Unilateral filtering • Smoothing using filtering • Bilateral filtering • Edge-preserving smoothing Gaussian Smoothing Input Log(Intensity) Bilateral Smoothing [Ben Weiss, Siggraph 2006]