CS 691 Computational Photography

1. CS 691 Computational Photography Instructor: Gianfranco Doretto Image Pyramids

2. Linear image transformations • In analyzing images, it’s often useful to make a change of basis. transformed image Vectorized image Basis

3. Spatial Domain Basis Basis functions: Tells you where things are…. … but no concept of what it is …………..

4. Identity transform 1000… 01000… 00100… 000100… = * …0001000… …00010 …0001 Pixel domain image Spatial bases are local: each transform coefficient depends on one pixel location. Pixel domain image

5. Fourier Domain Basis Basis functions: Tells you what(frequency)is in the image…. … but not where it is ……… ………

6. Fourier transform = * Fourier transform Fourier bases are global: each transform coefficient depends on all pixel locations. Pixel domain image

7. Image Analysis • Want representation that combines what and where.  Image Pyramids

8. Overview • Gaussian Pyramid • Laplacian Pyramid

9. Image Pyramids • Known as a Gaussian Pyramid [Burt and Adelson, 1983] • In computer graphics, a mip map [Williams, 1983] • A precursor to wavelet transform

10. Gaussian pyramid construction filter mask • Repeat • Filter (keep filter the same size) • Subsample (by a factor of two) • Until minimum resolution reached • can specify desired number of levels (e.g., 3-level pyramid) • Total number of pixels in pyramid? • 1 + ¼ + 1/16 + 1/32…….. = 4/3  Over-complete representation

11. A bar in the big images is a hair on the zebra’s nose; in smaller images, a stripe; in the smallest, the animal’s nose

12. What are they good for? • Improve Search • Search over translations • Classic coarse-to-fine strategy • Search over scale • Template matching • E.g. find a face at different scales • Pre-computation • Need to access image at different blur levels • Useful for texture mapping at different resolutions (called mip-mapping) • Compression • Capture important structures with fewer bytes • Denoising • Model statistics of pyramid sub-bands • Image blending

13. Gaussian pyramid = * Pixel domain image Gaussian pyramid Overcomplete representation. Low-pass filters, sampled appropriately for their blur.

14. Overview • Gaussian Pyramid • Laplacian Pyramid

15. What does blurring take away? original

16. What does blurring take away? smoothed (5x5 Gaussian)

17. High-Pass filter smoothed – original

18. Band-pass filtering Gaussian Pyramid (low-pass images) • Laplacian Pyramid (subband images) • Created from Gaussian pyramid by subtraction

19. Laplacian filter - ≅ Unit impulse Laplacian of Gaussian Gaussian

20. Laplacian pyramid algorithm subsample subsample subsample blur blur blur _ _ _

21. Can we reconstruct the original? Need this! interpolate interpolate interpolate

22. The Laplacian Pyramid • Synthesis • preserve difference between upsampled Gaussian pyramid level and Gaussian pyramid level • band pass filter - each level represents spatial frequencies (largely) unrepresented at other levels • Analysis • reconstruct Gaussian pyramid, take top layer

23. Laplacian pyramid

24. Laplacian pyramid = * Laplacian pyramid Pixel domain image Overcomplete representation. Transformed pixels represent bandpassed image information.

25. Hybrid Image in Laplacian Pyramid Extra points for project 1 High frequency  Low frequency

26. Slide Credits • This set of sides also contains contributionskindly made available by the following authors • Alexei Efros • Svetlana Lazebnik • Frédo Durand • Bill Freeman • Steve Seitz • Derek Hoiem • David Forsyth