Multiscale Analysis of Images

1. Multiscale Analysis of Images Gilad Lerman Math 5467 (stealing slides from Gonzalez & Woods, and Efros)

2. The Multiscale Nature of Images

3. Recall: Gaussian pre-filtering G 1/8 G 1/4 Gaussian 1/2 • Solution: filter the image, then subsample • Filter size should double for each ½ size reduction.

4. Subsampling with Gaussian pre-filtering Gaussian 1/2 G 1/4 G 1/8 • Solution: filter the image, then subsample • Filter size should double for each ½ size reduction.

5. Image Pyramids • Known as a Gaussian Pyramid [Burt and Adelson, 1983] • In computer graphics, a mip map [Williams, 1983] • A precursor to wavelet transform

6. A bar in the big images is a hair on the zebra’s nose; in smaller images, a stripe; in the smallest, the animal’s nose Figure from David Forsyth

7. Gaussian pyramid construction filter mask • Repeat • Filter • Subsample • Until minimum resolution reached • can specify desired number of levels (e.g., 3-level pyramid) • The whole pyramid is only 4/3 the size of the original image!

8. What does blurring take away? (recall) original

9. What does blurring take away? (recall) smoothed (5x5 Gaussian)

10. High-Pass filter smoothed – original

11. Band-pass filtering • Laplacian Pyramid (subband images) • Created from Gaussian pyramid by subtraction Gaussian Pyramid (low-pass images)

12. Laplacian Pyramid • How can we reconstruct (collapse) this pyramid into the original image? Need this! Original image

15. Image resampling (interpolation) • So far, we considered only power-of-two subsampling • What about arbitrary scale reduction? • How can we increase the size of the image? d = 1 in this example 1 2 3 4 5 • Recall how a digital image is formed • It is a discrete point-sampling of a continuous function • If we could somehow reconstruct the original function, any new image could be generated, at any resolution and scale

16. Image resampling • So far, we considered only power-of-two subsampling • What about arbitrary scale reduction? • How can we increase the size of the image? d = 1 in this example 1 2 3 4 5 • Recall how a digital image is formed • It is a discrete point-sampling of a continuous function • If we could somehow reconstruct the original function, any new image could be generated, at any resolution and scale

17. Answer: guess an approximation • Can be done in a principled way: filtering 1 d = 1 in this example 1 2 2.5 3 4 5 • Image reconstruction • Convert to a continuous function • Reconstruct by cross-correlation: Image resampling • So what to do if we don’t know

18. Resampling filters • What does the 2D version of this hat function look like? performs linear interpolation (tent function) performs bilinear interpolation • Better filters give better resampled images • Bicubic is common choice • Why not use a Gaussian? • What if we don’t want whole f, but just one sample?

19. Bilinear interpolation • Smapling at f(x,y):

20. Applications (more efficient with wavelets) • Coding • High frequency coefficients need fewer bits, so one can • collapse a compressed Laplacian pyramid • Denoising/Restoration • Setting “most” coefficients in Laplacian pyramid to zero • Image Blending

21. 0 1 0 1 0 1 Application: Pyramid Blending Left pyramid blend Right pyramid

22. Image Blending

23. + = 1 0 1 0 Feathering Ileft Iright right left Encoding transparency I(x,y) = (aR, aG, aB, a) Iblend = left Ileft + right Iright

24. 0 1 0 1 Affect of Window Size left right

25. 0 1 0 1 Affect of Window Size

26. 0 1 Good Window Size “Optimal” Window: smooth but not ghosted Burt and Adelson (83): Choose by pyramids…

27. Pyramid Blending

28. Pyramid Blending (Color)

29. Laplacian Pyramid: Blending • General Approach: • Build Laplacian pyramids LA and LB from images A and B • Build a Gaussian pyramid GR from selected region R (black white corresponding images) • Form a combined pyramid LS from LA and LB using nodes of GR as weights: • LS(i,j) = GR(I,j,)*LA(I,j) + (1-GR(I,j))*LB(I,j) • Collapse the LS pyramid to get the final blended image

30. laplacian level 4 laplacian level 2 laplacian level 0 left pyramid right pyramid blended pyramid

31. Blending Regions

32. Simplification: Two-band Blending • Brown & Lowe, 2003 • Only use two bands: high freq. and low freq. • Blends low freq. smoothly • Blend high freq. with no smoothing: use binary mask Can be explored in a project…