Iterated Denoising for Image Recovery

1. Presentation given at DCC 02. Iterated Denoising for Image Recovery Onur G. Guleryuz onur@danbala.poly.edu To see the animations and movies please use full-screen mode. Clicking on pictures to the left of PSNR curves should start the movies. There are also reminder notes for some slides.

2. Notices: • Errata for manuscript. • Brought code. Can run for other images, for your images, etc. • If interested, please find me during breaks or evenings. Overview • Problem definition. • Main Algorithm. • Rationale. • Choice of transforms. • Many simulation examples, movies, etc.

3. Pretend “Image + Noise” Use surrounding spatial information to recover lost block via overcomplete denoising with hard-thresholding.* Problem Statement Image Lost Block Applications: Error concealment, damaged images, ... Generalizations: Irregularly shaped blocks, partial information, ...

4. Hard threshold coefficients with T Partially denoised result 1 Partially denoised result 2 Hard threshold coefficients with T . . . Average partially denoised results for final denoised image. What is Overcomplete Denoising with Hard-thresholding? Image y x DCT (MxM) tilings Utilized transform will be very important!

5. (Figure 1 in the paper) Examples +9.37 dB +8.02 dB +11.10 dB +3.65 dB

6. (Figure 2 in the paper) Main Algorithm I Denoising with hard-thresholding using overcomplete transforms Recover layer P by mainly using information from layers 0,…,P-1

7. Main Algorithm II • Assign initial values to layer pixels. • T=T 0 • while ( T > T ) F • for i=1: number_of_layers recover layer i by overcomplete denoising with threshold T • end • T=T- dT • end

8. th k DCT block y o (k) y o (k) x Hard threshold block k coefficients if OR x DCT (MxM) tiling 1 o (k) < M/2 o (k) < M/2 y x (Figure 3 in the paper) *Main Algorithm III Image Lost block Outer border of layer P

9. (Figure 4 in the paper) Example DCT Tilings and Selective Hard Thresholding

10. original transform coefficient error Assume that the transform yieldsa sparse image representation: Hard thresholding removes more noise than signal. Rationale: Denoising and Recovery Main intuition: Keep coefficients of high SNR, zero out coefficients of low SNR.

11. . . . Best subspaces to zero-out in a POCS setting. Optimal linear estimators. Sparse transforms. Rationale: Other Analogies Band limited reconstructions via POCS: Set of possible signals given the available information. Set of bandlimited (low pass) signals Assumes low frequency Fourier coefficients are important and zeros out high frequencies coefficients. This work: Adaptively change sets at each iteration. Let data determine the important coefficients and which coefficients to zero out.

12. Properties of Desired Transforms • Periodic, approximately periodic regions: Transform should “see” the period Example: Minimum period 8 at least 8x8 DCT, ~ 3 level wavelet packets. • Edge regions (sparsity may not be enough): Transform should “see” the slope of the edge.

13. +11.10 dB DCT 9x9 (Figure 1 in the paper) Periodic Example

14. Perf. Rec. DCT 8x8 (Figure 5 in the paper) Periodic Example (period=8)

15. +3.65 dB DCT 16x16 (Figure 6 in the paper) Periodic Example

16. +5.91 dB DCT 24x24 Periodic Example

17. +7.2 dB DCT 16x16 “Periodic” Example

18. +10.97 dB DCT 24x24 “Periodic” Example

19. +25.51 dB DCT 8x8 Edge Example

20. +9.37 dB Complex wavelets (Figure 6 in the paper) Edge Example

21. +16.72 dB Complex wavelets (Figure 6 in the paper) Edge Example

22. +9.26 dB DCT 24x24 (Figure 6 in the paper) Edge Example

23. +8.02 dB Complex wavelets (Figure 1 in the paper) Edge Example

24. -1.00 dB DCT 16x16 (Figure 7 in the paper) Unsuccessful Recovery Example

25. +4.11 dB DCT 16x16 (Figure 7 in the paper) Partially Successful Recovery Example

26. -1.06 dB DCT 4x4 Edges and “Small Transforms”

27. +5.56 dB DCT 4x4 Edges and “Small Transforms”

28. +9.26 dB DCT 24x24 (Figure 6 in the paper) Edge Example