Sparse Approximation by Wavelet Frames and Applications

1. Sparse Approximation by Wavelet Frames and Applications Bin Dong Department of Mathematics The University of Arizona 2012 International Workshop on Signal Processing , Optimization, and ControlJune 30- July 3, 2012 USTC, Hefei, Anhui, China

2. Outlines • Wavelet Frame Based Models for Linear Inverse Problems (Image Restoration) • Applications in CT Reconstruction • 1-norm based models • Connections to variational model • 0-norm based model • Comparisons: 1-norm v.s. 0-norm • Quick Intro of Conventional CT Reconstruction • CT Reconstruction with Radon Domain Inpainting

3. Tight Frames in Unique Not unique • Orthonormal basis • Riesz basis • Tight frame: Mercedes-Benz frame • Expansions:

4. Tight Frames • General tight frame systems • Tight wavelet frames • Construction of tight frame: unitary extension principles [Ron and Shen, 1997] • They are redundant systems satisfying Parseval’s identity • Or equivalently where and

5. Tight Frames • Example: • Fast transforms • Lecture notes: [Dong and Shen, MRA-Based Wavelet Frames and Applications, IAS Lecture Notes Series,2011] • Decomposition • Reconstruction • Perfect Reconstruction • Redundancy

6. Image Restoration Model • Image Restoration Problems • Challenges: large-scale & ill-posed • Denoising, when is identity operator • Deblurring, when is some blurring operator • Inpainting, when is some restriction operator • CT/MR Imaging, when is partial Radon/Fourier • transform

7. Frame Based Models • Image restoration model: • Balanced model for image restoration [Chan, Chan, Shen and Shen, 2003], [Cai, Chan and Shen, 2008] • When , we have synthesis based model [Daubechies, Defrise and De Mol, 2004; Daubechies, Teschke and Vese, 2007] • When , we have analysis based model [Stark, Elad and Donoho, 2005; Cai, Osher and Shen, 2009] Resembles Variational Models

8. Connections: Wavelet Transform and Differential Operators • Nonlinear diffusion and iterative wavelet and wavelet frame shrinkage • 2nd-order diffusion and Haar wavelet: [Mrazek, Weickert and Steidl, 2003&2005] • High-order diffusion and tight wavelet frames in 1D: [Jiang, 2011] • Difference operators in wavelet frame transform: • True for general wavelet frames with various vanishing moments [Weickert et al., 2006; Shen and Xu, 2011] Filters Transform Approximation

9. Connections: Analysis Based Model and Variational Model • [Cai, Dong, Osher and Shen, Journal of the AMS, 2012]: • The connections give us • Leads to new applications of wavelet frames: Converges For any differential operator when proper parameter is chosen. • Geometric interpretations of the wavelet frame transform (WFT) • WFT provides flexible and good discretization for differential operators • Different discretizations affect reconstruction results • Good regularization should contain differential operators with varied orders (e.g., total generalized variation [Bredies, Kunisch, and Pock, 2010]) • Image segmentation: [Dong, Chien and Shen, 2010] • Surface reconstruction from point clouds: [Dong and Shen, 2011] Standard Discretization Piecewise Linear WFT

10. Frame Based Models: 0-Norm • Nonconvex analysis based model [Zhang, Dong and Lu, 2011] • Motivations: • Related work: • Restricted isometry property (RIP) is not • satisfied for many applications • Penalizing “norm” of frame coefficients • better balances sparsity and smoothnes • “norm” with : [Blumensath and Davies, 2008&2009] • quasi-norm with : [Chartrand, 2007&2008]

11. Fast Algorithm: 0-Norm • Penalty decomposition (PD) method [Lu and Zhang, 2010] • Algorithm: Change of variables Quadratic penalty

12. Fast Algorithm: 0-Norm • Step 1: • Subproblem 1a): quadratic • Subproblem 1b): hard-thresholding • Convergence Analysis [Zhang, Dong and Lu, 2011] :

13. Numerical Results • Comparisons (Deblurring) PFBS/FPC: [Combettes and Wajs, 2006] /[Hale, Yin and Zhang, 2010] Balanced Split Bregman: [Goldstein and Osher, 2008] & [Cai, Osher and Shen, 2009] Analysis PD Method: [Zhang, Dong and Lu, 2011] 0-Norm

14. Numerical Results • Comparisons Portrait Couple Balanced Analysis

15. Faster Algorithm: 0-Norm • Start with some fast optimization method for nonsmooth and convex optimizations: doubly augmented Lagrangian (DAL) method [Rockafellar, 1976]. Given the problem: The DAL method: where We solve the joint optimization problem of the DAL method using an inexact alternative optimization scheme

16. Faster Algorithm: 0-Norm • Start with some fast optimization method for nonsmooth and convex optimizations: doubly augmented Lagrangian (DAL) method [Rockafellar, 1976]. • The inexact DAL method: Given the problem: The DAL method: where Hard thresholding

17. Faster Algorithm: 0-Norm • However, the inexact DAL method does not seem to converge!! Nonetheless, the sequence oscillates and is bounded. • The mean doubly augmented Lagrangian method (MDAL) [Dong and Zhang, 2011] solve the convergence issue by using arithmetic means of the solution sequence as outputs instead: MDAL:

18. Comparisons: Deblurring • Comparisons of best PSNR values v.s. various noise level

19. Comparisons: Deblurring • Comparisons of computation timev.s. various noise level

20. Comparisons: Deblurring • What makes “lena” so special? • Decay of the magnitudes of the wavelet frame coefficients is very fast, which is what 0-norm prefers. • Similar observation was made earlier by [Wang and Yin, 2010]. 1-norm 0-norm: PD 0-norm: MDAL

21. With the Center for Advanced Radiotherapy and Technology (CART), UCSD Applications in CT Reconstruction

22. 3D Cone Beam CT Cone Beam CT

23. 3D Cone Beam CT Discrete = • Animation created by Dr. XunJia

24. Cone Beam CT Image Reconstruction • Goal: solve • Difficulties: • Related work: Unknown Image Projected Image • In order to reduce dose, the system is highly underdetermined. Hence the solution is not unique. • Projected image is noisy. • Total Variation (TV):[Sidkey, Kao and Pan 2006], [Sidkey and Pan, 2008], [Cho et al. 2009], [Jia et al. 2010]; • EM-TV: [Yan et al. 2011]; [Chen et al. 2011]; • Wavelet Frames:[Jia, Dong, Lou and Jiang, 2011]; • Dynamical CT/4D CT:[Chen, Tang and Leng, 2008], • [Jia et al. 2010], [Tian et al., 2011]; [Gao et al. 2011];

25. CT Image Reconstruction with Radon Domain Inpainting • Idea: start with • Benefits: • Instead of solving • We find both and such that: • is close to but with better quality • Prior knowledge of them should be used • Safely increase imaging dose • Utilizing prior knowledge we have for both CT • images and the projected images

26. CT Image Reconstruction with Radon Domain Inpainting • Model [Dong, Li and Shen, 2011] • Algorithm: alternative optimization & split Bregman. where • p=1, anisotropic • p=2, isotropic

27. CT Image Reconstruction with Radon Domain Inpainting • Algorithm [Dong, Li and Shen, 2011]: block coordinate descend method [Tseng, 2001] • Convergence Analysis Problem: Algorithm: Note: If each subproblem is solved exactly, then the convergence analysis was given by [Tseng, 2001], even for nonconvex problems.

28. CT Image Reconstruction with Radon Domain Inpainting • Results: N denoting number of projections N=15 N=20

29. CT Image Reconstruction with Radon Domain Inpainting • Results: N denoting number of projections N=15 N=20 W/O Inpainting With Inpainting

30. Thank You Collaborators: • Mathematics • Stanley Osher, UCLA • ZuoweiShen, NUS • Jia Li, NUS • JianfengCai, University of Iowa • Yifei Lou, UCLA/UCSD • Yong Zhang, Simon Fraser University, Canada • Zhaosong Lu, Simon Fraser University, Canada • Medical School • Steve B. Jiang, Radiation Oncology, UCSD • XunJia, Radiation Oncology, UCSD • Aichi Chien, Radiology, UCLA