On the Role of Localized Relationships in Image Interpolation and CCD Demosaicing

1. On the Role of Localized Relationships in Image Interpolation and CCD Demosaicing RONEN SHER and MOSHE PORAT

2. Agenda • Image Interpolation Methods • Image Regions • B&W Method • Algorithm • Results • Conclusions • 1D Signal Method • CCD Demosaicing • Structure • Components correlation • Statistical extention • Results

3. The Problem • Enlargement of an Image by 2x2 Input

4. Interpolations Methods • Bilinear • Bi-Cubic • Spline • Diffusion Equation

5. Interpolation Methods • Bilinear, in the case of enlargement by 2 In smooth eras it is a satisfying prediction

6. Pixels Correlation 1 • Normalized histograms of Lena gray Levels 256x256 -solid and 512x512-dashed

7. Pixels Correlation 2

8. Image Sub-Regions • In edges regions an average prediction will result in a smoothness effect. • The edge must be preserved. • The edges exist in the input image and the same distribution is assumed in the large image.

9. Image Regions • In case of a horizontal edge: and • In case of a vertical edge: and • Depending on the four surrounding neighbors, there will be 4!=24 permutations

10. Image Regions • In each region a different waited sum is valid for the prediction • The coefficients • are learned from the input image

11. Coefficients calculation 1 • Scanning the Input Image for the x-kind pixels we determine its permutation from its four neighbors and saving it’s value and it’s neighbors value in VMx • modeling only the regions with significant changes in gray levels • The same rule holds for the +kind pixels

12. Coefficients calculation 2 • For each permutation we find the four coefficients using the Least Square solution • Same technique for the + coefficients

13. Black and White Images

14. Algorithm B&W images • Scanning the Sparse Image • for each pixel we determine its matching permutation (coefficients) from its four neighbors and predict its value using

15. Algorithm B&W images • The Input is Ix For each “+” pixel we find its matching permutation (coefficients) and calculate its prediction by

16. Experiments - B&W images • The 24 vectors of coefficients of x-kind Lena with size 512x512- black disks and 256x256- empty circles

17. Experiments 1 - B&W images Original Bilinear Nearest neighbor (Input) proposed Bi-Cubic Bi-Cubic Spline

18. Experiments 2 - B&W images Original Bilinear Nearest neighbor (Input) Bi-Cubic Bi-Cubic Spline proposed

19. One Dimension Interpolation Interpolating yd, by using NR. Its adjacent samples serve as the four neighbors to find the coefficients.

20. 1D Interpolation results 1 Sinc-MSE= 2.6651 NR-MSE= 0.8540 Signal is composed from sum of Sincs

21. 1D Interpolation results 2 Voice signal: the word “Diskette”

22. CCD Demosaicing

23. CCD structure

24. Simple Method Treating each color component as individual B&W image

25. Components method • Using all colors neighbors for the green reconstruction • Reconstruct the difference of the colors components – Hues (R-G, B-G, R-B). Processing smoother signals.

26. Statistical extension • Separating each case to sub regions for better characterization. • Using the mean and the standard deviation of each neighbors’ set for the subtraction.

27. Results 1 Original Bi-Linear Gunturk Optimalrecovery Kimmel Neighbors Rule

28. Results 2 Original Bi-Linear Gunturk Optimalrecovery Kimmel Neighbors Rule

29. Summery • A new reconstruction method was presented for 1D signals, B&W images and CCD demosaicing using the correlation between low and high resolution versions. • The new method showed satisfying results compared to other known techniques.