1 / 47

Conventional Camera (F/4)

Spectral Focal Sweep (SFS): Extended Depth of Field from Chromatic Aberrations ICCP 2010 Ollie Cossairt, Shree Nayar Columbia University. Conventional Camera (F/4). BLURRY. Conventional Camera (F/16). NOISY. EDOF Camera (F/4). CAPTURED. EDOF Camera (F/4). DEBLURRED. Related Work.

heckler
Download Presentation

Conventional Camera (F/4)

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Spectral Focal Sweep (SFS):Extended Depth of Field from Chromatic AberrationsICCP 2010Ollie Cossairt, Shree NayarColumbia University

  2. Conventional Camera (F/4) BLURRY Spectral Focal Sweep: ICCP 2010

  3. Conventional Camera (F/16) NOISY Spectral Focal Sweep: ICCP 2010

  4. EDOF Camera (F/4) CAPTURED Spectral Focal Sweep: ICCP 2010

  5. EDOF Camera (F/4) DEBLURRED Spectral Focal Sweep: ICCP 2010

  6. Related Work Coded Aperture [Welford ’60] [Levin et al. ‘07] [Veeraraghavan et al. ’07] [Zhou and Nayar ’09] Phase Plates [Dowski and Cathey ’95] [Chi and George ’01] [Levin et al. ’09] Focus Sweep [Nagahara et al. ’08] [Hausler ’72] Sensor Lens Focal Plane Color [Guichard et al. ’09] Blue Green Red RGB Sensor Lens Focal Planes Spectral Focal Sweep: ICCP 2010

  7. Spectral Focal Sweep Idea Broadband reflectance spectra Poorly Corrected Lens B&W Sensor Focus Sweep = + + Sensor Lens Focal Plane No moving parts or custom elements Spectral Focal Sweep: ICCP 2010

  8. Spectral Focal Sweep Idea Broadband reflectance spectra • Works exactly for black and white scenes • Many naturally occurring reflectance spectra are also broadband [Parkkinen ‘89] Poorly corrected lens • Simpler lens design optimization Black & White Sensor • Can also approximate with luminance channel from color sensor Spectral Focal Sweep: ICCP 2010

  9. Conventional Focus Sweep PSF u = object depth (fixed) v= sensor distance (varying) Instantaneous PSF Focal Plane Lens EFL = f Sensor u v1 Scene Depth Range Spectral Focal Sweep: ICCP 2010

  10. Conventional Focus Sweep PSF u = object depth (fixed) v= sensor distance (varying) Instantaneous PSF Focal Plane Lens EFL = f Sensor d2 u v2 Scene Depth Range Spectral Focal Sweep: ICCP 2010

  11. Conventional Focus Sweep PSF u = object depth (fixed) v= sensor distance (varying) Instantaneous PSF Focal Plane Lens EFL = f Sensor d3 u v3 Scene Depth Range Spectral Focal Sweep: ICCP 2010

  12. Conventional Focus Sweep PSF Integrate over time to get final PSF Circ/Top-hat function Defocus disk diameter (A = aperture diameter) Varying either object/sensor distance, or focal length will produce the same PSF Spectral Focal Sweep: ICCP 2010

  13. Refractive Dispersion (prism) Snell’s Law of refraction Refraction depends on wavelength Spectral Focal Sweep: ICCP 2010

  14. Refractive Dispersion (lens) For an uncorrected lens, Focal length depends on wavelength R1, R2 =lens radii of curvature Focal Planes PSF Sensor v u Scene Depth Range Spectral Focal Sweep: ICCP 2010

  15. Spectral Focal Sweep PSF Each wavelength produces a different defocus disk A B&W Sensor integrates over wavelength, weighted by the spectral reflectance of the scene For white objects, a SFS PSF is identical to a mechanical focal sweep PSF Spectral Focal Sweep: ICCP 2010

  16. SFS Lens Design • Design Criteria • Wavelength range 400-700nm • 75mm EFL F/4 • 1/3” Sensor, 10um pixel size • Optimization Criteria • Zemax Optical Design Software • Maximize chromatic defocus across wavelength range • Minimize spot size at 550nm • Fit to off-the-shelf lenses Maximum defocus diameter 100um at 400nm and 700nm Spectral Focal Sweep: ICCP 2010

  17. Zemax SFS PSFs(white spectrum) field Corrected lens SFS Lens 15x depth Spectral Focal Sweep: ICCP 2010

  18. Zemax SFS PSFs(white spectrum) 1 x 15 field SFS Lens Corrected lens depth Spectral Focal Sweep: ICCP 2010

  19. Simulating Real Spectra • Munsell database of 1250 measured reflectance spectra • Zemax simulation of SFS PSF at 12 depth locations (1.5m-3m, 100um max defocus dia.) • For each Munsell spectrum • Calculate average deblurring error when a white spectrum is used for deblurring Spectral Focal Sweep: ICCP 2010

  20. Average Deblurring Error Input Image: Munsell color PSFs: White PSF at center depth: PSF Distance Metric: [Zhou and Nayar ‘09] Average Deblurring Error: Averaged over many images Spectral Focal Sweep: ICCP 2010

  21. Munsell Spectra PSFs 1 x 15 Corrected lens depth Most Munsell spectra show little PSF variation Spectral Focal Sweep: ICCP 2010

  22. Average Deblurring Error(%95 of Munsell spectra) Most Munsell PSFs introduce negligible deblurring artifacts Spectral Focal Sweep: ICCP 2010

  23. Doublet SFS Implementation Implementation Zemax Design Edmund Optics #48184 1” Dia. 100mm FL Plano-convex asphere Thorlabs #LE1929 1” Dia. 300mm FL positive meniscus Spectral Focal Sweep: ICCP 2010

  24. Experimental Setup Corrected Lens SFS Doublet Lens B&W Examples (Basler 1/3” sensor) F/4 SFS 75mm EFL Doublet F/1.4 75mm EFL Cosmicar Color Examples (Canon 450D sensor) F/4 SFS 75mm EFL Doublet F/2.8 100mm EFL Cannon Spectral Focal Sweep: ICCP 2010

  25. Measured SFS PSFs(white spectrum) depth Cosmicar SFS lens Measured SFS PSF approximately depth-invariant Spectral Focal Sweep: ICCP 2010

  26. Deblurring Deblur with measured white PSF B&W Images • Wiener deconvolution Color Images • Transform from RGB to YUV space • Wiener deconvolution of Y image • Transform back to RGB space Spectral Focal Sweep: ICCP 2010

  27. EDOF Examples Spectral Focal Sweep: ICCP 2010

  28. F/4 Corrected Lens 8ms exposure Spectral Focal Sweep: ICCP 2010

  29. F/4 SFS Camera (Captured) 8ms exposure Spectral Focal Sweep: ICCP 2010

  30. F/4 SFS Camera (Deblurred) 8ms exposure Spectral Focal Sweep: ICCP 2010

  31. F/4 Corrected Lens Foreground Background 8ms exposure Spectral Focal Sweep: ICCP 2010

  32. F/4 SFS Lens (Captured) 8ms exposure Spectral Focal Sweep: ICCP 2010

  33. F/4 SFS Lens (Deblurred) 8ms exposure Spectral Focal Sweep: ICCP 2010

  34. F/4 Corrected Lens Foreground Background 16ms exposure Spectral Focal Sweep: ICCP 2010

  35. F/4 SFS Lens (Captured) 16ms exposure Spectral Focal Sweep: ICCP 2010

  36. F/4 SFS Lens (Deblurred) 16ms exposure Spectral Focal Sweep: ICCP 2010

  37. F/4 Corrected Lens Background Foreground 16ms exposure Spectral Focal Sweep: ICCP 2010

  38. F/4 SFS Lens (Captured) 16ms exposure Spectral Focal Sweep: ICCP 2010

  39. F/4 SFS Lens (Deblurred) 16ms exposure Spectral Focal Sweep: ICCP 2010

  40. F/4 Corrected Lens Background Foreground 16ms exposure Spectral Focal Sweep: ICCP 2010

  41. F/4 SFS Lens (Captured) 16ms exposure Spectral Focal Sweep: ICCP 2010

  42. F/4 SFS Lens (Deblurred) 16ms exposure Spectral Focal Sweep: ICCP 2010

  43. F/4 Corrected Lens Background Foreground 16ms exposure Spectral Focal Sweep: ICCP 2010

  44. F/4 SFS Lens (Captured) 16ms exposure Spectral Focal Sweep: ICCP 2010

  45. F/4 SFS Lens (Deblurred) 16ms exposure Spectral Focal Sweep: ICCP 2010

  46. Limitations • Broadband assumption does not always hold (e.g. %5 of Munsell Spectra) • deblurring artifacts introduced • Color deblurring method does not correct blur in chrominance channels Spectral Focal Sweep: ICCP 2010

  47. Conclusion Broadband spectra Chromatic aberrations B&W sensor Focus sweep + + = = • Works exactly for B&W scenes • Works well for many naturally occurring spectra • Results for color images “look” good • Easy to build EDOF camera Spectral Focal Sweep: ICCP 2010

More Related