1 / 52

Light field photography and microscopy

Light field photography and microscopy. Marc Levoy. Computer Science Department Stanford University. two-plane parameterization [Levoy and Hanrahan 1996]. The light field [Gershun 1936]. Radiance as a function of position and direction for general scenes the “plenoptic function”

randolphc
Download Presentation

Light field photography and microscopy

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Light field photography and microscopy Marc Levoy Computer Science Department Stanford University

  2. two-plane parameterization [Levoy and Hanrahan 1996] The light field[Gershun 1936] Radiance as a function of position and direction • for general scenes • the “plenoptic function” • five-dimensional • L ( x, y, z, , f ) (w/m2sr) • in free space • four-dimensional • L ( u, v, s, t )

  3. bigbaseline small baseline Devices for recording light fields (using geometrical optics) • handheld camera [Buehler 2001] • camera gantry [Stanford 2002] • array of cameras [Wilburn 2005] • plenoptic camera [Ng 2005] • light field microscope [Levoy 2006]

  4. Light fields at micron scales • wave optics must be considered • diffraction limits the spatial × angular resolution • most objects are no longer opaque • each pixel is a line integral through the object • of attenuation • or emission • can reconstruct 3D structure from these integrals • tomography • 3D deconvolution

  5. High performance imagingusing large camera arrays Bennett Wilburn, Neel Joshi, Vaibhav Vaish, Eino-Ville Talvala, Emilio Antunez, Adam Barth, Andrew Adams, Mark Horowitz, Marc Levoy (Proc. SIGGRAPH 2005)

  6. Stanford multi-camera array • 640 × 480 pixels ×30 fps × 128 cameras • synchronized timing • continuous streaming • flexible arrangement

  7. Manex’s bullet time array Ways to use large camera arrays • widely spaced light field capture

  8. Ways to use large camera arrays • widely spaced light field capture • tightly packed high-performance imaging

  9. Ways to use large camera arrays • widely spaced light field capture • tightly packed high-performance imaging • intermediate spacing synthetic aperture photography

  10. Synthetic aperture photography

  11. Synthetic aperture photography

  12. Synthetic aperture photography

  13. Synthetic aperture photography

  14. Synthetic aperture photography

  15. Synthetic aperture photography

  16. Example using 45 cameras[Vaish CVPR 2004]

  17. Light field photography using a handheld plenoptic camera Ren Ng, Marc Levoy, Mathieu Brédif, Gene Duval, Mark Horowitz and Pat Hanrahan (Proc. SIGGRAPH 2005 and TR 2005-02)

  18. Conventional versus plenoptic camera

  19. uv-plane st-plane Conventional versus plenoptic camera

  20. Adaptive Optics microlens array 125μ square-sided microlenses Prototype camera 4000 × 4000 pixels ÷ 292 × 292 lenses = 14 × 14 pixels per lens Contax medium format camera Kodak 16-megapixel sensor

  21. Σ Digital refocusing • refocusing = summing windows extracted from several microlenses Σ

  22. A digital refocusing theorem • an f / N light field camera, with P × P pixels under each microlens, can produce views as sharp as an f / (N × P) conventional camera – or – • it can produce views with a shallow depth of field ( f / N ) focused anywhere within the depth of field of an f / (N × P) camera

  23. Example of digital refocusing

  24. Refocusing portraits

  25. Action photography

  26. Extending the depth of field conventional photograph,main lens at f / 4 conventional photograph,main lens at f / 22 light field, main lens at f / 4,after all-focus algorithm[Agarwala 2004]

  27. Macrophotography

  28. Digitally moving the observer • moving the observer = moving the window we extract from the microlenses Σ Σ

  29. Example of moving the observer

  30. Moving backward and forward

  31. Implications • cuts the unwanted link between exposure(due to the aperture) and depth of field • trades off (excess) spatial resolution for ability to refocus and adjust the perspective • sensor pixels should be made even smaller, subject to the diffraction limit 36mm × 24mm ÷ 2μ pixels = 216 megapixels 18K× 12K pixels 1800 × 1200 pixels × 10 × 10 rays per pixel

  32. Light Field Microscopy Marc Levoy, Ren Ng, Andrew Adams, Matthew Footer, and Mark Horowitz (Proc. SIGGRAPH 2006)

  33. A traditional microscope eyepiece intermediate image plane objective specimen

  34. A light field microscope (LFM) • 40x / 0.95NA objective ↓ 0.26μ spot on specimen× 40x = 10.4μ on sensor ↓ 2400 spots over 25mm field • 1252-micron microlenses ↓ 200 × 200 microlenses with12 × 12 spots per microlens sensor eyepiece intermediate image plane objective specimen → reduced lateral resolution on specimen= 0.26μ × 12 spots = 3.1μ

  35. A light field microscope (LFM) sensor

  36. Example light field micrograph • orange fluorescent crayon • mercury-arc source + blue dichroic filter • 16x / 0.5NA (dry) objective • f/20 microlens array • 65mm f/2.8 macro lens at 1:1 • Canon 20D digital camera ordinary microscope light field microscope

  37. f The geometry of the light fieldin a microscope • microscopes make orthographic views • translating the stage in X or Y provides no parallax on the specimen • out-of-plane features don’t shift position when they come into focus • front lens element size =aperture width + field width • PSF for 3D deconvolution microscopy is shift-invariant (i.e. doesn’t change across the field of view) objective lenses are telecentric

  38. Panning and focusing panning sequence focal stack

  39. Real-time viewer

  40. Other examples fern spore (60x, autofluorescence) mouse oocyte (40x, DIC) Golgi-stained neurons (40x, transmitted light)

  41. eyepiece Nikon 40x 0.95NA (dry) Plan-Apo Extensions • digital correction of aberrations • by capturing and resampling the light field

  42. Extensions • digital correction of aberrations • by capturing and resampling the light field

  43. Extensions • digital correction of aberrations • by capturing and resampling the light field correcting for aberrations caused by imaging through thick specimens whose index of refraction doesn’t match that of the immersion medium

  44. neutral density Extensions • digital correction of aberrations • by capturing and resampling the light field • multiplexing of variables other than angle • by placing gradient filters at the aperture plane,such as neutral density, spectral, or polarization

  45. wavelength neutral density Extensions • digital correction of aberrations • by capturing and resampling the light field • multiplexing of variables other than angle • by placing gradient filters at the aperture plane,such as neutral density, spectral, or polarization

  46. wavelength neutral density Extensions • digital correction of aberrations • by capturing and resampling the light field • multiplexing of variables other than angle • by placing gradient filters at the aperture plane,such as neutral density, spectral, or polarization ... or polarization direction ... or ??? • gives up digital refocusing?

  47. Extensions • digital correction of aberrations • by capturing and resampling the light field • multiplexing of variables other than angle • by placing gradient filters at the aperture plane,such as neutral density, spectral, or polarization • microscope scatterometry • by controlling the incident light fieldusing a micromirror array + microlens array

  48. Programmableincident light field • light source +micromirror array +microlens array • 800 × 800 pixels =40 × 40 tiles ×20 × 20 directions • driven by image from PC graphics card

More Related