lightfields lumigraphs and image based rendering n.
Skip this Video
Loading SlideShow in 5 Seconds..
Lightfields, Lumigraphs, and Image-based Rendering PowerPoint Presentation
Download Presentation
Lightfields, Lumigraphs, and Image-based Rendering

Loading in 2 Seconds...

  share
play fullscreen
1 / 27
Download Presentation

Lightfields, Lumigraphs, and Image-based Rendering - PowerPoint PPT Presentation

aglaia
174 Views
Download Presentation

Lightfields, Lumigraphs, and Image-based Rendering

- - - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

  1. Lightfields, Lumigraphs, and Image-based Rendering

  2. 2D to 4D • More than can be captured in one image • All light passing through a region of space • General (5D) vs. unoccluded space (4D) “Rebinning” pixels

  3. Image-Based Modeling and Rendering • Generate new views of a scene directly from existing views • “Pure” IBR (such as lightfields): no model of scene • Other IBR techniques try to obtain higher quality with less storage by building a model

  4. Lightfields • Advantages: • Simpler computation vs. traditional CG • Cost independent of scene complexity • Cost independent of material properties and other optical effects • Disadvantages: • Static geometry • Fixed lighting • High storage cost

  5. Using Lightfields • Obtain 2D slices of 4D data set • Arbitrary views: take other 2D slices • Challenges: • Capture • Parameterization • Compression • Rendering

  6. Capturing Lightfields • Need a 2D set of (2D) images • Choices: • Camera motion: human vs. computer • Constraints on camera motion • Coverage and sampling uniformity • Aliasing

  7. Lightfield Parameterization • Point / angle • Two points on a sphere • Points on two planes • Original images and camera positions

  8. Compression • Compress individual images (JPEG, etc.) • Adapt video compression to 2D arrays • Decomposition into basis functions • Vector quantization

  9. Rendering • How to select rays? • Interpolation • Taking advantage of hardware • Graphics hardware • Compression hardware

  10. Implementations • Lightfields, SIGGRAPH 96,Levoy and Hanrahan • Lumigraphs, SIGGRAPH 96,Gortler et al. • Unstructured lumigraphs, SIGGRAPH 01, Buehler et al.

  11. Light Field Rendering • Capture: • Computer-controlled camera rig • Move camera to grid of locations on a plane

  12. Light Field Rendering • Parameterization: • Two planes, evenly sampled: “light slab” • In general, planes in arbitrary orientations • In practice, one plane = camera locations • Minimizes resampling

  13. Light Field Coverage

  14. Multi-Slab Light Fields

  15. Rendering • For each desired ray: • Compute intersection with (u,v) and(s,t) planes • Take closest ray • Variants: interpolation • Bilinear in (u,v) only • Bilinear in (s,t) only • Quadrilinear in (u,v,s,t)

  16. Light Field Compression • Based on vector quantization • Preprocessing: build a representative codebook of 4D tiles • Each tile in lightfield represented by index • Example: 2x2x2x2 tiles, 16 bit index = 24:1 compression

  17. Lightfield Compression byVector Quantization • Advantages: • Fast at runtime • Reasonable compression • Disadvantages: • Preprocessing slow • Manually-selected codebook size • Does not take advantage of structure

  18. The Lumigraph • Capture: move camera by hand • Camera intrinsics assumed calibrated • Camera pose recovered from markers

  19. Lumigraph Postprocessing • Obtain rough geometric model • Chroma keying (blue screen) to extract silhouettes • Octree-based space carving • Resample images to two-plane parameterization

  20. Lumigraph Rendering • Use rough depth information to improve rendering quality

  21. Lumigraph Rendering • Use rough depth information to improve rendering quality

  22. Without usinggeometry Using approximate geometry Lumigraph Rendering

  23. Unstructured Lumigraph Rendering • Further enhancement of lumigraphs:do not use two-plane parameterization • Store original pictures: no resampling • Hand-held camera, moved around an environment

  24. Unstructured Lumigraph Rendering • To reconstruct views, assign penalty to each original ray • Distance to desired ray, usingapproximate geometry • Resolution • Feather near edges of image • Construct “camera blending field” • Render using texture mapping

  25. Unstructured Lumigraph Rendering Blending field Rendering

  26. Other Lightfield Acquisition Devices • Spherical motionof camera aroundan object • Samples space ofdirections uniformly • Second arm tomove light source –measure BRDF

  27. Other Lightfield Acquisition Devices • Acquire an entirelight field at once • Video rates • Integrated MPEG2compression foreach camera (Bennett Wilburn, Michal Smulski, Mark Horowitz)