1 / 27

Lightfields, Lumigraphs, and Image-based Rendering

Lightfields, Lumigraphs, and Image-based Rendering. 2D to 4D. More than can be captured in one image All light passing through a region of space General (5D) vs. unoccluded space (4D). “Rebinning” pixels. Image-Based Modeling and Rendering.

aglaia
Download Presentation

Lightfields, Lumigraphs, and Image-based Rendering

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Lightfields, Lumigraphs, and Image-based Rendering

  2. 2D to 4D • More than can be captured in one image • All light passing through a region of space • General (5D) vs. unoccluded space (4D) “Rebinning” pixels

  3. Image-Based Modeling and Rendering • Generate new views of a scene directly from existing views • “Pure” IBR (such as lightfields): no model of scene • Other IBR techniques try to obtain higher quality with less storage by building a model

  4. Lightfields • Advantages: • Simpler computation vs. traditional CG • Cost independent of scene complexity • Cost independent of material properties and other optical effects • Disadvantages: • Static geometry • Fixed lighting • High storage cost

  5. Using Lightfields • Obtain 2D slices of 4D data set • Arbitrary views: take other 2D slices • Challenges: • Capture • Parameterization • Compression • Rendering

  6. Capturing Lightfields • Need a 2D set of (2D) images • Choices: • Camera motion: human vs. computer • Constraints on camera motion • Coverage and sampling uniformity • Aliasing

  7. Lightfield Parameterization • Point / angle • Two points on a sphere • Points on two planes • Original images and camera positions

  8. Compression • Compress individual images (JPEG, etc.) • Adapt video compression to 2D arrays • Decomposition into basis functions • Vector quantization

  9. Rendering • How to select rays? • Interpolation • Taking advantage of hardware • Graphics hardware • Compression hardware

  10. Implementations • Lightfields, SIGGRAPH 96,Levoy and Hanrahan • Lumigraphs, SIGGRAPH 96,Gortler et al. • Unstructured lumigraphs, SIGGRAPH 01, Buehler et al.

  11. Light Field Rendering • Capture: • Computer-controlled camera rig • Move camera to grid of locations on a plane

  12. Light Field Rendering • Parameterization: • Two planes, evenly sampled: “light slab” • In general, planes in arbitrary orientations • In practice, one plane = camera locations • Minimizes resampling

  13. Light Field Coverage

  14. Multi-Slab Light Fields

  15. Rendering • For each desired ray: • Compute intersection with (u,v) and(s,t) planes • Take closest ray • Variants: interpolation • Bilinear in (u,v) only • Bilinear in (s,t) only • Quadrilinear in (u,v,s,t)

  16. Light Field Compression • Based on vector quantization • Preprocessing: build a representative codebook of 4D tiles • Each tile in lightfield represented by index • Example: 2x2x2x2 tiles, 16 bit index = 24:1 compression

  17. Lightfield Compression byVector Quantization • Advantages: • Fast at runtime • Reasonable compression • Disadvantages: • Preprocessing slow • Manually-selected codebook size • Does not take advantage of structure

  18. The Lumigraph • Capture: move camera by hand • Camera intrinsics assumed calibrated • Camera pose recovered from markers

  19. Lumigraph Postprocessing • Obtain rough geometric model • Chroma keying (blue screen) to extract silhouettes • Octree-based space carving • Resample images to two-plane parameterization

  20. Lumigraph Rendering • Use rough depth information to improve rendering quality

  21. Lumigraph Rendering • Use rough depth information to improve rendering quality

  22. Without usinggeometry Using approximate geometry Lumigraph Rendering

  23. Unstructured Lumigraph Rendering • Further enhancement of lumigraphs:do not use two-plane parameterization • Store original pictures: no resampling • Hand-held camera, moved around an environment

  24. Unstructured Lumigraph Rendering • To reconstruct views, assign penalty to each original ray • Distance to desired ray, usingapproximate geometry • Resolution • Feather near edges of image • Construct “camera blending field” • Render using texture mapping

  25. Unstructured Lumigraph Rendering Blending field Rendering

  26. Other Lightfield Acquisition Devices • Spherical motionof camera aroundan object • Samples space ofdirections uniformly • Second arm tomove light source –measure BRDF

  27. Other Lightfield Acquisition Devices • Acquire an entirelight field at once • Video rates • Integrated MPEG2compression foreach camera (Bennett Wilburn, Michal Smulski, Mark Horowitz)

More Related