A cave system for interactive modeling of global illumination in car interior
1 / 33

A Cave System for Interactive Modeling of Global Illumination in Car Interior - PowerPoint PPT Presentation

  • Uploaded on

A Cave System for Interactive Modeling of Global Illumination in Car Interior. Kirill Dmitriev, Thomas Annen, Grzegorz Krawczyk, Rafal Mantiuk, Karol Myszkowski, and Hans-Peter Seidel Max-Planck-Institut f ür Informatik, Saarbrücken, Germany. Karol Myszkowski. High Dynamic Range.

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about ' A Cave System for Interactive Modeling of Global Illumination in Car Interior' - kenley

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
A cave system for interactive modeling of global illumination in car interior

A Cave System for Interactive Modeling of Global Illumination in Car Interior

Kirill Dmitriev, Thomas Annen,

Grzegorz Krawczyk, Rafal Mantiuk,

Karol Myszkowski, and Hans-Peter Seidel

Max-Planck-Institut für Informatik,

Saarbrücken, Germany

Karol Myszkowski

High Dynamic Range Illumination in Car Interior

Human eye adjusts comfortably up to 12 orders of magnitude

and can see simultaneously up to 4 orders of magnitude

Hdr pipeline acquisition
HDR Pipeline: Acquisition Illumination in Car Interior

  • Global illumination

  • Products: HDR cameras

    • Lars III (Silicon Vision), Autobrite (SMal Camera Technologies), HDRC (IMS Chips), LM9628 (National), Digital Pixel System (Pixim)

  • Technology [Nayar 2003]:

    • Trading-off spatial resolution

      • Multiple sensor elements within pixel

      • Spatially varying pixel exposures

      • Mosaicing with spatially varying filter

    • Trading-off temporal resolution

      • Multi-exposure LDR capture

    • Multiple image detectors (beam splitters)

    • Smart pixels

Hdr pipeline storage
HDR Pipeline: Storage Illumination in Car Interior

  • Several formats for still images

    • Radiance

    • OpenEXR (new industrial standard)

    • logLuv tiff

    • HDR JPEG


Hdr pipeline display
HDR Pipeline: Display Illumination in Car Interior

  • LDR displays: luminance compression required

    • Solution: tone mapping

    • Important factors in tone mapping selection

      • Dynamic range of display device

      • Video display conditions

        • Background lighting

      • Application

        • Just nice looking video

        • Visually plausible results

        • Optimizing visibility of details

        • Improving contrast …

  • HDR displays start to appear

    • Sunnybrook Technologies and University of British Columbia

Hdr pipeline applications
HDR Pipeline: Applications Illumination in Car Interior

IMS Chips

Realistic Image Synthesis Illumination in Car Interior

Light Reflection

Light Transport


Visual Display

Display Observer















Greenberg et al. Siggraph’97. Cornell University

Acquisition materials
Acquisition: Materials Illumination in Car Interior

  • Shift variant Bi-directional Reflectance Distribution Function (BRDF)

    • Lensch et al.

  • Transluscency

    • Goesele et al.

Acquisition luminaires
Acquisition: Luminaires Illumination in Car Interior

  • Luminaire spatial power distribution (Goesele et al.)

    • Near field photometry

    • Emitted energy represented as 4D light field

      • Relevant for light sources installed in the car interior


VR: rendering

Acquisition luminaires1
Acquisition: Luminaires Illumination in Car Interior

  • Natural lighting

    • Light probes and multi-exposure techniques

    • SpheroCam from Spheron VR

    • Emitted energy represented as an environment map

      • Relevant for external car illumination

Light probe

VR: rendering

Paul Debevec

Lighting simulation rendering
Lighting Simulation + Rendering Illumination in Car Interior

1) Photographs of mirror sphere at varying exposure times

3) Use as light source in Monte

Carlo radiosity algorithm

2) High-dynamic

range environment map

Philippe Bekaert

Our goal
Our Goal Illumination in Car Interior

  • CAVE system for car interior rendering

    • Real-time lighting simulation

    • Dynamic real world lighting

      • HDR video environment maps

    • Free observer position

      • Head tracking

  • Special focus:

    • Predict impact of quickly changing lighting conditions on the visibility of information displayed at the LCD panel

      • Precise modeling of light reflections from the LCD panel

      • Predicting effective contrast and information readability for various viewing angles

      • Taking into account light adaptation conditions for the human eye

Requirements for rendering algorithm
Requirements for Rendering Algorithm Illumination in Car Interior

  • 5x2 full screen resolution frames at interactive rates

  • LCD panel illumination must be computed precisely

  • Higher error tolerance for the car interior illumination

    • Only low spatial frequencies in reconstructed lighting acceptable

  • Distant lighting assumption holds

    • HDR environment maps acceptable

  • Lighting can change quickly

    • Relying on temporal coherence between frames impractical

  • Car interior geometry static

Rendering algorithm selection
Rendering Algorithm Selection Illumination in Car Interior

  • Requirements

    • Exploit all computational resources available in the CAVE: 20 CPUs + 10 GPUs

  • Rendering solution

    • Use Precomputed Radiance Transfer method for car interior: 10 GPUs + 10 CPUs

    • Use Final Gathering for the LCD panel: 10 CPUs

Spherical harmonics
Spherical Harmonics Illumination in Car Interior

  • Spherical Harmonics:

    • Orthonormal basis over the sphere

    • Analogous to Fourier transform

  • Projection:

  • Reconstruction:

  • Integration:

Rendering equation

Incident Light Illumination in Car Interior



Rendering Equation


Jan Kautz

Precomputed radiance transfer

into SH Illumination in Car Interior

into SH

light function:


Precomputed Radiance Transfer

"light vector"

"transfer vector"

Such a dot product must be computed for each mesh vertex: n= 25

Jan Kautz

Prt for arbitrary meshes
PRT for Arbitrary Meshes Illumination in Car Interior



Handling two sided geometry
Handling Two-Sided Geometry Illumination in Car Interior

  • Car geometry is two sided

    • Need to store SH coefficients for both sides and render geometry twice with back face culling

    • This leads to lower performance and twice larger memory consumption

  • Better store SH coefficients only for one side of geometry:

    • Only the rays going through the windows contribute to SH coefficients

    • User has to point out the mesh parts representing car windows

Lcd panel modeling
LCD Panel Modeling Illumination in Car Interior

  • Emission characteristic of the LCD-sandwich:

    • Spectral emission-function of the backlight

    • Transmission characteristic of the polarizer and other layers (e.g. BEF, d-BEF, LCF, rgb-filters, …)

    • Transmission characteristic of the LC-cell

    • Transmission of other optical elements – e.g. glass

  • Reflective characteristic in case of incoming light

    • Surface coating (e.g. AR/AG)

    • Reflection characteristic of the LCD sandwich


Incoming lighting




Computing lcd panel lighting
Computing LCD Panel Lighting Illumination in Car Interior

  • DIMOS or SPECTER systems can be used for lighting simulation within the LCD panel

    • We use just external spectral emissivity and reflectance data provided with a very high angular resolution

  • Algorithm

    • Cover geometry of LCD panel with texture

    • For each texel

      • Compute energy emitted in the observer direction

        • Use tabulated emissivity data

        • Modulate emissivity as a function of the displayed information

      • Compute energy reflected in the observer direction

        • Use the final gathering method

        • Improve performance using BRDF-weighted importance sampling

          • If traced ray hits the car window, query the environment map

          • If traced ray hits the car interior, query PRT data structures

System architecture
System Architecture Illumination in Car Interior

Distributing the computation
Distributing the Computation Illumination in Car Interior

  • Use “Lightning” system that takes care of OpenGL synchronization between different PCs

  • LCD display computation is distributed in an asynchronous way

Camera parameters

Distributing the computation1
Distributing the Computation Illumination in Car Interior

  • Use “Lightning” system that takes care of OpenGL synchronization between different PCs

  • LCD display computation is distributed in an asynchronous way

Broadcast the display image as it is ready

Tone mapping in the cave
Tone Mapping in the CAVE Illumination in Car Interior

  • Visual adaptation is influenced by light projected on a small area around the center of retina

  • Head tracking enables precise estimation of the gaze direction

    • We assume that the adaptation is affected by scene luminances within the region of 10° surrounding the gaze direction

    • This region can be mapped to 1-3 walls in the CAVE

    • Luminance data within this region is collected and send to the master

  • Master computes common tone mapping parameters and broadcasts them to all computers in the cluster

    • There is a small delay in illumination update, but it is hidden by temporal adaptation model anyway

Rendering with tone mapping
Rendering with Tone Mapping Illumination in Car Interior

  • 3 Passes:

    • Compute tone mapping parameters

      • Preview rendering of 128x128 HDR images for the CPU processing

    • Select visible geometry for tone mapping

      • Lighting OFF

      • Render geometry to z-buffer

    • Final rendering with tone mapping

      • Lighting ON

      • Use z-buffer test to tone map only visible fragments

Results car interior part
Results (Car Interior Part) Illumination in Car Interior

  • Car model contains approx. 500K triangles

  • Preprocessing of SH coefficients takes about 100 minutes

  • Full model size with SH coefficients is about 60 MB

  • Rendering frame rate is about 10 FPS

    • For each frame current environment map is projected into SH basis and lighting in every vertex is computed

    • Full image tone mapping is applied

Results lcd panel part
Results (LCD Panel Part) Illumination in Car Interior

  • One processor on each PC is busy with sending data to OpenGL

  • Another processor computes draft images (40 samples per pixel) of LCD panel and sends them to other PCs

  • Draft image display is available almost immediately after each head movement or environment maps rotation, converged image is computed in approx. 2 seconds

Conclusions Illumination in Car Interior

  • We proposed efficient global illumination and tone mapping solutions for CAVE VR applications involving static geometry and dynamic environment lighting

  • We successfully applied those solutions to the car interior modeling, efficiently utilizing all CPUs and GPUs resources on the CAVE cluster

  • We proposed an accurate algorithm for the LCD panel simulation and rendering based on measured BRDF and emission data.

Future work
Future work Illumination in Car Interior

  • Use HDR video stream as dynamic environment map lighting

    • Trivial to do, we are just waiting for a fish-eye lens suitable for our HDR camera

  • Use HDR display (0.05-3,000 cd/m2) to render the LCD panel with luminance levels similar to real world driving conditions

    • Display-in-display rendering problem

  • Consider recently proposed techniques for all frequency PRT lighting

Acknowledgements Illumination in Car Interior

  • We would like to thank the following people

    • Michael Arnold (Virtual Reality Center, DaimlerChrysler AG)

    • Thomas Ganz (Research & Technology Displays and Controls (RBP/BM), DaimlerChrysler AG)

    • Matthias Bues (VR Lab., Fraunhofer IAO)

Adaptive logarithmic tone mapping
Adaptive Illumination in Car InteriorLogarithmic Tone Mapping

Tone mapping time dependent visual adaptation
Tone Mapping: Time-Dependent Visual Adaptation Illumination in Car Interior

  • Light adaptation

  • Dark adaptation