A cave system for interactive modeling of global illumination in car interior
This presentation is the property of its rightful owner.
Sponsored Links
1 / 33

A Cave System for Interactive Modeling of Global Illumination in Car Interior PowerPoint PPT Presentation


  • 103 Views
  • Uploaded on
  • Presentation posted in: General

A Cave System for Interactive Modeling of Global Illumination in Car Interior. Kirill Dmitriev, Thomas Annen, Grzegorz Krawczyk, Rafal Mantiuk, Karol Myszkowski, and Hans-Peter Seidel Max-Planck-Institut f ür Informatik, Saarbrücken, Germany. Karol Myszkowski. High Dynamic Range.

Download Presentation

A Cave System for Interactive Modeling of Global Illumination in Car Interior

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


A cave system for interactive modeling of global illumination in car interior

A Cave System for Interactive Modeling of Global Illumination in Car Interior

Kirill Dmitriev, Thomas Annen,

Grzegorz Krawczyk, Rafal Mantiuk,

Karol Myszkowski, and Hans-Peter Seidel

Max-Planck-Institut für Informatik,

Saarbrücken, Germany

Karol Myszkowski


A cave system for interactive modeling of global illumination in car interior

High Dynamic Range

Human eye adjusts comfortably up to 12 orders of magnitude

and can see simultaneously up to 4 orders of magnitude


Hdr pipeline acquisition

HDR Pipeline: Acquisition

  • Global illumination

  • Products: HDR cameras

    • Lars III (Silicon Vision), Autobrite (SMal Camera Technologies), HDRC (IMS Chips), LM9628 (National), Digital Pixel System (Pixim)

  • Technology [Nayar 2003]:

    • Trading-off spatial resolution

      • Multiple sensor elements within pixel

      • Spatially varying pixel exposures

      • Mosaicing with spatially varying filter

    • Trading-off temporal resolution

      • Multi-exposure LDR capture

    • Multiple image detectors (beam splitters)

    • Smart pixels


Hdr pipeline storage

HDR Pipeline: Storage

  • Several formats for still images

    • Radiance

    • OpenEXR (new industrial standard)

    • logLuv tiff

    • HDR JPEG

  • HDR MPEG


Hdr pipeline display

HDR Pipeline: Display

  • LDR displays: luminance compression required

    • Solution: tone mapping

    • Important factors in tone mapping selection

      • Dynamic range of display device

      • Video display conditions

        • Background lighting

      • Application

        • Just nice looking video

        • Visually plausible results

        • Optimizing visibility of details

        • Improving contrast …

  • HDR displays start to appear

    • Sunnybrook Technologies and University of British Columbia


Hdr pipeline applications

HDR Pipeline: Applications

IMS Chips


A cave system for interactive modeling of global illumination in car interior

Realistic Image Synthesis

Light Reflection

Light Transport

Simulation

Visual Display

Display Observer

emission

geometry

BRDF

displayed

image

radiometric

values

27

15

22

13

HDR

HDR

LDR

Greenberg et al. Siggraph’97. Cornell University


Acquisition materials

Acquisition: Materials

  • Shift variant Bi-directional Reflectance Distribution Function (BRDF)

    • Lensch et al.

  • Transluscency

    • Goesele et al.


Acquisition luminaires

Acquisition: Luminaires

  • Luminaire spatial power distribution (Goesele et al.)

    • Near field photometry

    • Emitted energy represented as 4D light field

      • Relevant for light sources installed in the car interior

measurement

VR: rendering


Acquisition luminaires1

Acquisition: Luminaires

  • Natural lighting

    • Light probes and multi-exposure techniques

    • SpheroCam from Spheron VR

    • Emitted energy represented as an environment map

      • Relevant for external car illumination

Light probe

VR: rendering

Paul Debevec


Lighting simulation rendering

Lighting Simulation + Rendering

1) Photographs of mirror sphere at varying exposure times

3) Use as light source in Monte

Carlo radiosity algorithm

2) High-dynamic

range environment map

Philippe Bekaert


Our goal

Our Goal

  • CAVE system for car interior rendering

    • Real-time lighting simulation

    • Dynamic real world lighting

      • HDR video environment maps

    • Free observer position

      • Head tracking

  • Special focus:

    • Predict impact of quickly changing lighting conditions on the visibility of information displayed at the LCD panel

      • Precise modeling of light reflections from the LCD panel

      • Predicting effective contrast and information readability for various viewing angles

      • Taking into account light adaptation conditions for the human eye


Requirements for rendering algorithm

Requirements for Rendering Algorithm

  • 5x2 full screen resolution frames at interactive rates

  • LCD panel illumination must be computed precisely

  • Higher error tolerance for the car interior illumination

    • Only low spatial frequencies in reconstructed lighting acceptable

  • Distant lighting assumption holds

    • HDR environment maps acceptable

  • Lighting can change quickly

    • Relying on temporal coherence between frames impractical

  • Car interior geometry static


Rendering algorithm selection

Rendering Algorithm Selection

  • Requirements

    • Exploit all computational resources available in the CAVE: 20 CPUs + 10 GPUs

  • Rendering solution

    • Use Precomputed Radiance Transfer method for car interior: 10 GPUs + 10 CPUs

    • Use Final Gathering for the LCD panel: 10 CPUs


Spherical harmonics

Spherical Harmonics

  • Spherical Harmonics:

    • Orthonormal basis over the sphere

    • Analogous to Fourier transform

  • Projection:

  • Reconstruction:

  • Integration:


Rendering equation

Incident Light

Visibility

Cosine

Rendering Equation

ReflectedLight

Jan Kautz


Precomputed radiance transfer

into SH

into SH

light function:

transfer:

Precomputed Radiance Transfer

"light vector"

"transfer vector"

Such a dot product must be computed for each mesh vertex: n= 25

Jan Kautz


Prt for arbitrary meshes

PRT for Arbitrary Meshes

After

Before


Handling two sided geometry

Handling Two-Sided Geometry

  • Car geometry is two sided

    • Need to store SH coefficients for both sides and render geometry twice with back face culling

    • This leads to lower performance and twice larger memory consumption

  • Better store SH coefficients only for one side of geometry:

    • Only the rays going through the windows contribute to SH coefficients

    • User has to point out the mesh parts representing car windows


Lcd panel modeling

LCD Panel Modeling

  • Emission characteristic of the LCD-sandwich:

    • Spectral emission-function of the backlight

    • Transmission characteristic of the polarizer and other layers (e.g. BEF, d-BEF, LCF, rgb-filters, …)

    • Transmission characteristic of the LC-cell

    • Transmission of other optical elements – e.g. glass

  • Reflective characteristic in case of incoming light

    • Surface coating (e.g. AR/AG)

    • Reflection characteristic of the LCD sandwich

observer

Incoming lighting

glassplate

LCD-sandwich

backlight


Computing lcd panel lighting

Computing LCD Panel Lighting

  • DIMOS or SPECTER systems can be used for lighting simulation within the LCD panel

    • We use just external spectral emissivity and reflectance data provided with a very high angular resolution

  • Algorithm

    • Cover geometry of LCD panel with texture

    • For each texel

      • Compute energy emitted in the observer direction

        • Use tabulated emissivity data

        • Modulate emissivity as a function of the displayed information

      • Compute energy reflected in the observer direction

        • Use the final gathering method

        • Improve performance using BRDF-weighted importance sampling

          • If traced ray hits the car window, query the environment map

          • If traced ray hits the car interior, query PRT data structures


System architecture

System Architecture


Distributing the computation

Distributing the Computation

  • Use “Lightning” system that takes care of OpenGL synchronization between different PCs

  • LCD display computation is distributed in an asynchronous way

Camera parameters


Distributing the computation1

Distributing the Computation

  • Use “Lightning” system that takes care of OpenGL synchronization between different PCs

  • LCD display computation is distributed in an asynchronous way

Broadcast the display image as it is ready


Tone mapping in the cave

Tone Mapping in the CAVE

  • Visual adaptation is influenced by light projected on a small area around the center of retina

  • Head tracking enables precise estimation of the gaze direction

    • We assume that the adaptation is affected by scene luminances within the region of 10° surrounding the gaze direction

    • This region can be mapped to 1-3 walls in the CAVE

    • Luminance data within this region is collected and send to the master

  • Master computes common tone mapping parameters and broadcasts them to all computers in the cluster

    • There is a small delay in illumination update, but it is hidden by temporal adaptation model anyway


Rendering with tone mapping

Rendering with Tone Mapping

  • 3 Passes:

    • Compute tone mapping parameters

      • Preview rendering of 128x128 HDR images for the CPU processing

    • Select visible geometry for tone mapping

      • Lighting OFF

      • Render geometry to z-buffer

    • Final rendering with tone mapping

      • Lighting ON

      • Use z-buffer test to tone map only visible fragments


Results car interior part

Results (Car Interior Part)

  • Car model contains approx. 500K triangles

  • Preprocessing of SH coefficients takes about 100 minutes

  • Full model size with SH coefficients is about 60 MB

  • Rendering frame rate is about 10 FPS

    • For each frame current environment map is projected into SH basis and lighting in every vertex is computed

    • Full image tone mapping is applied


Results lcd panel part

Results (LCD Panel Part)

  • One processor on each PC is busy with sending data to OpenGL

  • Another processor computes draft images (40 samples per pixel) of LCD panel and sends them to other PCs

  • Draft image display is available almost immediately after each head movement or environment maps rotation, converged image is computed in approx. 2 seconds


Conclusions

Conclusions

  • We proposed efficient global illumination and tone mapping solutions for CAVE VR applications involving static geometry and dynamic environment lighting

  • We successfully applied those solutions to the car interior modeling, efficiently utilizing all CPUs and GPUs resources on the CAVE cluster

  • We proposed an accurate algorithm for the LCD panel simulation and rendering based on measured BRDF and emission data.


Future work

Future work

  • Use HDR video stream as dynamic environment map lighting

    • Trivial to do, we are just waiting for a fish-eye lens suitable for our HDR camera

  • Use HDR display (0.05-3,000 cd/m2) to render the LCD panel with luminance levels similar to real world driving conditions

    • Display-in-display rendering problem

  • Consider recently proposed techniques for all frequency PRT lighting


Acknowledgements

Acknowledgements

  • We would like to thank the following people

    • Michael Arnold (Virtual Reality Center, DaimlerChrysler AG)

    • Thomas Ganz (Research & Technology Displays and Controls (RBP/BM), DaimlerChrysler AG)

    • Matthias Bues (VR Lab., Fraunhofer IAO)


Adaptive logarithmic tone mapping

Adaptive Logarithmic Tone Mapping


Tone mapping time dependent visual adaptation

Tone Mapping: Time-Dependent Visual Adaptation

  • Light adaptation

  • Dark adaptation


  • Login