Virtual Light Transport and Real-Time LF Rendering

1. Virtual Light Transport andReal-Time LF Rendering Insu Yu i.yu@cs.ucl.ac.uk

2. Scopes • Light Transport for Full Global Illumination • General BRDF Support • Point & Area Light Sources • Reusing Visibility Field • Independent of Scene Complexity • Real-Time Rendering walk-through on GI

3. Overall Structures

4. Light Transport

5. Plenoptic Function Reduce to 5D Parameter space (Ignore Time, Selective wavelength)

6. Plenoptic Function • Flow of Light • The set of rays visible from any point in space, at any time, and over any range of wavelengths Plenoptic (x, y, z, ,  , λ, t ) • In graphics term, a set of all possible environment maps for a scene

7. Light Transport (Radiance) Equation Plenoptic(, ,λ,x,y,z) →L(p,ω) • Solving the Radiance Equation provides the Plenoptic Function for Light Field • L(p, wi) = incoming radiance at point p along the direction wi • V(p)i = Visibility (mutually visible) • G(p, w) = cos θ = Geometric Term • Le(p, wi) = Emitted Radiance • F(p, wi ,w) = scattering term (BRDF function) • Ω= union of all surfaces • dw = Solid angle

8. Neumann Series • Define a linear operator M • The rendering equation (rewritten using operator notation) L(x) = Le(x) + ML(x) (I - M)L(x) = Le(x) • Apply inverse of (I - M) to the emission function

9. Parameterisation

10. Parameterisation • Discrete Parameterisation of Plenoptic function Plenoptic(, ,λ,x,y,z) →L(p,ω)→ L(u,v,s,t)λ • Parameterisation • Position(x,y,z): Uniform Plane UV-Subdivision • Direction(, ): Hemi-sphere Subdivision (ST) • Material-Based Parameterisation • Exploit Material Properties to choose subdivision levels.

11. Parameterisation(Position) • Material-Based Parameterization • Position(x,y,z): UV plane subdivision • Flexible Subdivision Level • Possible Multi-resolution/Quad-Tree Structure • F(u,v) is differential tixel with uniform characteristic Position Subdivision

12. Parameterisation (Directions) • BRDF-Based Subdivision • Directional Maps • Spherical Map • Cube Environment Map • ,  Projection Map • Uniform Tetrahedron Map • Uniform Spherical Grid • Paraboloid Map Ideal Diffuse Glossy Ideal Specular

13. Uniform Spherical Grid Tetrahedron Subdivision Uniform Spherical Grid

14. Uniform Spherical UV Grid

15. Uniform Spherical Grid • Analytic Function to represent Uniform Subdivision of Hemisphere • Constant Search time for Direction and Neighbours • Solid Angle Variance is low • Minimum distance to nearest cell is smaller • Flexible Direction • BUT Computationally expensive (sin, cos)

16. Paraboloid Maps • Analytic Function • Reversible Mapping • A Fast Parameterisation • Simple Query of Direction • Solid Angle Variation is smaller then spherical or cube environment maps. top left right front bottom

17. Paraboloid Projection • Mapping Direction (x, y, z) unit vector to (s, t) texture coordinate in range [-1, 1] • Reverse Mapping (s, t) to (x, y, z)

18. Propagation

19. Radiance Equation Revisited Radiance = Emitted Radiance + Total Reflected Radiance

20. Irradiance Estimation Irradiance Map

21. Visibility Field • Finding Mutually Visible Ray Segment • Ray Cast for every two pair of tixels on surface • Binary Information (Visible:1 Invisible:0) • Most Computational Complex • Store Visibility Field (possibly compressed form) • Reuse Visibility Field rather than recalculate them

22. Linear Operator (BRDF) • Define a linear operator, M. • Redistribute Incoming Energy • Apply Local BRDF to Radiance Textures • Specular reflection: A simple reordering of tixel postion • Add out-going Energy to the Light Field

23. Diffuse Reflectance (Lambertian) = × Outgoing Energy Incoming Energy Local Operator

24. Ideal Specular Reflectance = × Outgoing Energy Incoming Energy Local Operator • BRDF is non-zero in only one exitant direction

25. Glossy Reflectance = × Outgoing Energy Incoming Energy Local Operator Analytic Operator is very expensive, instead approximation function is used

26. Local Operator (BRDF Function)

27. Energy Transfer Function • Pre-calculate Visibility Field • Energy Transfer • For Every Radiance Tixels • For Visible Tixels • Calculate Incoming Energy using Radiance Estimation • Apply Local BRDF to every Radiance textures • Repeat Propagation and Local Operator until system is converged

28. Light Field

29. Light Field Radiance Map (Low) • Hemi-sphere Radiance L(p,ω) Map • A collection of Directional Radiance

30. Light Field Radiance Map(High)

31. Global Tone Mapping • Employ a simple Reinhard’s Tone mapping • Convert RGB to YUV Model and rescale the Luminance • Apply to All Radiance Textures

32. Compression

33. Compression on Textures • Light Field Compression • Tone-mapped RGB Directional Radiance textures • Huge Coherences on Radiance Textures • S3TC Hardware-Accelerated compression • Apply Entropy Encoding (LZW) on Differential Textures • Radiance Textures are a stream of images

34. Hardware Compression (S3TC) • Lossy compression with 6:1 Ratios (Degradation) S3TC Compressed Image Original Image

35. Real-Time Rendering

36. Rendering • Geometry + Light Field (Image-Based) Rendering • Benefit of Analytic Geometry Structures • Radiance Textures per polygon • Scalability of Texture size • Possible Lens model Rendering using Directional Radiance Texture maps • 16 Neighbours Interpolation

37. Real-Time Rendering • Aim • Minimum 20 fps/sec on a Single computer • Full Screen Mode Size (640 x 480) • Real-Time Walkthrough of Global Illumination • Scene complexity independent speed

38. Maximising Resources • OpenGL False Colour • Dual CPUs (Xeon 2.8Ghz) • Single Instruction Multiple Data (Intel SSE2) • Hyper-Threading Technology • Parallel Processing (CPU + GPU) • Inline Assembly

39. Hyper-Threading • Hyper-Threading (Pentium 4 above) • Multi-Threads concurrently process non-sharing data

40. Handling Threads • Create/Destroy Thread is Expensive • Create Thread Once and Use Resume, Suspend, Functions • Talk to Main Thread with Signal Function • Main thread is ideal WaitForMultipleObjects • Main Thread Thread 1 Thread 2 Thread 3 Thread 4 Thread 5 - User Input & OpenGL (Next Iteration) • Final Drawing on Screen

41. Rendering • OpenGL False Colour Rendering (Ray-Cast) • Get Polygon ID j and WC2TEXMatrix (Convert Matrix from World coordinate to Local Coordinates) • Transform View point Origin P0 and Directional Point P1 to polygon local coordinates system • Find Intersection Point in a local UV coordinates • Find ST values from direction D • Get a Radiance at Fj(U,V,S,T)

42. Results - Reconstruction Radiances size 16x16, 32x32, 64x64/ Textures 256x256

43. Results – Lambertian

44. Results

45. Results – Ideal Specular

46. Results – Global Tone mapping Increasing Gamma Increasing Exposure

47. Results - Compression • Compression of High Dynamic Range Light Field LF(u=64,v=64, s=128,t=128)

48. Rendering Frames Rates

49. Future Works • Transparent surfaces • BSSRDF (bi-directional sub-surface scattering reflectance distribution function) • Dynamic Compression on Visibility Field • Hardware-Based Implementation

50. Question ?