Real-time Global Illumination with precomputed probe

1. Real-time Global Illumination with precomputed probe Baek Jun Kim KAIST (Korea Advanced Institute of Science and Technology)

2. Review • Ambient Occlusion Rendering with Neural Networks (20184230 김세훈) • 1) Deep shading (2D frame based learning) • 2) DRAO (3D geometry based learning)

3. Global Illumination • Direct illumination only • Global illumination (indirect illumination)

4. Ray Tracing • Ray Tracing • To implement Global Illumination • In Real-time? • Too costly to compute at interactive rates • Even several hours for a image • How to realize Real-time Ray Tracing • Precomputation • … and others

5. Papers • Real-Time Global Illumination using Precomputed Light Field Probes • Real-time Global Illumination by Precomputed Local Reconstruction from Sparse Radiance Probes

6. Real-Time Global Illumination using Precomputed Light Field Probes Morgan McGuire, Michael Mara, Derek Nowrouzezahrai, and David Luebke. In I3D 2017. 11.

7. Summary Precomputing Probe Radiance, normal, and distance map Intersection testing with distance map Ray casting using Probe Shading with Probes 1) Spatial-Temporal Radiance Denoising (BRDF factorization) 2) Irradiance with Pre-Filtered Visibility (Geometry aware)

8. Real-Time GI using Precomputed Light Field Probes • A scene computed in 42ms (212fps) • Let me show its previous works briefly: • Image Based Lighting • Voxel Cone Tracing

9. Image-based Lighting • Cube map (Environment mapping) • In this paper, every probe get its cube map.

10. Voxel Cone Tracing • Using Sparse Voxel Octree • Optimized in modern GPU • Hierarchical

11. Precomputed Light Field Probes • Rasterization to render cube map • Octahedral parameterization for efficiency • Two-level MIPmap hierarchy (heuristic)

12. Light Field Probe Ray Tracing • How to use precomputed probe? • We can trace a ray! • Procedure • Selecting a probe • Single-probe tracing routine • Selecting fall-back probes

13. Selecting a Probe • The probe whose center lies closest to the ray Ray Probe

14. Single-probe tracing routine

15. Intersection test • Intersection test by comparing depth map value in the probe and radial distance from the probe’s center Ray Depth map in the probe Surface Probe

16. Intersection test (2) • If depth map value is smaller than radial distance, the ray has either hit a surface or pass behind the surface Ray Depth map in the probe Surface Probe

17. Intersection test (3) • By using normal map in probe, we can distinguish between HIT and behind the surface Ray Depth map in the probe Surface Probe

18. Selecting fall-back probes • If unable to find a definitive hit or miss, pick another probe

19. Shading with Light Field Probes • Now we can trace a ray with probes, then? • 3 Global Illumination Algorithm • Compute indirect lighting using MC & Importance Sampling • Unbiased, but slow • We have to reduce the number of ray samples • Spatial-Temporal Radiance Denoising • Irradiance with Pre-Filtered Visibility

20. Spatial-Temporal Radiance Denoising • Factorize BRDF into Lambertian term and Glossy Term • Indirect illumination pass traces one ray sampled from each of them and writes the result to each buffer • For Lambertian reflection, we compute incident irradiance instead of radiance

21. Spatial-Temporal Radiance Denoising (2) • Lambertian reflections is spatially slower-varying • Use of wider filter • Glossy reflection is spatially faster-varying • Use of narrower filter

22. Irradiance with Pre-Filtered Visibility • How to compute irradiance? • Precompute irradiance map • Interpolate irradiance map at eight probes within the cube of the intersection • Reduce number of texture operation → really fast • Limitation: Lighting and darkness leaking

23. Irradiance with Pre-Filtered Visibility (2) • Geometry-aware approach • While precomputing irradiance map, we also compute radial distance map • At runtime, we use the radial distance map for interpolating irradiance

24. Summary Precomputing Probe Radiance, normal, and distance map Intersection testing with distance map Ray casting using Probe Shading with Probes 1) Spatial-Temporal Radiance Denoising (BRDF factorization) 2) Irradiance with Pre-Filtered Visibility (Geometry aware)

25. Real-Time Global Illumination using Precomputed Light Field Probes Ari Silvennoinen and Jaakko Lehtinen. ACM Transactions on Graphics, November 2017.

26. Summary • To get indirect illumination, combined two methods • Precomputed Light Transport • Interpolation • Factorize Global Transport into (1) Local Transport and (2) Probe Transport

27. Real-time GI by Precomputed Local Reconstruction from Sparse Radiance Probes • GI = Direct illumination + Indirect illumination • How to get indirect illumination?

28. How to get indirect illumination? • In this paper, • Precomputed light transport • Direct-to-Indirect transfer [Hašan et al. 2006] • Interpolation method • Interpolated irradiance and bandlimited radiance [E.g. Ward 1992]

29. Direct-to-Indirect transfer [Hašan et al. 2006] • Precomputed radiance transfer method • Precompute transfer matrix that map direct illumination into indirect illumination

30. Interpolation method • Interpolation from radiance probes • are spatial interpolation weight ω p2 p1 x

31. Limitations • a) Direction mismatch • b) Visibility mismatch

32. Visibility-Aware Interpolation • Ψ for direction • Vi(ω) for Visibility

33. Probe Radiance • High-resolution angular sampling of the probes’ radiance is expensive • Replace it by Spherical Harmonics form • λ: probe radiance vector • Y: SH basis function (basis func. index j) Spherical harmonics is a kind of Laplace transform of Sphere

34. Local transport Operator • Local transport Operator transforms the probe radiance vector λ to interpolated incident radiance at x as a function of the continuous direction ω

35. Precomputed Transport • transport coefficients ϕij

36. Factorized transport • Factorize global transport into local transport and probe transport • Interpolate with probes

37. Local Transport • Precomputed local transport matrix: accounts for near field effects between the receiver and the mutually visible surfaces seen by the probes • Vi(ω) makes matrix

38. Probe Transport • Captures radiance in a sparse set of probes at runtime

39. Scene Example

40. Scene Exmaple (2)

41. Implementation - Precomputation • Probe locations and radii: • Every surface point is under the support of at least one probe • Every surface point is visible from at least one probe • Use greedy algorithm • Cluster Receivers into a vector

42. Implementation - Runtime • Computing Probe Transport • Direct illumination light map is updated every frame

43. Result

44. Result

45. Summary • To get indirect illumination, combined two methods • Precomputed Light Transport • Interpolation • Factorize Global Transport into (1) Local Transport and (2) Probe Transport

46. Pros and Cons • Real-Time Global Illumination using Precomputed Light Field Probes • Dynamic Camera • Static and Dynamic Environment • Real-time Global Illumination by Precomputed Local Reconstruction from Sparse Radiance Probes • Only Static Environment • Dynamic Light Source

47. Quiz • In Real-Time Global Illumination using Precomputed Light Field Probes, which map of the probe used to test intersection during Light Field Probe Ray Tracing? (1) Shadow map (2) Distance map (3) Irradiance map (4) Radiance map • In Real-time Global Illumination by Precomputed Local Reconstruction from Sparse Radiance Probes, global transport is factorized into 2 parts: precomputed ( ) transport and ( ) transport which is computed at runtime (1) probe, local (2) local, probe (3) SH, SVD (4) SVD, SH