Surface Signals for Graphics

1. Surface Signals for Graphics John Snyder Researcher 3D Graphics Group Microsoft Research

2. Why Surface Signals? • Many useful types of surface signals: • texture map [Catmull74, Blinn&Newell76](color) • bump map [Max81] (normal) • displacement map [Cook 84] (geometric offset) • geometry image (geometry) • bidirectional texture function (precomputed shading) • self-transfer texture (spherical harmonic coefs) • … • Simplicity of regular 2D image • Support on current graphics hardware (e.g. pixel shaders) • Research questions: • How to generate and manipulate signals? • What new graphics architectures?

3. Surface Signal Research Projects Creation precomputed radiance transfer Parameterization signal-specialized param. Rendering signal-based graphics architecture Representation geometry images

4. Motivation for Precomputed Transfer • better light integration and light transport • dynamic, area lights • shadowing • interreflections • in real-time area light point light area lighting, no shadows area lighting, shadows

5. . . . . . . Basis 16 Basis 17 Basis 18 Self-Transfer Signal (25D) illuminate result Reduces shading to a 25D dot product (low-frequency lighting)

6. Self-Transfer Results (Diffuse) No Shadows/Inter Shadows Shadows+Inter

7. Self-Transfer Results (Glossy) No Shadows/Inter Shadows Shadows+Inter

8. Self-Transfer Demo

9. Parameterization of Surface Signals Geometry-based (know geometry only) Signal-specialized (know geometry+signal)

10. linear map singular values: γ , Γ g G Measuring Parameterization Quality 2D texture domain surface in 3D

11. T linear map singular values: γ , Γ Geometric Stretch Metric 2D texture domain surface in 3D geometric stretch = γ2 + Γ2 Parameterize = minimize surface integral of geometric stretch

12. Signal Stretch Metric domain surface f g h = gf signal Parameterize = minimize surface integral of signal stretch • geometric stretch: γf2 + Γf2 • signal stretch: γh2 + Γh2

13. Conformal [Floater97] Geometric stretch [Sander01] Signal stretch [Sander02]

14. Results: Scanned Color (64x64 texture) Geometric stretch Signal stretch

15. Results: Normal Map Geometric stretch Signal stretch 128x128 texture - multichart

16. Results: Precomputed Radiance Transfer Geometric stretch Signal stretch 25D signal – 256x256 texture

17. 3D graphics = 2D image processing? not quite → use images but of surface signals not views • synthesize images from 3D surface descriptions • run-time flexibility – change view, lighting, rendering params • compactness – single surface parameterization, not multiple views • high quality (global illumination)– resolution independence • cheap creation – no costly rigs & operator, easy to edit • as preprocess, convert 3D descriptions to 2D image reps (surface signals) to accelerate run-time • signals can be represented as regular 2D images • rendering via general, programmable image processing ops

18. Rendering Factorization global illumination computation is too expensive from scratch Preprocess(slow) Run-Time(fast) surface signals • 3D surfaces (meshes) • 3D graphics: • ray tracing, Monte Carlo integration, dynamics simulation, encoding • 2D images, streams • 2D image processing • decoding, interpolation / decimation, programmable pixel “shaders”, sample gather

19. End

20. People • Microsoft Research 3D Graphics Group: • Jim Blinn, Conal Elliot, Brian Guenter, Hugues Hoppe, Charles Loop, Don Mitchell, Kirk Olynyk, Peter-Pike Sloan, John Snyder, Turner Whitted • Collaborators: • Steven Gortler, Xianfeng Gu,Ziyad Hakura, Jesse Hall, Jan Kautz, Leonard McMillan, Pedro Sander, Zoe Wood