Spherical Harmonics in Actual Games

1. Spherical Harmonics inActual Games Tom Forsyth Muckyfoot Productions Ltd tomf@muckyfoot.com

2. What they are… • Representation of an image over a sphere • Something like a cube-map • Not like a cube-map: • Frequency-space representation • Spherical equivalent of post-Fourier-transform data • A bit like a JPEG, but a 2D sphere, not a 2D plane • Continuous, rotationally invariant • No sharp edges, no rotational “lumpiness” • Easy for vector units to manipulate

3. What they are not… • Not an algorithm • Just a data representation • Not Precomputed Radiance Transform • That is an algorithm that happens to use SHs • Not actually that complex • Just the maths language barrier • Read the Ramamoorthi & Hanrahan paper – it makes sense eventually!

4. The Basics • “Looking up a direction in the cube-map” • A set of N basis functions – used by all SHs • Components of the direction being looked up • 1, X, Y, Z, XY, YZ, ZX, X2-Y2, 1-3Z2,… • A set of N weights – this stores the data • Each weight is usually an RGB colour • Multiply each basis function by its weight • And sum – result is the looked-up value • Number of basis functions goes on for ever, but first 9 is good enough for diffuse lighting

5. Features of Harmonics • Smooth over the sphere • No seams or “hot spots” • Easy to reconstruct in vector unit • Rotation loses no data, has no aliasing • Rotate then convert to SH is identical to convert to SH then rotate • Basis functions used are orthonormal • To add two SHs, add their weights • To multiply two SHs, multiply their weights

6. Irradiance Concept • Ignore SH for now – think of cubemaps • Representing incoming distant lights • Representing the diffuse cosine response • Cosine response = max(0,N.L), summed for all lights • Only normals matter, not positions • Draw response to single light on cubemap • The “picture” of a lit sphere • Add together each light’s cubemap • To shade any object, look up normal on the resulting cubemap

7. Irradiance Concept

8. Irradiance Concept

9. Irradiance Concept

10. Using SH • Final cubemap is smooth, low frequency • Encode as SH weights • Feed weights to vertex unit • Vertex unit shades normals with SH • 9 RGB values to feed to vertex unit rather than a cubemap • 9 weights represents diffuse shading with maximum 6% error

11. Using SH Better • But wait! Why use a cubemap at all? • Go from light description to SH weights • Adding two SHs = add weights together • So: • For each light, find the SH weights • Sum weights • Feed final weights to vertex unit • No cubemaps anywhere!

12. CPU Code per Object per Light void AddLightToSH ( Colour *sharm, Colour &colour, Vector &dirn ) { sharm[0] += colour * fConst1; sharm[1] += colour * fConst2 * dirn.x; sharm[2] += colour * fConst2 * dirn.y; sharm[3] += colour * fConst2 * dirn.z; sharm[4] += colour * fConst3 * (dirn.x * dirn.z); sharm[5] += colour * fConst3 * (dirn.z * dirn.y); sharm[6] += colour * fConst3 * (dirn.y * dirn.x); sharm[7] += colour * fConst4 * (3.0f * dirn.z * dirn.z - 1.0f); sharm[8] += colour * fConst5 * (dirn.x * dirn.x - dirn.y * dirn.y); }

13. Vertex Processor Code const Colour sharm[9]; Colour LightNormal ( Vector &dirn ) { Colour colour = sharm[0]; colour += sharm[1] * dirn.x; colour += sharm[2] * dirn.y; colour += sharm[3] * dirn.z; colour += sharm[4] * (dirn.x * dirn.z); colour += sharm[5] * (dirn.z * dirn.y); colour += sharm[6] * (dirn.y * dirn.x); colour += sharm[7] * (3.0f * dirn.z * dirn.z - 1.0f); colour += sharm[8] * (dirn.x * dirn.x - dirn.y * dirn.y); return colour; }

14. In Practice • Find brightest n lights (n=1 to 3 usually) • Shade those conventionally • Specular, bumpmapped, self-shadowing, etc • Rest of lights accumulated into SH • Last real light fades between real and SH • Provides smooth transition as object moves

15. Gotchas • Only vertex diffuse lighting • No bumpmaps (PS2.0 shaders can!) • No specular, only diffuse • So do brightest 2 lights normally • Only directional lights • Point lights must be approximated • Discontinuities on abutting objects (walls, etc)

16. Cool Stuff • Take (HDR) skybox, convert to SH offline • Add to all computed SHs • Perfect sunset lighting • Artists can tweak by drawing on bitmaps! • Presampled “radiosity” • Pick some sample points in environment • Render cubemaps at those points, convert to SH • At runtime, for any object, linearly interpolate between nearest sample point SHs. • Gives lighting off walls, through doorways, etc

17. Demo • Compare many real lights to just 2 • Too dark – lights are discarded. • Compare many real lights to 2 + ambient • Too washed out • Compare many real lights to just SH • Pretty good – slight differences • Compare many lights to 2 real + SH • Excellent, plus bumpmapping works

18. Questions

19. Conclusions • SH is excellent for images on a sphere • SH irradiance makes lights cheap • Standard lighting has per-vertex cost • Using SHs converts it to a per-object cost • Per-vertex cost is constant • Artists can put down far more lights • And the game is still faster! (10-20% on PS2)

20. References • Ramamoorthi & Hanrahan • “An Efficient Representation for Irradiance Environment Maps” • Peter-Pike Sloan • Siggraph 2002 & 2003 papers • Robin Green • “Spherical Harmonic Lighting: The Gritty Details” • Google for “spherical harmonic irradiance” • Without “irradiance” you get lots of chemists