Multisample Antialiasing

1. Multisample Antialiasing Kurt Akeley CS248 Lecture 6 11 October 2007 http://graphics.stanford.edu/courses/cs248-07/

2. Overview Two approaches to antialiasing • Pre-filter • Over-sample: increase sample rate Pre-filter • Works well for points and lines, not for triangles • Can’t reasonably represent pre-filtered triangles • Can’t handle occlusion • Can effectively eliminate aliasing with a moderately-large filter window Over-sample • Works well for all primitives • Though less well than pre-filtering for points and lines • Cannot eliminate aliasing! • Let’s see why …

3. Aliasing of fundamentals and harmonics Imperfect analogy, but useful pedagogical tool: • Fundamental: spatial density of edges • Harmonics: spectrum introduced by a single edge Over-sampling cannot overcome increased fundamental frequency • Demo But over-sampling, followed by a resampling at a lower rate, can overcome harmonics due to a single edge

4. Practical solution Over-sample to reduce jaggies due to edges • Optimize for the case of a single edge Use other approaches to limit edge rates • Band-limit fundamental geometric frequencies • E.g., level-of-detail (LOD) model selection • Band-limit image fundamentals and harmonics • E.g., texture MIPmapping

5. LOD model selection Low LOD Medium LOD High LOD Demo

6. Supersampling

7. Supersampling Supersampling algorithm: • Over-sample, e.g., at 4x the pixel rate • Reconstruct at the over-sampled rate • Band-limit to match the pixel rate • Resample at the pixel rate to yield pixel values • Reconstruct to display

8. Sample at 4x pixel rate Pixel-rate fundamental * x = = f(x) F(s)

9. Reconstruct 4x samples * x Cutoff at ½ the over-sample rate = = Aliased ! f(x) F(s)

10. Band-limit 4x reconstruction to pixel-rate * x = = f(x) F(s)

11. Resample at pixel rate * x = = f(x) F(s)

12. Reconstruct pixel samples * Rect filter approximates LCD display x = = f(x) F(s)

13. Optimizations The over-sample reconstruction convolution and the band-limiting convolution steps can be combined: • Convolution is associative • (f*g) * h=f * (g * h) • And g and h are constant • f * (g * h) = f * filter The filter convolution can be reduced to a simple weighted sum of sample values: • The result is sampled at pixel rate • So only values at pixel centers are needed • These are weighted sums of the 4x samples

14. Optimized supersampling Do once • Compute filter by convolving the over-sample reconstruction and band-limiting filters • Compute filter weights for a single pixel sample by sampling the filter waveform During rendering: • Over-sample • Filter to compute pixel values from sample values using the filter weights During display • Reconstruct

15. Sample Patterns

16. Sample patterns Pattern categories • Regular grid • Sparse regular grid • Pseudo-random (stochastic) Filter extent (filter combines reconstruction and band-limiting) • Within the unit square of a pixel • Beyond the unit square Pattern variation • Temporal • Spatial Shading rate vs. geometric sampling rate • Same • Different

17. Regular-grid sample patterns Implementation • Regular sample spacing simplifies attribute evaluation • Mask, color, and depth • Large sample count is expensive Edge optimization • Poor, especially for screen-aligned edges Image quality • Good for large numbers of samples Single Pixel

18. Sparse regular-grid sample patterns 8-rooks pattern Implementation • Regular sample spacing simplifies attribute evaluation • Mask, color, and depth • Moderate sample count is cost-effective Edge optimization • Good, especially for screen-aligned edges Image quality • Good for moderate numbers of samples 8-queens pattern Single Pixel

19. NVIDIA Quincunx Quincunx pattern Quincunx is a special example of a sparse regular grid • Only two samples per pixel (center and one corner) • Filter accesses five samples • Center • Four corners Implementation • Regular sample spacing simplifies attribute evaluation • Mask, color, and depth • Minimal sample count (2 per pixel) is very cost-effective Edge optimization • OK, though less optimal for screen-aligned edges Image quality • Better than without antialiasing Single Pixel This information is from memory, I did not confirm it with NVIDIA.

20. Stochastic sample patterns random pattern Implementation • Irregular sample spacing complicates attribute evaluation • Mask, color, and depth • Moderate sample count is cost-effective Edge optimization • Good, though less optimal for screen-aligned edges Image quality • Excellent for moderate numbers of samples Single Pixel

21. Filter extent Within pixel unit square • Near optimal for small sample counts (<=8) • Equal sample weighting is typical • Easily parallelizable • Simplifies immediate filtering Beyond the pixel unit square • Required for large sample counts (>=16) • Sinc/Gaussian sample weights • Implementation is more difficult • May filter only during display Single Pixel “The SAGE Graphics Architecture”, Michael Deering and David Naegle, Proceedings of SIGGRAPH 2002

22. Variation in sample patterns Vary patterns • Temporally: frame to frame • Spatially: pixel to pixel Improves image quality by replacing regular aliasing patterns with noise Complicates implementation Interferes with repeatability • Violated for temporal variation • Requires spatial pattern to be window-aligned

23. The accumulation buffer OpenGL mechanism to support supersampling Additional color buffer • Same spatial dimensions as the frame buffer • Greater color resolution and range (-1,1) Used to accumulate (sum) the results of multiple rendering passes Many applications • Supersample antialiasing • Motion blur • Depth of field blur • Combinations of these effects and more …

24. Accumulation buffer supersampling glutInitDisplay(… | GLUT_ACCUM); // setup desired rendering modes glAccum(GL_LOAD, 0.0); for (int i=0; i<n; ++i) { // jitter scene to sample location i // render the entire scene glAccum(GL_ACCUM, sampleweight[i]); } glAccum(GL_RETURN, 1.0);

25. Supersampling features and limitations Features • Unconstrained sample locations (stochastic) • Unconstrained filter shapes and extents • Adaptive quality / performance trade-off • Utilitizing glAccum(GL_MULT, factor); Limitations • No per-pixel variation in sample pattern • Low performance

26. Multisampling

27. Shading rate (another pattern variation) Shading - the computation of color for a fragment Supersampling • Shades each sample • E.g., accumulation buffer • Or shades at an independently specified rate • E.g., REYES Multisampling is an optimized form of supersampling • Optimized • To eliminate jaggies due to edge aliasing • For efficiency and performance • Shades each fragment once (potentially huge savings!) • There are no geometric edges in a fragment • Use pre-filtering to eliminate edges in shading source data • Stores color with each sample • There may be edges within a pixel

28. Multisample implementation (n samples) Application struct { float x,y,z,w; float r,g,b,a;} vertex; Vertex assembly Vertex operations struct { vertex v0,v1,v2 } triangle; Primitive assembly Point sampling at multisample locations sets mask and computes depths. Color is taken from center-most sample. struct { short x,y; bool mask[n]; float depth[n]; float r,g,b,a;} fragment; Primitive operations Rasterization Fragment operations Fragment samples are depth-merged into the multisample buffer, then filtered to the color buffer. struct { short R,G,B; int depth[n]; byte r[n],g[n],b[n];} pixel; Frame buffer Display

29. Multisample rasterization operations Fragment selection • Identify pixels for which fragments are to be generated • New: generate fragment if any sample is within the primitive • Requires tiled sampling, rather than point sampling • Generates more fragments Attribute assignment • Assign attribute values to each fragment • Sample color at the center of the pixel (as before) • New: compute the Boolean per-sample coverage mask • True if-and-only-if the sample is within the primitive • New: compute depth values for each sample location

30. Point-sampled fragment selection Generate fragment if pixel center is inside triangle Implements point-sampled aliased rasterization 9 fragments generated

31. Tiled fragment selection Generate fragment if unit square intersects triangle Implements multisample rasterizations • 4x4 sample pattern with unit-square filter extent 21 fragments generated

32. Multisampling in OpenGL “Multisampling is trivial to add to an application.” ― OpenGL Programming Guide glutInitDisplay(… | GLUT_MULTISAMPLE); glEnable(GL_MULTISAMPLE); // render as usual, almost everything works

33. Occlusion revisited The conundrum: • Pre-filtering blurs geometry • Accurate occlusion requires sampling of exact (sharply defined) geometry • Can’t have both  A partial solution: • Pre-filtered geometry cannotocclude accurately • occluded objects bleed through the seems • But pre-filtered geometry canbe occluded by exact geometry

34. Getting the best of both approaches glutInitDisplay(…, GLUT_MULTISAMPLE); // setup occlusion modes (e.g. depth buffering) glEnable(GL_MULTISAMPLE); // render triangles (must happen first) glDisable(GL_MULTISAMPLE); glEnable(GL_LINE_SMOOTH); glEnable(GL_BLEND); glBlendFunc(GL_SRC_ALPHA, GL_ONE); // render lines and/or points (must happen last) glDisable(GL_LINE_SMOOTH); glDisable(GL_BLEND);

35. Summary Supersampling does not eliminate aliasing • It can reduce jaggies • It must be used with other pre-filtering to eliminate aliasing Multisampling • Is supersampling with pixel-rate shading • Optimized for efficiency and performance • Is a full-scene antialiasing technique • Works for all rendering primitives • Interacts well with other OpenGL mechanisms • Especially fragment/frame buffer operations • Handles occlusions correctly • Pre-filtering does not • Can be used in conjunction with pre-filter antialiasing • To optimize point and line quality

36. Assignments Before next Tuesday’s class, read • FvD 5 • OpenGL multisample specification(s) • http://www.opengl.org/registry/specs/SGIS/multisample.txt • http://www.opengl.org/registry/specs/ARB/multisample.txt Project 2: • Will be assigned next Tuesday

37. End