Hardware-Assisted Visibility Sorting for Tetrahedral Volume Rendering

1. Hardware-Assisted Visibility Sortingfor Tetrahedral Volume Rendering Steven Callahan Milan Ikits João Comba Cláudio Silva

2. Overview • Introduction • Previous Work • Hardware-Assisted Visibility Sorting • Results • Future Work • Conclusion

3. Research Goal • Real-time volume rendering • Scalable (machine performance) • Data of arbitrary size • Simple and robust implementations

4. Volume Rendering Regular Irregular

5. Why Irregular Grids ? Unstructured grids are the preferred data type in scientific computations Level-Of-Detail (LOD) techniques intrinsically need unstructured grids El-Sana et al, Ben-Gurion

6. Optical Models Absorption plus emission Light s s

7. Compositing Front-to-back I1 I0 I2 1 2 0 I01 I2 2 01

8. Volume Rendering: (Intersection) Sampling + Sorting

9. Sampling: Triangle-Based Approach Class 1 (+, +, +, -) Class 2 (+, +, -, -) Projected Tetrahedra [Shirley-Tuchman 1990]

10. Sorting Application Object-Space Sorting i.e., let’s sort the geometry! Rasterization Image Space Display

11. Cell-Projection B 7 5 6 A 3 4 p 2 1 A < B p

12. Object-Space Sorting: Williams’ MPVO B < A A < C B < E C < E C < D E < F D < F Idea: Define ordering relations by looking at shared faces. D A C F B E Viewing direction

13. MPVO Limitations Missing relations!

14. XMPVO A < C B < D Idea: Using ray shooting queries to complement ordering relations. C D A B A < B Viewing direction

15. Sorting Application Object Space Rasterization Image-Space Sorting Display i.e., let’s sort the pixels!

16. Image-Space Sorting: A-Buffer • Idea: Keep a list of intersections for each pixel. [Carpenter 1984]

17. Cell-Projection With An A-Buffer

18. Cell-Projection With An A-Buffer

19. Cell-Projection With An A-Buffer

20. Cell-Projection With An A-Buffer

21. Cell-Projection With An A-Buffer

22. Cell-Projection With An A-Buffer

23. Cell-Projection With An A-Buffer

24. Cell-Projection With An A-Buffer

25. Cell-Projection With An A-Buffer

26. Cell-Projection With An A-Buffer

27. Cell-Projection With An A-Buffer

28. Cell-Projection With An A-Buffer

29. Cell-Projection With An A-Buffer Not sorted!

30. Cell-Projection With An A-Buffer Sorted!

31. A-Buffer Limitations 2 Number of Intersections: O(cn ) n x n pixels c cells • Problems • Time: sorting takes too long • Memory: storage too high

32. Sorting Application Object-Space Sorting Rasterization Image-Space Sorting Display

33. Approximate Object-Space Sorting 1

34. Approximate Object-Space Sorting 1 2

35. Approximate Object-Space Sorting 3 1 2

36. Approximate Object-Space Sorting 3 5 1 4 2

37. Approximate Object-Space Sorting 3 6 5 7 1 4 2

38. Approximate Object-Space Sorting 3 7 5 6 1 4 2 A Solution: Use an insertion-sort A-buffer!

39. Approximate Object-Space Sorting What about the space problem? 3 7 5 6 1 4 2  Use a conservative bound on the intersections

40. Hardware Assisted Visibility Sorting (HAVS) • Sort in image-space and object-space • Do an approximate object-space sorting of the cells on the CPU (i.e. sort by face centroid) • Complete the sort in image-space by using a fixed depth A-buffer (called a k-buffer) implemented on the GPU • Can handle non-convex meshes, has a low memory overhead, and requires minimal pre-processing of data

41. HAVS Overview

42. k-buffer • Fixed size A-buffer of depth k • Fragment stream sorter • Stores k entries for each pixel. Each entry consists of the fragment’s scalar value and its distance to the viewpoint • An incoming fragment replaces the entry that is closest to the eye (front-to-back compositing) • Given a sequence of fragments such that each fragment is within k positions from its position is sorted order, it will output the fragments in sorted order

43. k-buffer: Hardware Implementation r a b g g comp r comp b comp a comp v1 v2 d2 d1 v3 d4 d3 v4 d5 v6 v5 d6 • Use multiple render target capability of ATI graphics cards (ATI_draw_buffers in OpenGL) • Use P-buffer to accumulate color and opacity and three Aux buffers for the k-buffer entries P-buffer Aux 0 Aux 1 Aux 2

44. Fragment Shader Overview

45. Details • Fix incorrect screen-space texture coordinates caused by perspective-correct interpolation Projecting vertices to find tex coords Projecting tex coords in shader Perspective interpolation

46. Details • Simultaneously reading and writing to a buffer is undefined when fragments are rasterized in parallel

47. Details • The buffers are initialized and flushed using k screen-aligned rectangles with negative scalar values • Handling non-convex objects requires the exterior faces to be tagged with a negative distance d and keeping track of when we are inside or outside of the mesh with the sign of the scalar value v

48. Details • Early ray termination reads accumulated opacity and kills fragment if it is over a given threshold. Early z-test is currently not available on ATI 9800 when using multiple rendering targets

49. Pre-Integrated Transfer Function • Previous Work • Volume density optical model • Williams and Max 1992 • Pre-integration on GPU • Roettger et al. 2000 • 5 s to update a 128x128x128 table • Incremental pre-integration on CPU • Wieler et al. 2003 • 1.5 s to update a 128x128x128 table

50. Pre-Integrated Transfer Function S S f b l • Williams and Max