Enhancing Virtual Reality Through Advanced Graphics and Algorithms

1. Graphics Topics in VR By Shaun Nirenstein

2. Overview • Impact of graphics/geometric algorithms on VR • Visibility in VR • Rendering Topics in VR • Collision Detection • Animation (I won’t talk about this)

3. Impact of graphics/geometric algorithms on VR • Constrains the VR • Visibility • Scene complexity • Performance • Shadows • Cues • Soft shadows offer more information about the light source • Shadows are difficult(slow), soft shadows are more difficult(slow)

4. Impact of graphics/geometric algorithms on VR • Constrains the VR (cont.) • Illumination models • Defines the set of surface materials available • Global vs. local illumination • What can be done at real-time? • Global specular? • Global diffuse? • Arbitrary BRDF (Bidirectional Reflectance Distribution Function)? • Collision detection • How accurate? • How fast? • (Agent/Avatar) Animation • Scripted, captured, simulated

5. Visibility • Obvious application is to manage geometric complexity • Don’t render what you cannot see • Less obvious application is to manage global complexity • Don’t “simulate” invisible objects • Can X see Y (e.g. AI) • Don’t illuminate invisible objects • Predictive cache management!!! • Could Y be visible from X soon • Invisible geometry, textures, bump maps, vertex/pixel programs do not have to be resident

6. Visibility Algorithms • From-point techniques: • What is visible from view point X • From-region techniques: • What is visible from view region R • I.e. Y is visible from R, if there exists a point P in R, which can see Y • Can bind a visible set to time

7. From Point Visibility • Occluder shadow volumes Invisible Occluder

8. Occluder Fusion (from point) • Occluder shadow volumes Invisible Occluder Invisible??? Invisible

9. Occluder Fusion (from point) • Cells and portals

10. Occluder Fusion (from point) • Cells and portals

11. Occluder Fusion (from point) • Cells and portals

12. Problems • Cells and portals • Only work (well) for “architectural scenes” • Occluder Shadows • Fusion is difficult • Only works well for a small number of occluders • Other techniques offer various improvements and trade-offs • Hierarchical Occlusion Maps • Occlusion bit testing • Hierarchical Z-Buffer • Many more…

13. From Region Visibility • Occluder shadow volumes Invisible Occluder

14. From Region Visibility • Area light source analogy Separating lines Penumbra Umbra Supporting lines Penumbra

15. From Region Visibility • Area light source analogy Umbra?

16. Occluder Fusion (from region) • Cells and portals

17. Occluder Fusion (from region) • Cells and portals

18. Occluder Fusion (from region) • Cells and portals (visible volume)

19. Occluder Fusion (from region) • Cells and portals (visible volume)

20. Occluder Fusion (from region) • Cells and portals (visible cell iff. stabbing line does exist)

21. Finding a stabber (2D) • Find a line which separates two set of points

22. Finding a stabber (2D) • Find a line which separates two set of points • Left and right sets (defined by orientation)

23. Finding a stabber (2D) • Find a line which separates two set of points • Left and right sets (defined by orientation)

24. Finding a stabber (2D) • Find a line which separates two set of points • Left and right sets (defined by orientation) • Solved using duality

25. Finding a stabber (3D) • Also solved with duality • Lines in 3D go to points in 5D Pluecker coordinates

26. Constructing Cells/Portals • BSP Tree – Splits volume until leaves are convex • Portals are sides of leaf nodes which do not correspond to scene polygons

27. Constructing Cells/Portals • Optimal portal finding is an open problem • Portals / Cells (sectors) are usually defined by hand

28. Soft and Hard Shadows • Hard Shadows – small (points) or far • Soft Shadows – area/volume light source Hard Soft

29. Soft and Hard Shadows • Soft shadows != Blurring!!! Light Occluder Ground

30. Soft and Hard Shadows • Algorithms • Point sources are easy! • Clip shadow volume against scene • Ray tracing • Shadow map • Shadow volume • Soft shadows are difficult! • Soft shadow volumes • Point sampling of area light source • Ray tracing • Hard and soft shadows may be pre-computed by sampling surface geometry and applying lightmaps

31. Lightmaps • Can be used for shadows, lighting, diffuse illumination • Need to be recomputed when geometry and/or lights move = *

32. Global vs. Local Illumination • Global looks better – local is faster • Trade offs between complexity, performance, ability to move geometry, quality, etc. • What do games do? • Radiosity preprocess into light maps • Light grid for dynamic objects • Bump mapping is applied in addition • Many passes: textures, bumps, lightmaps, shadow passes, etc. • Doom III? • No global illumination • Not necessary for the “feel” of the game • Allows for dynamic hard shadows (cast BY everything ONTO everything) • Photo-realism does not add as much to mood as shadows in this context

33. Photon Mapping

34. Photon Mapping

35. Photon Mapping

36. Collision Detection • Why? • Need to know if movement results in solid bodies colliding • Physics • Visibility

37. Collision Detection • Broad Phase • Conservative – only test whether or not further testing(narrow phase) is required • Uses bounding volumes and bounding hierarchies

38. Collision Detection • Narrow Phase • Accurate • Triangle-triangle intersection • Segment-triangle intersection • Bounding-volume triangle intersection

39. Collision Detection(cont.) • What makes a good bounding box? • Easy to test for collision • Tightly represents the model – minimal extraneous volume • More information implies better potential • What about moving objects? • On the fly creation? • Conservative updates • Tracking nearest geometry

40. Collision Detection(cont.) • AABB (Axially aligned bounding boxes) • Two points in 3D (6 scalars) • BS (Bounding Sphere ) • Point and radius (4 scalars) • Not always a good fit – may require OBB (Oriented bounding box) • Three points in 3D (9 scalars ) • Principal component analysis

41. Collision Detection(cont.) • Convex Hull • Accurate • Conservative • bounding planes • faster, but less accurate

42. Collision Detection(cont.) • k-dops (discrete oriented polytopes) • Accurate – more planes • AABB == 6-dop

43. Collision Detection(cont.) • 8-dop 45o

44. Collision Detection(cont.) • Hierarchies – AABB • Test top down

45. Collision Detection(cont.) • Hierarchies – OBBTree • Test top down

46. Collision Detection(cont.) • Polytope-polytope intersection (narrow phase) • Assuming non-containment: Intersection is equivalent to the existence of an edge of one polytope which intersects the face of another Containment