SIGGRAPH 2005 Course on Interactive Ray Tracing Session IV: Using Realtime Ray Tracing

1. SIGGRAPH 2005 Course on Interactive Ray TracingSession IV:Using Realtime Ray Tracing Ingo Wald SCI Institute, University of Utah wald@openrt.de

2. Using Realtime Ray Tracing • First of all: Important announcement

3. Using Realtime Ray Tracing • First of all: Important announcement • We want YOU to play around with RTRT… • Not only listen in abstract way, but really play with it ! • Write cool applications… • Write cool shaders… • Build a user-base of realtime ray tracing • Be “early adapters” and “pioneers” of this new technology… • Come up with new ideas (and new problems…) !

4. Using Realtime Ray Tracing • First of all: Important announcement • We want YOU to play around with RTRT… • Therefore: As of this SIGGRAPH, noncommercial version of OpenRT publicly available ! More infos: www.openrt.de

5. Getting OpenRT • Today: Public (noncommercial) release • Only for noncommercial use ! • Only available for PC/Linux (for now) • Slightly restricted (Scene size, no cluster-mode, …) • Reason: Limit potential sources of problems/questions/… • Actual download • Available shortly after SIGGRAPH (next week…) • For infos, see http://www.openrt.de • DONT SEND email to noncommercial@openrt.de

6. Outline • Using OpenRT (programming guide, tutorial,…) • What is OpenRT • Writing frontend applications with/for OpenRT • Writing OpenRT shaders • Practical applications of Realtime Ray Tracing • … using OpenRT, of course...

7. What is OpenRT ? What is OpenRT ?

8. What is OpenRT ? What is OpenRT ? • Actually, three different things

9. What is OpenRT ? What is OpenRT ? • Actually, three different things • A Project: The Saarland Realtime Ray Tracing Project

10. What is OpenRT ? What is OpenRT ? • Actually, three different things • A Project: The Saarland Realtime Ray Tracing Project • A specific realtime ray tracing rendering engine • Outcome of the OpenRT project • RTRT core, shared library with SDK

11. What is OpenRT ? What is OpenRT ? • Actually, three different things • A Project: The Saarland Realtime Ray Tracing Project • A specific realtime ray tracing rendering engine • Outcome of the OpenRT project • RTRT core, shared library with SDK • A Realtime Ray Tracing API • Not necessarily restricted to current rendering engine • SaarCOR/RPU soon also driven by same API.

12. What is OpenRT ? What is it NOT ? • A “animation program” • Rather: a rendering engine (available as library) • A “global illumination renderer” • Rather: “tool” to build such… • OpenRT in itself only a ray tracer • Not automatically global illumination/caustics etc(can write shaders, but don’t have to…) • Something that solves all your problems… • … it just makes solving them much easier.

13. Using Realtime Ray TracingPart I:- Programming Applications with OpenRT - Ingo Wald SCI Institute, University of Utah wald@openrt.de

14. Using Realtime Ray Tracing:Programming with OpenRT • The OpenRT API • Architecture and design guidelines • Brief API overview • Tutorial: Writing OpenRT applications • Performance optimizations • Tips & Tricks

15. OpenRT APIMotivation • Sessions I-III: Have discussed technical issues of realtime ray tracing • Fast ray traversal and intersection • Handling dynamic scenes • Fast CPU Implementation • Hardware • … • Together: Have all TECHNICAL components of a complete rendering engine

16. OpenRT APIMotivation • How to make that technology available to the user(s) ? • Need an API • Lack of common API one of the historical problems of RT ! • Hundreds of known/published improvements (better algorithms, optimizations, tricks, …) • But: usually each only implemented in one software package • Not directly available to others, slow spreading of technology • Result: Many, many different ray tracers out there … • … but most use only tiny part of known optimizations/technology

17. OpenRT APIMotivation • Lack of common API one of the historical problems of RT ! • Result: Many, many different ray tracers out there … • … but most use only tiny part of known optimizations/technology

18. OpenRT APIMotivation • Lack of common API one of the historical problems of RT ! • Result: Many, many different ray tracers out there … • … but most use only tiny part of known optimizations/technology • Test: Who in this room has implemented • A ray tracer ?

19. OpenRT APIMotivation • Lack of common API one of the historical problems of RT ! • Result: Many, many different ray tracers out there … • … but most use only tiny part of known optimizations/technology • Test: Who in this room has implemented • A ray tracer ? • A kd-tree ?

20. OpenRT APIMotivation • Lack of common API one of the historical problems of RT ! • Result: Many, many different ray tracers out there … • … but most use only tiny part of known optimizations/technology • Test: Who in this room has implemented • A ray tracer ? • A kd-tree ? • A kd-tree packet traverser ?

21. OpenRT APIMotivation • Lack of common API one of the historical problems of RT ! • Result: Many, many different ray tracers out there … • … but most use only tiny part of known optimizations/technology • Test: Who in this room has implemented • A ray tracer ? • A kd-tree ? • A kd-tree packet traverser ? • SSE ?

22. OpenRT APIMotivation • Lack of common API one of the historical problems of RT ! • Result: Many, many different ray tracers out there … • … but most use only tiny part of known optimizations/technology • Test: Who in this room has implemented • A ray tracer ? • A kd-tree ? • A kd-tree packet traverser ? • SSE ? • A (good) kd-tree construction (SAH) ?

23. OpenRT APIMotivation • Lack of common API one of the historical problems of RT ! • Result: Many, many different ray tracers out there … • … but most use only tiny part of known optimizations/technology • Test: Who in this room has implemented • A ray tracer ? • A kd-tree ? • A kd-tree packet traverser ? • SSE ? • A (good) kd-tree construction (SAH) ? • Got the point ?

24. OpenRT APIMotivation (Standardized) APIs are crucial for new technologies • Allow to build a common user base • Knowledge base of how to use the technology • Allows users to abstract from underlying technology • Can become productive without having to understand all details • Allows technicians to continually improve the technology • If hidden behind API, user won’t even see any changes… • Technical improvements directly reach the users • If similar to exiting/known APIs, yield steep learning curve  Need to define a (quasi-)standard API for RTRT

25. OpenRT APIMotivation • Problem: What is a “good” API for realtime ray tracing • Existing “ray tracing” APIs: Inherently offline(POVRay, RenderMan,…) • Existing “realtime” APIs: • Not designed to support advantages/demands of ray tracing • Historically all designed for rasterization • Need to build our own

26. OpenRT APIDesign Goals • As powerful as RenderMan etc. • Offer all features of ray tracing (shaders, complex geometry, …) • As similar to OpenGL as possible: • Except where it doesn’t make sense… • Allows for easy “migration” of existing OpenGL knowledge • Be as low-level as possible  Flexibility • Optimally support existing RTRT implementations • Suitable for parallelization, handling dynamic scenes, … • But: Hide technological aspects (distribution, HW) where possible • Users just don’t (want to) care about distribution, different kd-trees, …

27. OpenRT APIDesign Goals • Fundamental design decision: Differentiate between “application API” and “shader API” • RenderMan-like for shader writers • C++ framework, shader classes, recursive ray tracing • OpenGL like for application programmers • Just “load” and “use” shaders like e.g., display lists • Describing geometry etc. like with OpenGL

28. OpenRT Core Application API:Very much like OpenGL… Geometry: Almost exactly like OpenGL • Vertices: rtVertex3f(…), rtNormal3f(…), rtColor3f(…), • Triangles: rtBegin(RT_TRIANGLES/RT_POLYGON/…); … rtEnd(); • Typical OpenGL transformation operations available • rtPushMatrix(), rtRotatef(…), rtLoadMatrixf(…), … • But: No Immediate-mode rendering • Rather: Geometry objects (Remember part II on dynamic scenes ?)

29. OpenRT Core Application API:Very much like OpenGL… Geometry objects • Define/use like OpenGL display lists • rtGenObjects(…), rtNewObject(…), rtEndObject(…) • Each object gets its own ADS • ADS built once on object definition (invisible to user) • Objects lateron “instantiated”: rtInstantiate(…) • Like calling a display list • Instance generated with current transformation stack • Support framework for dynamic scenes (see Part II) • Difference to OpenGL: MUST use objects

30. The OpenRT API:Defining Geometry • A simple example: Trivial for any OpenGL programmerint objID; rtGenObjects(&objID,1); rtNewObject(objID,RT_COMPILE); rtPushMatrix(); rtTranslatef(1,0,0); rtBindShader(shaderID); rtBegin(RT_TRIANGLES); rtVertex3f(….); rtNormal3f(…); … rtEnd(); rtPopMatrix(); rtEndObject();

31. OpenRT Core Application API:Very much like OpenGL… Apart from Geometry • Texture objects • Exactly like OpenGL – Shaders access via tex.ID • Shader objects • All appearance-related operations done via shaders! • Camera shaders, light shaders, surface shaders, environment, … • No explicit handling of lights or materials any more • NO rtMaterialf, rtLightf, etc.  Use shaders instead… • “Well-known” helper functions • rtPerspective(…), rtLookAt(…), rtSwapBuffers(…), rtFlush(…)

32. OpenRT Core Application API:Loading and using shaders Similar to Stanford Prog.Shading API • Dynamically loaded from DLLs/.so’s: rtGenShaderClasses(&cID,1); rtNewShaderClass(cID, “simpleShader”, “libSimpleShader.so”); rtGenShaders(&ID,1); rtShader(ID); • App/shader communication: • Shader classes “export” parameters (by name) rtDeclareParameter(“diffuseColor”, PER_SHADER,offsetof(…),sizeof(…)); • App requests handle to shader’s parameter & writes it int diffuseHandle = rtParameterHandle(“diffuseColor”); rtParameter3fv(diffuseHandle,&red);

33. Writing OpenRT applicationsUsing light shaders • Lights in OpenRT • No explicit “light” calls (like rtLight(…)) • ALL realized via light shaders • Using light shaders • Generate just like surface shaders • Need to know light shader parameters • E.g., “PointLight” shader: “position”, “intensity”, “atten”, … • Typical light shaders (point,spot,directional) already included • Once generated, call “rtUseLight(RT_GLOBAL,lightID)”

34. Writing OpenRT applicationsOther kinds of shaders • Other kinds of shaders • Environment shader • Camera shader • Rendering Object • Handled just like light shaders • Load, parameterize, call “rtUse<…>(ID)” • Default implementations already available

35. Writing OpenRT applicationsTextures Handling textures: Two steps • Step 1: Declare the texture object • … just like OpenGL rtTexImage2D(texID,…) • Step 2: Pass texture ID to shader (as parameter) rtBindShader(shaderID); int handle = rtParameterHandle(“textureID”); rtParameter1i(handle,texID); • Multiple textures per shader: no problem…

36. Writing OpenRT applicationsRendering a frame • Once all geometry/lights/etc are set up: Render Frame • Calling “rtSwapBuffers()” renders currently specified scene • Into frame buffer provided by application • Note: In the distributed variant, rtSwapBuffers() has one frame latency: • In that case, always returns previous frame ! • Once frame is rendered: Display via e.g. OpenGL • Simple X11-based helper library (RTUT)available as well

37. Writing OpenRT applicationsAnimating the scene • Animating lights, shaders, camera, … trivial • Just change it’s respective shader parametersrtBindLight(lightID); rtParameter3f(LIGHT_POSITION,newPosition); • Animating geometry: Re-instantiate object(s) with new transformation matrix • See previous talk on “Handling Dynamic Scenes”… • Simplest (most typical) way: • Delete all instances every frame (rtDeleteInstances()) • Re-traverse Scenegraph • Instantiate all objects with updated transformation matrix

38. Writing OpenRT applicationsAnimating the scene • More advanced way of doing that: • Change transformation of only a single instance:rtLoadMatrixf(&newTransformation); rtSetInstanceXfm(instID); • Much more efficient, but often harder to implement • Keyframe animation: • Build one object for every timestep. Then, per frame:rtDeleteInstance(instID); rtLoadMatrixf(&currentTransform); instID = rtInstantiate(objID[currentTimeStep]);

39. OpenRT Core Application API:Comparison to OpenGL Biggest difference to OpenGL: Semantics • Everything is explicit! • E.g., rtPerspective/rtLookAt DIRECTLY set the camera, • It does NOT modify the transformation stack • Frame semantics vs. immediate-mode • Any “state change” effective ONLY at end of frame • I.e., changing a shader applies ALSO to already specified triangles • Lots of “side effects” • Intentional, but sometimes confusing to (ex-)OpenGL’ers

40. OpenRT Core Application API:Comparison to OpenGL • Everything is world space • NOT geometry transformed to camera space • RATHER camera explicitly defined in world space • Everything is “retained mode” • Until explicitly deleted, objects/instances/shaders/etc stay active in successive frames • NO re-instantiation per frame necessary (unlike disp.lists) • Framebuffer operations don’t make sense • Stipples, masks, blending, … not supported at all. • Rather: Use shaders

41. Using Realtime Ray Tracing:Programming with OpenRT • A brief tutorial • Initializing • Defining geometry • Loading and using shaders • Rendering a frame • Animating the scene • Performance optimizations • Tips & Tricks

42. Using Realtime Ray Tracing:Programming with OpenRT • A brief tutorial  extended slides (www.openrt.de) • Initializing • Defining geometry • Loading and using shaders • Rendering a frame • Animating the scene • Performance optimizations • Tips & Tricks  extended slides

43. Performance optimizations Key to good performance: • Minimize state changes • Reason: All state changes have to be transferred across network to clients  VERY costly • Minimize number of objects/instances • Core performance best for few, large objects • Good kd-trees can’t help if geometry is not in same object! • Use lightweight applications • Don’t spend 90% of time in GUI and scene graph trav.!

44. Performance optimizations Minimizing state changes: • Don’t delete/re-instantiate all instances per frame • Rather: selectively update only changed instances • Don’t re-issue data that has not changed • Shader parameters, lights’ positions, … • Often need to track current state in application • Remember: If not explcitly changed, everything is still available in next frame • Shaders, objects, shader parameters, …

45. Performance optimizations Minimize number of objects • Typical scene graph problem: Lots of small nodes • Don’t generate a new object for every 10 triangles ! • Generally: Put all primitives in same object, … • … except if they are to to be moved differently • Need to combine small sub-graphs to one object • Often coding effort for existing scene graphs • … but VERY important

46. Tips & Tricks Some typical tips&tricks when using OpenRT: • Progressive Supersampling • Using (pre-computed) binary scene dumps • Subsampling during motion

47. Tips & Tricks Some typical tips&tricks when using OpenRT: • Progressive Supersampling • OpenRT also supports “accumulation buffer” concept • During motion: use plain “rtSwapBuffers()” • At camera standstill: Use “rtAccum(…)” to accumulate images computed with different pixel samples • Result: Fast rendering during interaction • But: Aliasing quickly disappears after end of interaction • Can do similar for smooth shadows, glossyness, …

48. Tips & Tricks • Using binary object dumps • OpenRT supports “binary dumps” of complete objects • Including kd-trees and geometry • Next time app starts, just load/map the binary objects • Much faster startup times, no need to build kd-trees • In particular, can “mmap” the binaries • No need to even load the data into application • Can copy binaries to client’s local disks, map from there • No need to send full geometry every frame!

49. Tips & Tricks • Subsampling during motion • If your machine/cluster is too slow: • Let OpenRT render at reduced resolution during motion • Use OpenGL to “zoom” image to fullscreen • blurry, but faster • During camera standstill, switch to fullscreen ray tracing • Similar to progressive supersampling

50. OpenRT API: Lessons learned • Powerful: Lots of existing applications • Including industrial VR software • All realized exclusively via the API • Distribution framework completely hidden • Application often doesn’t know at all if it’s running standalone or distributed • Programmer doesn’t have to care • Except: performance (see below) • Except: file names… (might be different on clients!)