Automatic Generation of Dynamics Models

1. Automatic Generation of Dynamics Models John W. Ratcliff Most Worshipful Senior Programmer Simutronics Corporation

2. Introduction Work for this presentation was supported by Ageia Technologies. This is an open source project to facilitate the rapid production of physics content from raw source graphics data Much assistance provided by Matthias Mueller, Pierre Terdiman, Adam Moravansky, Billy Zelsnack and Dilip Sequeira

3. The Code Suppositorywww.codesuppository.blogspot.com “A Place Where I Insert My Code into the Anus of the Internet” John W. Ratcliff, Flipcode, January 12, 2000

4. The Problem The rapid acceptance of pervasive physics in games has created new authoring and asset requirements. There is a general lack of tools to facilitate the rapid creation of dynamics assets. Many games send their raw graphics assets to the physics engine which is highly inefficient and often causes significant problems.

5. Where should Physics Content be Authored? Physics models are usually derived from a final production art asset. Iteration on physics assets is primarily a ‘design’ problem rather than an art issue. Ideally editing of dynamics data assets should occur directly in the game engine/editor for rapid iteration cycles to test the behavior and interaction of the models in real-time.

6. Existing Tools to Author Physics Content Feeling Software : www.feelingsoftware.com PhysX Plugin for 3D Studio Max (free open source) Nima for Maya : (Unsupported version Free) Lead developer Christian Laforte Blender 3D : www.blender.org (Free open source) Lead developer Ton Roosendaal Bullet Physics Library : www.continuousphysics.com (Free open source) Developed by Erwin Coumans CreateDynamics : www.codesuppository.blogspot.com (Free open source) Developed by John W. Ratcliff Unreal Editor : www.epicgames.com Proprietary tools. Physics authoring happens in the game editor Uses some of the CreateDynamics toolkit for generating compound objects. Lead developer on the physics tools James Golding Scythe Physics Editor : www.physicseditor.com Supports ODE, Newton, PhysX Free and Commercial vesions available Lead developer Chris Hunt

7. Existing Standardized File Formats for Physics Content COLLADA 1.4.1 : www.collada.org Supports basic rigid body physics markup in an XML format. NxuStream : www.ageia.com Provided as part of the free PhysX SDK. Source library. Exact reflective object model of every component in the SDK, including rigid body, constraints, all shapes, including heightfields, cloth, softbodies, fluids, collision filters, energy fields, and much more. XML, Binary, and supports COLLADA 1.4.1 physics only specification. Scythe : www.physicseditor.com

8. Multiple Collision Models Per Asset For any given art asset there are usually at least three, if not more, physics representations. Raycast model: A static triangle mesh that is identical to the original graphics model for precise line of sight evaluation as well as certain graphics support such as decal generation. Static collision model: An often dramatically simplified version which corresponds to what the player character would collide with (rarely should be the raycast version.) May be a simplified mesh, a convex hull, or a compound shape. Dynamic collision model(s): One or more versions to be used when the object is dynamic. Should never be a triangle mesh. Typically a single convex hull or compound object.

9. Components of a Physics Asset Rigid Body properties Collision Shapes Triangle Mesh Box Capsule Convex Hull Sphere HeightField Constraints Cloth properties and topology Soft Body properties and topology Fluid and fluid emitter properties Energy Fields

10. Typical Rigid Body Properties World Transform (global pose) List of collision shapes Collision group and/or other flags Dynamics Properties Linear Velocity and Angular Velocity Mass or Density Mass Local Pose and Mass Space Inertia Tensor Engine specific components Solver Iterations Sleep Threshold Maximum Angular Velocity Etc.

11. Collision Shape Properties Local Transform (relative to parent rigid body) Material Index Collision filtering information Contact report flags Shape data: Box dimensions Sphere radius Capsule height and radius Convex hull faces Triangle mesh triangles Heightfield sample data Plane equation

12. Programmable ConstraintCustomizable on all 6 degrees of freedom angular and linearPhysX SDK implementation by Matthias Mueller-Fischer A constraint that can be configured on three angular and three linear degrees of freedom. Each degree of freedom can be fixed, free, or limited and can be configured to support drive, motors, and springs. Advantages A single model can be configured to behave in any of the most common custom joint configurations. Allows for a much simpler and easier to maintain code path. Provides new types of constraints not typically available with most engines. Predictable behavior in all configurations. Orthogonal data definition. Disadvantages Initially can be more difficult to configure. This can be improved by providing helper setup routines for common constraint types.

13. Authoring Collision Shapes For simple objects, when a single shape is sufficient, collision fitting is not difficult

14. Compound ShapesApproximate Convex Decomposition Published work Approximate Convex Decomposition Texas A&M University Algorithms & Applications Group Jyh-Ming Lien and Nancy M. Amato (parasol.tamu.edu/groups/amatogroup/research/app-cd) Variational, meaningful shape decomposition University of British Columbia Vladislav Krayevoy and Alla Scheffer (www.cs.ubc.ca/~vlady/Papers/Segmentation.pdf)

15. Convex DecompositionAlgorithm by John W. Ratcliff ‘Inspired’ by the work of Jyh-Ming Lien and Nancy M. Amato at Texas A&M. However, this brute force implementation has little in common with their technique. Accepts open meshes Not real time Available as a plug-in or open source library Simple interface

16. The Algorithm A recursive routine is passed, as input, the source triangle mesh The first step is to compute the ‘volume of concavity’ Build a convex hull around the mesh. Project each triangle on the original mesh onto the hull. Compute the volume of the mesh that results from the projected triangle onto the convex hull skin. The sum of the volume of all projected triangles to the convex hull surrounding it becomes the ‘volume of concavity’ Compute the percentage volume of concavity by dividing it by the volume of the hull around the original source mesh. If the percentage volume of concavity is below a user supplied threshold then accept this mesh as sufficiently convex. By measuring against the total volume this allows highly concave pieces to be ignored if they are quite small and unimportant relative to the overall size of the object.

17. The Algorithm cont. Compute the best fit oriented bounding box If the mesh is concave then compute the best fit oriented bounding box around the mesh. Compute a split plane along the long axis of the OBB Split all of the input triangles against the plane producing two output meshes Each triangle is tested to see which side of the plane it lies on. If the triangle straddles the plane it is split and one triangle piece is sent to the ‘top’ mesh and the other sent to the ‘bottom’ mesh. The ‘edge’ of each split triangle is added to an edge list.

18. The Algorithm cont. Using the edges that lie along the split plane compute a delaunay triangulation. Project the points onto the plane so the operation can be performed in 2d Must support multiple polygons Must support ‘holes’ by testing polygons inside other polygons (example splitting a glass in half) See: “Triangle : A Two-Dimensional Quality Mesh Generator and Delaunay Triangulator “ by Jonathan Richard Shewchuk http://www.cs.cmu.edu/~quake/triangle.html Project the polygon points back into 3d and weld them into the mesh producing a whole piece. Without this operation deep recursion produces hollow objects and does not converge to an optimal solution set.

19. The Algorithm cont. Pass the top and bottom split meshes back into the recursive routine. Wash, rinse, and repeat until there are no concave pieces left or at the maximum recursion depth provided by the user. This results in an oriented bounding volume tree based on the original source mesh. The output hulls are over described since an optimal split was not performed. To correct for this a cleanup phase is run where hulls are merged back together again if they are largely convex afterwards. In short, what the process has done is created too many hulls but a straightforward cleanup pass can easily stitch them back together again. Simply compute the volume of two hulls separate then compute the volume of the two hulls combined. If the volume is within a percentage threshold provided by the user (called the merge threshold) then these two hulls are combined back into one.

20. The Algorithm cont. Attempt to merge hulls by largest volume first. At each phase of the merge process select the largest hull first and then attempt to merge it in order of the largest volume to the smallest. The end result is that even though the mesh was chopped up more than was necessary, the post-process cleanup phase ends up producing results as if the most optimal split had been performed in the first place. Behold the power of brute force, sometimes a blunt instrument beats out the most advanced mathematics.

21. Create DynamicsConvex Decomposition User Interface

22. Approximate Convex DecompositionExamples using various fitting rules

23. Approximate Convex Decomposition Examples

24. VideoA high poly count helix mesh simulated as a rigid body composed of only 15 convex hull shapes

25. Automatic Rag Doll GenerationDirectly from Skeletal Deformed Mesh Creating Ragdoll models can be very time consuming. Artists spend a great deal of time rigging up a skeletal system for an art asset to begin with. There should not be a need to spend an equal amount of time producing a physics representation of the same. Embedded within a standard skeletal deformed mesh is sufficient information to auto-generate a reasonable approximate ragdoll model.

26. Automatic Rag Doll GenerationStep 1 : Skeleton Pruning The skeleton used for a ragdoll physics simulation is typically far simpler than what is used for graphics. The graphics version may include many details such as fingers, toes, facial bones and marker bones for game play elements. None of these bones are desirable in the physics representation Before attempting to generate constraints for all of the bones in the skeleton the skeleton needs to be pruned and a new skeleton created which is usable for a real-time physics simulation.

27. Automatic Rag Doll GenerationStep 1 : Skeleton Pruning Traverse the skeleton and examine the triangles which have significant weightings to each bone. Bones which are not influenced by any geometry are marked for deletion. For the rest of the bones compute the volume of the convex hull around the triangles weighted to it. Compare the volume of the geometry associated with the volume of the whole. If the volume is below a user supplied percentage threshold this bone should be marked for removal. This will leave only bones associated with larger components of the source mesh; in a humanoid character this would correspond to head, arms, legs, and torso, but not eyes, fingers, or toes.

28. Automatic Rag Doll GenerationStep 1 : Skeleton Pruning Now a new skeleton must be produced that leaves only the bones which were marked for removal. Leaf node bones marked for deletion may simply be removed. Bones marked for removal that are not leaf nodes must have their child/parent relationship patched up so the skeleton will still be intact. The parent should keep track of the now dead child bones it has inherited. Now that a reduced skeleton has been created the deformed mesh geometry has to be revised so the bone indices point to the correct locations in the new skeleton. For each bone referenced in the source mesh re-assign it to the corresponding bone in the new skeleton for the output mesh. If a vertex refers to a bone that has been deleted then attach it to the parent bone that the original bone was collapsed into.

29. Automatic Rag Doll GenerationStep 2 : Generating Collision Shapes Walk the reduced skeleton and collect the triangles associated with each bone. On a per-bone basis run the CreateDynamics shape fitting tool and approximate the shape as a box, sphere, capsule, or convex hull based on user input selection. Using a rules based system, different shape fitting techniques can be applied to the skeleton using wildcards and inheritance.

30. Automatic Rag Doll GenerationStep 3 : Generating Constraints Constraints are created based on the transforms of the reduced skeleton. Joint types and limits are selected from user input using the same rules system for deciding shape fitting. Though not currently implemented it would be feasible to infer joint types and limits by interpreting a reference animation. Joint limits and types are supported by Maya and might be able to be passed as part of the export process.

31. Automatic Rag Doll GenerationStep 4 : Generating Collision Filters To prevent the ragdoll model from constantly colliding with itself collision filters are needed to prevent this artifact. Walk the skeleton and compute the skeletal distance between bones. Bones which are near each other along the skeleton should have collisions disabled between their bodies. For more accurate collision modeling (such as convex hulls) the filtering need not be too deep. For cruder collision models such as capsules or boxes, filtering to avoid the model colliding with itself will need to be more aggressive. Though not currently implemented, it might be best to decide where collision filters are needed by detecting contacts while the skeletal model is in its rest pose.

32. Simplified Auto Generated RagDoll

33. Detailed Auto Generated Ragdoll

34. Video : Auto Generated RagDoll Models Simulated with the PhysX SDK

35. Auto Generated Constraints The previous example showed how to create a ragdoll from a skeletal deformed mesh rigged up by an artist. If you don’t have an already rigged up skeletal deformed mesh but still want to get a fast ragdoll or cheap illusion of soft body then a skeletal deformed mesh can be automatically generated.

36. Auto Generated Constraints First generate an oriented bounding volume system for the raw source mesh. This is easy to implement because all we have to do is disable the merge portion of the convex decomposition algorithm. By removing the merge option the system is much more balanced and will simulate more realistically.

37. Auto Generated Constraints

38. Auto Generated Constraints Auto-generating the constraints uses a recursive algorithm that begins with finding the largest volume hull in the decomposed object. The largest hull is used as the root node for the skeleton. From this hull locate all hulls which ‘touch it’. Touching is determined by finding two triangles which have roughly the same vector normal pointing in opposite directions. If the two triangles lie along the same plane, then they are projected into 2d along that plane and a 2d triangle-triangle intersection test is performed. If these two triangles intersect then they are considered ‘touching’ and are added to an intersection bounding volume. If the two hulls were found to be touching then the median intersection point of those touching surfaces becomes the anchor point for the constraint. This same process is repeated for every hull in the object, each time starting with the last hulls that were connected. In the end each hull will have a single constraint at the location where it touches a neighbor hull.

39. Auto Generated Skinned Meshes Once the auto-generated constrained rigid body system has been saved there are now run-time considerations. A skeletal deformed mesh has to be created from the original raw source mesh. This could be done as a pre-process step but is more powerful to do on the fly at run time. Converting the raw mesh to a skeletal deformed mesh requires that each vertex be assigned bone indices and bone weightings. For each vertex in the source mesh, compute the four nearest bones (rigid bodies) and the distance of those bones from the vertex. Compute the weighting of each bone based on the ratio of the distance of that bone relative to the sum total of the distances of all four. An additional weighting bias can be applied if desired (Example: 4x bone1, 2x bone2, 1x, bone 3 and 4.)

40. Video : Auto-Generated constrained system mapped to an auto-generated skinned mesh

41. Auto-Generated Pre-Fractured Objects The same technique used to produce skeletal meshes from raw mesh data can be applied for pre-fractured objects Do not disable the merge in this case. Constraints are generated against fewer shapes at more natural break boundaries. The constraints are rigged up to be fixed joints with a particular breaking force based on game design. The difficult part is producing fracture artwork. This is a ‘hard’ problem, but not unsolvable. Currently the CreateDynamics toolkit cannot produce pre-fracture graphics, only pre-fracture physics.

42. Video

43. Cloth Authoring Cloth is simulated based on a source triangle mesh. Authoring is straightforward as the physics representation is simply the source triangle mesh with duplicate vertex positions removed. Additional data associated with cloth are numerous physics properties describing how the simulation should behave. Attachment points must be captured where needed.

44. Cloth at Run-Time The graphics representation of the cloth usually does not exactly match up to the physics representation due to material assignments. A mapping table must exist between the indices in the physics mesh to the indices in the graphics mesh. Additional challenges arise from dealing with torn cloth which creates new triangle indices, and those indices must be remapped to the graphics representation on the fly. Double sided cloth can be visualized using a shader that renders the cloth twice; flipping the normals and culling direction on the back side. Cloth is typically authored in a ‘rest pose’ so a simulation may need to be run a number of frames before rendering it initially to avoid watching the cloth ‘fall into place’.

45. Cloth Simulation Video

46. Authoring Fluids Fluids are represented as a set of simulation properties and emitters. 3D fluids have numerous and often complex physical properties that influence the behavior of the simulation. Emitters can be attached to rigid body actors. Numerous emitter properties can be edited as well. A real-time preview tool to rapidly iterate on fluid properties is essential to achieve good results.

47. Rendering Fluids The challenges in turning a set of 3d points into a compelling visual illusion of fluid surfaces would take a whole talk by itself. Think outside the box. Just because you are using a physics simulation called ‘fluids’ doesn’t mean you have to visualize it as a fluid surface. Smoke Sparks Micro-Debris Fog Energy fields And whatever your artists imagination can come up with.

48. Common Fluid Visualization Methods Sprites Each fluid particle is rendered using a camera facing billboard sprite. Excellent opportunity to use hardware point sprites. Multi-Pass techniques on each sprite can produce interesting layered effects. Full Screen multipass effects can produce the illusion of refraction and reflection by rendering sprites as normals which are then Gaussian blurred in an off-screen buffer. Surface Mesh Algorithm by Matthias Mueller Performs mesh operations in 2d and then back-projects the results producing a highly efficient 3d mesh.

49. Video : Screen Space Surfaces

50. Video : Comparison Fluid Visualization Techniques

51. Video

52. Soft Body Authoring Soft Bodies are simulated as a tetrahedral volumetric mesh. Similar to the cloth simulation presented in the paper ‘Position Based Dynamics’ authors Matthias Müller, Bruno Heidelberger, Marcus Hennix, and John Ratcliff. (http://graphics.ethz.ch/~mattmuel/publications/posBasedDyn.pdf) Rather than solving for stretching along vertex edges instead solve for conservation of volume. Highly dependent upon the quality of the input tetrahedral mesh.

53. Soft Body AuthoringMathias Mueller-Fischer, PhDhttp://graphics.ethz.ch/~mattmuel/Ageia Technologies Begin with arbitrary 3d mesh Does not need to be a closed mesh. Generate an isosurface for the input mesh. Generate an iso-surface of the distance field to the triangle mesh and triangulate it via marching cubes. Perform a quadric mesh optimization on the isosurface This is a mesh simplification technique which does not affect the morphology of large scale features. This will produce fewer tetrahdrons without affecting the visual quality of the simulation and, therefore, simulate faster. Using the optimized isosurface mesh perform the delaunay tetrahedralization to produce the set of tetrahedrons for simulation. Adding randomly distributed vertices inside the isosurface will reduce the tetrahedral size. Finally allow the user the option of editing individual tetrahedron based on game design needs.

54. Soft Body Visualization Use free form deformation to produce the real-time graphics mesh synchronized to the tetrahedral physics simulation. For each vertex in the graphics mesh compute the nearest tetrahedron to it. Compute the set of barycentric coordinates that map the four vertices of the tetrahedron to the single vertex on the graphics mesh. These barycentric coordinates can be pre-computed and stored as part of the graphics mesh data, or can be built on the fly when the mesh is initially loaded. During rendering, for each vertex in the graphics mesh deform it by the projection of the tetrahedron that maps to that vertex using the current positions and the previously computed barycentric values. This deformation is difficult to perfom in a vertex shader unless, perhaps under DirectX 10. Even then, the operation is so fast in software that it difficult to imagine significant performance gains.

55. Video

56. Video

57. The Production Pipeline For each source asset in the product generate a default INI file. The default file should correspond to the minimum expected set of physics models required for any asset. Typically this would include a raycast version, a static collision version, and a dynamic collision version. Provide a user interface within the game editor that allows designers to modify all of the default settings and add additional physical representations. Save the INI file as a permanent asset that corresponds to the source graphics model. The physics assets only need to be regenerated if the morphology of the graphics asset changes or the user makes explicit editing changes.

58. The Production Pipeline During game editing or prior to producing final production assets. Auto-generate the various physics versions for each asset by using the CreateDynamics library. This will produce an ASCII version of the physics representation as either COLLADA or NxuStream. Since the COLLADA specification cannot represent many of the types of physics presented here; cloth, soft body, fluids, heightfields, it is recommended to use NxuStream. If you do not use the PhysX SDK or COLLADA it is easy to modify the source code in CreateDynamics to output the physics data in a format that is of your choosing. (Accumulate.cpp) Store the various ASCII versions of the physics asset in your repository for day to day use.

59. The Production Pipeline At run-time, and for final production content, convert the XML version of the art assets into a binary cooked form. The XML version can be converted into a binary representation very easily and quickly. The binary versions are not backwards compatible and should never be used as a persistent storage format for the physics data. It is best to create the binary versions on the fly on the users machine and store it in a local repository. During level load time, the pre-cooked binary assets are in a format that can be rapidly submitted to the physics engine. Not only is parsing of XML and other ASCII data avoided, but all mesh preprocessing is bypassed as well. Extremely fast level load times can be achieved using this technique. Massive game levels can be loaded and submitted to a physics engine in under a second.

60. The Future of CreateDynamics This is a ‘hobby’ project and while it is being used in a production environment no guarantees can be made as to a schedule of features, bug fixes, or support. Open to discussion for collaboration or sponsorship of the future development of these tools. Special thanks to: Matthias Muller, Pierre Terdiman, Adam Moravansky, Billy Zelsnack, Dilip Sequeira, James Dolan, James Golding, David Whatley, Simon Schirm, Erwin Coumans, Christian Laforte, Stan Melax, Worshipful Brother Lou Castle, Jesus Christ, Mahatma Ghandi, etc.

61. Questions??? Contact: John W. Ratcliff Email: jratcliff@infiniplex.net Coding website: www.codesuppository.com Skype: jratcliff63367 Skype Phone: US (636)-486-4040 Karma Contribution Site: www.amillionpixels.us