Animating Real-Time Crowds

1. Animating Real-Time Crowds

2. Outline • Introduction • Description of Level of Detail • Level of Detail in… • (agent behavior) • …sensing • …behavioral architectures • (agent motion) • …motion simulation • …animation cache • …skeletal pruning • (rendering) • …geometry • …replication • References

3. Introduction • An agent-based simulation models the movements of independent entities in an environment. • The agent is responsible for controlling a single entity in the simulation. • The agent is responsible for setting goals, actions to achieve those goals, and react to external environmental stimuli. • This method has been used to model flocks of birds, schools of fish, herds of animals, and crowds of people. • This presentation will focus on modeling humanoid crowds in real-time.

4. Introduction • The primary difficulty in a real-time agent-based simulation is scalability. • Agent-agent interactions can be O(n2). • Sensing: Who is near me? • Behaviors / Response to Stimuli: What should I do? • Very application specific. • Animating an articulated human model is expensive. • Both areas will be addressed in this presentation.

5. Level of Detail • Level of Detail (LOD) is a common technique used to approximate the solution to a complex system. • LOD is commonly driven by an observer’s location. • Elements near the observer are simulated with a high degree of fidelity, while farther elements are simulated with any number of LODs. • LOD may be applied to sensing, behavior simulation, motion of articulated models, and model geometry.

6. LOD and Agent Behavior

7. Sensing • Agents use sensing to gather information about their world/situation. • Provides input to a behavior model which in-turn outputs actions/responses. • Unlike in the real world, agents can be omniscient. • Agents may know the locations of all others. • Agents may know the map of the entire environment. • Agents may communicate with all others, regardless of location. • The problem of sensing is in filtering all this data.

8. Sensing (2) • Sensors are used by agents to filter the world data. • Ideally, an agent should have visual, auditory, and tactile senses. • No need for taste and smell– at least just yet. • Vision is only commonly available.

9. Sensing (3) • Vision can be implemented in several ways: • Rendering-Based: Images are rendered from the agent’s point of view in false color. • Object identification. • Z-buffer for collision avoidance. • Not scalable to many agents in real-time. • Database-Access: Local neighborhood is queried for information. • Most common. • These methods are generally fast and can be tuned for level of detail. • Example: • Kd-tree database to find nearby agents and obstacles. • LOD: Limit search radius. Cap query results. • Search a patch of a raster grid to find nearby agents and obstacles. • LOD: Raster grid cell size. Size of patch. Cap query results. • Neither allow for long-range vision.

10. Behavioral Architectures • Real-time crowds nearly always require an interactive element (otherwise there would be no real-time requirement). • Support for interactivity and its impact on agent behavior (user  agent and agent  agent) drives much of the system’s software architecture.

11. Behavioral Architectures (2) • The typical agent cycle looks like [Niederberger]: • Sense • Think • Act • The first two steps can be difficult to perform efficiently if each agent interrogates its environment individually. • Shared sensing, thinking, and acting can be used to reduce computational cost if the agent is a part of a group. • A group generalization may be thought as a LOD.

12. Behavioral Architectures (3) • Shared Knowledge Base [Niederberger]: • All world data is stored in a common knowledge base (KB) shared by all agents. • Agent Positional Data • Sensors • Situations • Agent Groups • Topics within the KB are keyed with various topic IDs. • Each agent has KB keys for accessing data. • Agent Keys, Group Keys, Sensor Keys

13. Behavioral Architectures (4) • Advantages: • Agents can quickly access group awareness. • Sensors may be shared among agents. • Very easy to multithread the “think” phase since it is “read-only” on the KB.

14. Behavioral Architectures (5) • Group behaviors are realized through hierarchical relations between agents. • A typical hierarchy is a leader/follower model. • Heavy “thinking” (such as path planning) is only performed by the leader. • Followers only need to keep up and follow signals raised by their leader.

15. Behavioral Architectures (6) • Accelerating the Think phase: • Given a hard time limit (say 10ms) to spend on all agent processing, the simulation loop only Thinks for a subset of agents (scheduled using a Round Robin (2003) or Priority Queue (2005)). • LOD may be used to allocate time slice (2005). • Agents may give up slice to group leader if slice is too small to be useful. • Skipped agents continue to perform their previous actions.

16. Behavioral Architectures (7) LOD scheduling priorities are allocated by visibility and distance.

17. Behavioral Architectures (8) Images show agent autonomy. Agents in the field of view and near the camera act independently. Agents off camera are controlled by group leaders.

18. Behavioral Architectures (9) • An aside about Thinking: • Thinking makes up the agent’s behavior model and can be very simple or very complex. • Simple: Finite State Machine • Moderately Complex: Fuzzy Logic • Complex: Neural Network, Bayesian Network • It all depends upon the application and behaviors trying to be achieved. • Real-Time crowd sim will typically use a FSM. • Niederberger explores iterative algorithms for time-allocated thinking. Allows for a useful partial solution to be computed in allotted time that may be improved upon in the next time slice.

19. Behavioral Architectures (10) • Accelerating the Act phase: • A reduced set of actions can also be executed given time constraints. • Actions are prioritized in a queue. • Simulation tries to execute all queued actions, but may have to exit early if the time limit is exceeded. • Reactive actions (such as flee a predator) are given priority over less important actions such as follow the leader.

20. Behavioral Architectures (11) • Summary of performance advantages of Niederberger’s architecture: • Shared knowledge base. • Data quickly available. • Easy to multithread. • Hierarchical group organization allows “heavy” processing to be done by group leaders. • LOD-driven scheduling of behavioral updates. • Prioritization of actions.

21. Behavioral Architectures (12) • Hierarchical group structures are a common way to reduce the “heavy” computational costs of the crowd simulation. • Musse and Thalmann’s ViCrowds [Musse]: • Inherited state and behaviors (instead of shared KB). • Group splitting/merging. • Group dominated by single leader or more complex sociological models.

22. LOD and Agent Motion

23. Motion Simulation • Thus far in this class we have already discussed several methods for animating articulated models. • Motion Controllers • Inverse Kinematics • Motion Capture / Animator Key-Frames • Many of these are very computationally expensive.

24. Motion Simulation (2) • Using LOD, it is conceivable to create a crowd simulation where agents near the observer are animated with high quality techniques while farther agents use more simplified methods. • Switching between the appropriate LODs can be challenging, but may be addressed with the appropriate architecture.

25. Motion Simulation (3) • Giang et al. devise a architecture where an “LOD Resolver” sends LOD requests to a “Motion Controller”. • The Motion Controller invokes the appropriate animation method.

26. Motion Simulation (4) • The Motion Controller contains different sub-modules used for the various animation techniques. • The “Animation Cache” stores current “in-flight” animations.

27. Animation Cache (Not Giang’s animation cache) • Articulated models are typically have a skeletal structure (made up of bones) covered with a skin (made up of mesh vertices). • Animations are generated by moving the model’s bones. • A vertex is transformed with respect to the bone configuration by the following equation: Xti is the global transform of bone i at time t. X-refi is the inverse global transform of the bone in the reference position. vref is the vertex in the reference position. • Vertex positions are computed though the application of hierarchical transformations starting from the root bone.

28. Animation Cache (2) • Agents far from the observer may be animated at a low frequency (much lower than rendered frame rate). • Example: Thalmann’s Virtual Heritage used 4Hz for distant agents and 50Hz for nearby ones. • A single pose may be reused between several frames. • The animation cache is this reused pose. • Generating the cache: • Interpolate between two key frames. • Apply vertex skinning transformations. • Store result as the cache.

29. Skeletal Pruning • There is another LOD technique that uses the agent’s skeleton. • The hierarchical bone transformations can be costly for highly articulated models. • A cost savings would be realized if the number of bones could be reduced.

30. Skeletal Pruning (2) • By “fusing” bones together, a sub-hierarchy of the skeletal structure can be used to transform all vertices within the sub-hierarchy. • Fewer rotation interpolations. • Fewer matrix multiplications. • Costs: • On a new level-of-detail, bone assignments of vertices must be recomputed with respect to the fused bone.

31. LOD in Rendering

32. LOD in Rendering • While not explicitly related to simulation, fast rendering allows for more time to be spent performing simulation. • There are several techniques that are commonly used to speed up crowd rendering.

33. Geometric LOD • Geometric & Animation LOD • Reduce the number of polygons in the model at different LOD steps. • At a certain LOD, stop animating bones of figure and use static-keyframe meshes instead. • May continue to reduce polygons in the model. • At the farthest LOD, stop using static meshes and use 2D billboards. • May be a simple sprite animation. • Or more complex “polyposter” [Kavan] which is a collection of 2D deformable textured polygons.

34. Geometric LOD (2)

35. Geometric LOD (3) A 2D billboard from different views can be generated dynamically. Billboards can be used to dramatically increase crowd size.

36. Geometric LOD (4) Polyposter approximation of a full mesh animation.

37. Replication • Replication • Transformed agent meshes from an animation can be shared between agents. • For example, an two agents may share the same model at the same animation frame. • The model is replicated for both agents. • Model texturing may be altered for each instance. (Example: Different color of clothes.) • Commonly used for spectator crowds.

38. References • Aubel, A., Boulic, R., Thalmann, D.: Real-time display of virtual humans: Levels of detail and imposters. IEEE Transactions on Circuits and Systems for Video Technology 10(2), 207-217 (2000) • Azahar, M. A., Sunar, M. S., Daman, D., and Bade, A. 2008. Survey on Real-Time Crowds Simulation. In Proceedings of the 3rd international Conference on Technologies For E-Learning and Digital Entertainment (Nanjing, China, June 25 - 27, 2008). Z. Pan, X. Zhang, A. Rhalibi, W. Woo, and Y. Li, Eds. Lecture Notes In Computer Science, vol. 5093. Springer-Verlag, Berlin, Heidelberg, 573-580. • de Heras, P., Schertenleib, S., Maim, J., Thalmann, D.: Realtime shader rendering for crowds in virtual heritage. In: Proc. 6th International Symposium on Virtual Reality, Archaeology and Cultural Heritage (VAST 2005) (2005) • Giang, T., Mooney, R., Peters, C. O’Sullivan, C.: ALOHA: Adaptive Level Of Detail for Human Animation: Towards a new framework. Eurographics 2000 short paper proceedings. pp. 71-77 (2000) • Hery, C., Lippman, S., Ono, H., Regelous, S., Sutton, S., Thalmann, D.: Crowd and Group Animation, SIGGRAPH Course (2004) • Kavan, L., Dobbyn, S., Collins, S., Žára, J., and O'Sullivan, C. 2008. Polypostors: 2D polygonal impostors for 3D crowds. In Proceedings of the 2008 Symposium on interactive 3D Graphics and Games (Redwood City, California, February 15 - 17, 2008). I3D '08. ACM, New York, NY, 149-155. • Musse, S.R., Thalman, D.: A hierarchical model for real time simulation of virtual human crowds. IEEE Transactions on Visualization and Computer Graphics 7(2), 152-164 (2001) • Niederberger, C., Gross, M.: Hierarchical and heterogeneous reactive agents for real-time applications. Computer Graphics Forum 22(3) (Proc. Eurographics 2003) (2003) • Niederberger, C. and Gross, M. 2005. Level-of-detail for cognitive real-time characters. Vis. Comput. 21, 3 (Apr. 2005), 188-202. • Ulicnyn, B. Thalmann, D.: Towards interactive real-time crowd behavior simulation. Computer Graphics Forum 21(4), 767-775 (2002) • Ulicny B., de Heras Ciechomski P., Musse S. R., & Thalmann D.: State-of-the-Art: Real-time Crowd Simulation. In EuroGraphics 2006 Course on Populating Virtual Environments with Crowds. • Unattributed slide from SIGGRAPH Asia 2008 course.