Behavioral animation
1 / 85

- PowerPoint PPT Presentation

  • Updated On :

Behavioral Animation 數位內容學院 遊戲開發研究班第一期 3D 圖學 沈育德 Edward Shen May 21, 2005 Course Information Date : 5/19, 5/21, 5/26, 5/28 (2005) Lecturer : Edward Yu-Te Shen 沈育德 Course Website: Lecturer 沈育德 , Edward Shen

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about '' - jaden

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Behavioral animation l.jpg

Behavioral Animation




沈育德 Edward Shen

May 21, 2005

Course information l.jpg
Course Information

  • Date: 5/19, 5/21, 5/26, 5/28 (2005)

  • Lecturer: Edward Yu-Te Shen 沈育德

  • Course Website:

Lecturer l.jpg

  • 沈育德, Edward Shen

  • PhD Candidate (1st year)

  • Graphics group, Dept. of CSIE, National Taiwan University


  • [email protected]

Course overview l.jpg
Course Overview

  • This course course introduces several topics related to behavioral animation, including real-time crowd animation, collision detection and bounding boxes, self-generating behaviors through various techniques such as reinforcement learning, genetic algorithms, and so on.

  • I hope that the class helps the students to gain general knowledge in the related areas as well as to apply such techniques in computer games and animated films development. The four classes will address more about design methodologies than practical programming issues.

Schedule l.jpg

  • Artificial Lives: Fishes and Evolving Creatures (Saturday, May 21)

    • Introduction to Artificial Lives

    • Artificial Fishes and Reinforcement Learned Behavior

    • Evolving Creatures and Genetic Algorithm

  • Simulating Human Crowds (Thursday, May 26)

  • Collision Detection and Bounding Boxes (Saturday, May 28)

Artificial lives fishes and evolving creatures l.jpg
Artificial Lives: Fishes and Evolving Creatures

  • Introduction to Artificial Lives

  • Artificial Fishes and Reinforcement Learned Behavior

  • Evolving Creatures and Genetic Algorithm

Applications l.jpg


Slide8 l.jpg

Overview of Artificial Life /

Behavioral Animation

  • Artificial life modeling and the computer graphic modeling hierarchy

Demetri Terzopoulos, Artificial life for computer graphics, Communications of the ACM, v.42 n.8, p.32-42

Overview of artificial life behavioral animation l.jpg
Overview of Artificial Life / Behavioral Animation

  • Combine geometric models and kinematic models to simplify key-framing

  • Mid 1980s : Dynamic Simulation

    • particles

    • rigid bodies

    • deformable solids

    • fluids

Slide10 l.jpg

Overview of Artificial Life / Behavioral Animation

  • Biomechanical Modeling

    • Biological tissue

    • Internal muscles actuators

  • Behavior Modeling

    • Self-animating characters/ objects that react to environment stimuli

  • Cognitive Modeling

    • Artificial intelligence

    • Knowledge representation

    • Reasoning

    • Planning

Slide11 l.jpg

Examples of Behavioral Animation in the Movies

Demetri Terzopoulos, Artificial life for computer graphics, Communications of the ACM, v.42 n.8, p.32-42

Slide12 l.jpg

Examples of Behavioral Animation in the Video Games

  • Creatures (

  • The Sims(http://

  • Demosaurus Rex (an experimental interactive game environment)

  • etc

Slide13 l.jpg

Artificial Life in Computer Graphics

Artificial plants

Conference proceedings cover image, SIGGRAPH '94

[R. Mech, P. Prusinkiewicz, B. Wyvill, and A. Glassner]

Slide14 l.jpg

Artificial Evolution



  • Sims [Sims 91,94]

  • simulate evolution from genotypes to phenotypes

  • occasional mutations

Steering behaviors l.jpg
Steering Behaviors

Action Selection: strategy, goals, planning

  • Wandering

  • Seek / Flee

  • Pursuit / Evade

  • Path Following

  • Cohesion

  • Alignment

  • Separation

  • Leader Following

Steering: path determination

Locomotion: animation, articulation

Slide16 l.jpg

Artificial Animals

  • Terzopoulos [Tu and Terzopoulos 94]

  • Basic behaviors

  • Simulation of fish movements using a simple muscle system

Slide17 l.jpg

Artificial Humans


  • Thalmann [Thalmann 98]

  • Facial animation

  • Simulation of sensory perception

  • Interaction / communication between real humans and virtual ones

Slide18 l.jpg

Design Motion

Design Behavior

Behavioral Animation

  • Goal : Model animated characters / objects that interact autonomously within a virtual world

  • Behavioral Animation shifts the traditional role of the animator

  • We still probably want external control and direction on an autonomous agent

  • Behavioral animation can be used as a vehicle for setting up simulations of real world events

Slide19 l.jpg

Behavioral Animation v.s.


  • Key-framing

    • Animator : Specifies the first and last frames of the animation sequence

    • Computer : Generates the in-between frames

  • Behavioral Animation

    • Animator : Determines a set of rules for the behavior of a character

    • Computer : Animates the scene based on the rules specified

Slide20 l.jpg

Components of a Behavior Animation System

  • Framework of a generic Behavioral Animation system

[Millar et al. 99]

Slide22 l.jpg

Perception Techniques

  • The perception techniques are the means by which the animated character perceives its environment

  • Can be grouped roughly in 3 classifications :

    • Zonal approach

    • Sensory approach

    • Synthetic vision approach

Slide23 l.jpg

Zonal Approach

  • Objects in the zone are sensed by the character

  • Usually, perception zones are circular or spherical but it can be otherwise

Perception Zone

Slide24 l.jpg

Sensory Approach

  • There are areas where perception of an object is vague and where it is precise

  • There might be many perception regions depending on the senses involved (e.g. smell, touch, sight, etc.)

  • Each object in the environment has a transmissive area

  • Perception occurs when character’s sensory area intersects with the object’s transmissive area

Slide25 l.jpg

Behavioral Model

  • Decision making model of the system

  • Behavior can be

    • reactive

    • intelligent response

  • Form of response is various :

    • movement vector

    • change in internal attributes

    • etc.

  • Different approaches used to implement the behavioral techniques

    • Rules

    • Network

    • Artificial intelligence

    • Mathematical

Slide26 l.jpg

Rule-Based Approach

  • Involves giving each animated character a set of rules defining how they react to their environment

  • (+) Can produce reasonable behaviors in a dynamic environment

  • (+) Relatively easy to modify the rules to produce different behaviors

  • (-) Reduce the chance of modeling behavior which cannot be easily predicted

  • (-) Number of rules can grow very large in complex environments

  • (-) Specific to a particular environment

Slide27 l.jpg

Artificial Intelligence Approach

  • Artificial Intelligence Approach

  • Uses AI techniques such as reasoning engines and neural networks

  • (+) Provides the ability to generate behaviors which have not been explicitly programmed

  • (-) Requires a level of understanding by the animator of the techniques being used

Slide28 l.jpg

Mathematical Approach

  • The behavior of the character is defined in mathematical terms

  • Ex : Model of object’s nature & Model of observer’s liking

    • Nature = (N1, N2)

    • Liking = (2, -3.7)

  • Model of function “Attractiveness”

    • Attractiveness = N . L = 2N1 – 3.7N2

  • (+) Provides a means of specifying behavioral responses in a precise manner

  • (-) Not very intuitive for animators

  • Slide29 l.jpg

    Methods of Motion Generation

    • Traditional Principles (Keyframing)

    • Performance Capture (Motion Capture)

    • Modeling/Simulation (Physics, Behaviors)

    • Automatic Discovery (High-Level Control)

    Keyframing i l.jpg
    Keyframing (I)

    • Specify the key positions for the objects to be animated.

    • Interpolate to determines the position of in-between frames.

    Keyframing ii l.jpg
    Keyframing (II)

    • Advantages

      • Relatively easy to use

      • Providing low-level control

    • Problems

      • Tedious and slow

      • Requiring the animator to understand the intimate details about the animated objects and the creativity to express their behavior in key-frames

    Motion interpolation l.jpg
    Motion Interpolation

    • Interpolate using mathematical functions:

      • Linear

      • Hermite

      • Bezier

        … and many others (see Richard Parent’s online book)

    • Forward & inverse kinematics for articulation

    • Specifying & representing deformation

    Basic terminologies l.jpg
    Basic Terminologies

    • Kinematics: study of motion independent of underlying forces

    • Degrees of freedom (DoF): the number of independent position variables needed to specify motions

    • State Vector: vector space of all possible configurations of an articulated figure. In general, the dimensions of state vector is equal to the DoF of the articulated figure.

    Forward vs inverse kinematics l.jpg
    Forward vs. Inverse Kinematics

    • Forward kinematics: motion of all joints is explicitly specified

    • Inverse kinematics: given the position of the end effector, find the position and orientation of all joints in a hierarchy of linkages; also called “goal-directed motion”.

    Forward kinematics l.jpg
    Forward Kinematics

    • As DoF increases, there are more transformation to control and thus become more complicated to control the motion.

    • Motion capture can simplify the process for well-defined motions and pre-determined tasks.

    Inverse kinematics l.jpg
    Inverse Kinematics

    • As DoF increases, the solution to the problem may become undefined and the system is said to be redundant. By adding more constraints reduces the dimensions of the solution.

    • It’s simple to use, when it works. But, it gives less control.

    • Some common problems:

      • Existence of solutions

      • Multiple solutions

      • Methods used

    Motion capture i l.jpg
    Motion Capture (I)

    • Use special sensors (trackers) to record the motion of a performer

    • Recorded data is then used to generate motion for an animated character (figure)

    Motion capture ii l.jpg
    Motion Capture (II)

    • Advantages

      • Ease of generating realistic motions

    • Problems

      • Not easy to accurately measure motions

      • Difficult to “scale” or “adjust” the recorded motions to fit the size of the animated characters

      • Limited capturing technology & devices

        • Sensor noise due to magnetic/metal trackers

        • Restricted motion due to wires & cables

        • Limited working volume

    Slide39 l.jpg

    Basic Behavioral Component Loop

    [ after Hanna et al. 2001]

    • Execute all the mental triggers for each entity

    • Ex. : Increase hunger level

    • Execute all the rules for each entity

    • Enter associated goals in entity’s goal queue

    • Ex. : Go to; Wander; Feed; Mate

    • Select highest priority goal from queue and execute it

    • Delete all expired goals

    • Display state of the animation

    Artificial lives fishes and evolving creatures41 l.jpg
    Artificial Lives: Fishes and Evolving Creatures

    • Introduction to Artificial Lives

    • Artificial Fishes and Reinforcement Learned Behavior

      • Introduction

      • Physics-Based Fish Model & Locomotion

      • Sensory Perception

      • Behavioral Modeling & Animation

      • Intention Generator

      • Artificial Fish Types

    • Evolving Creatures and Genetic Algorithm

    Overview l.jpg

    Artificial Fish


    Intention Generator


    Behavior Routines









    Filtered sensory data

    Sensory data


    Control parameters



    Motor Controllers


    Underlying Physical Model

    Motor (Action)

    3D Virtual World

    Mechanics l.jpg

    • node i have mass mi, position xi(t) = [xi(t), yi(t), zi(t)], velocity vi(t) = dxi/dt, acceleration ai(t) = d2xi/dt2.

    • spring Sij connect node i to node j.

    • Denote springs constant as cij & natural, rest length as lij, eij(t) = ||rij|| -lij, where rij = xj(t) – xi(t)

    • The Lagrange equations of motion of the dynamics fish:

    i = 0, …, 22,

    where wi(t) = jNi

    Muscles hydrodynamics l.jpg
    Muscles & Hydrodynamics

    • The artificial fish moves as a real fish does, by contracting its muscles.

    • Characteristic of swinging tail: contracting one side, relaxing the other side.

    • When the fish’s tail swings, it sets in motion a volume of water. The inertia of the displaced water produces a reaction force normal to the fish’s body and proportional to the volume of water displaced per unit time, which propels the fish forward.

    • With tail swinging towards positive X axis, reaction force at point niacts along the inward normal. Component resists the lateral movement, while is forward thrust. Aggregate thrust propels fish towards positive Y axis.

    Motor controllers l.jpg
    Motor Controllers

    • A artificial fish has three MCs, swim-MC, left-turn-MC & right-turn-MC.

    • The MCs prescribe muscle contractions to the mechanical model. The swim-MC controls the swimming segment muscle, while the turning MCs control the turning segment muscles.

    • swim-MC(speed) {r1, s1, r2, s2}

    • Turn-MC(angle) {r1, s1, r2, s2}

    Pectoral fins l.jpg
    Pectoral Fins

    • The pectoral fins control pitching and yawing.

    • The pectoral fins work by applying reaction force to nodes in the midsection, i.e. nodes 1 ≦i≦ 12.

    • The pectoral fins are analogous to the wings of an airplane. Pitch, yaw, and roll control stems from changing their orientations relative to the body; i. e., the angle π/4 ≦ γ ≦ π.

    • A fin has an area A, surface n and the fish in velocity v relative to the water, the fin force

    Learning how to swim l.jpg
    Learning How to Swim

    • Eu: Guiding the muscle actuator control function u(t) by discouraging large, rapid flutuation of u

    • Ev: Guiding good trajectory of the artificial fish (final distance from a target location, closeness to a specified speed, etc)

    Sensory perception l.jpg
    Sensory Perception

    • The artificial fish has a vision sensor and a temperature sensor.

    • The temperature sensor samples the surrounding water temperature at the center of the artificial fish’s body.

    • The vision sensor has access to the geometry, material property, and illumination information. It can also interrogate the object database to identify nearby objects and interrogate the physical simulation to obtain information.

    Behavioral modeling animation l.jpg
    Behavioral Modeling & Animation

    • At each time step, the intention generator issues an intention based on the fish’s habits, mental state, and incoming sensory.

    • Habits: a set of parameters that determine fish habits.

    • Mental State function

      H(t) = min[1 –ne(t)R(∆tH)/, 1]

      L(t) = min[s(∆tL)(1-H(t)), 1]

      F(t) = min[Fi, 1], where Fi = min[D0/di(t), 1]

      H(t) : hunger function

      L(t) : libido function

      F(t) : fear function

      t: time

    Mental state function hunger l.jpg
    Mental State Function (Hunger)

    H(t) = min[1 –ne(t)R(∆tH)/, 1]

    • ne(t): amount of food consumed as measured by the number of food particles or prey fishes eaten.

    • R(x) : R(x) = 1-p0x with constant p0 is the digestion rate

    • ∆tH : the time since the last meal

    •  : a constant that dictates the appetite of the fish

    Mental state function libido l.jpg
    Mental State Function (Libido)

    L(t) = min[s(∆tL)(1-H(t)), 1]

    • s(x) : s(x) = p1x with constant p1 is the libido function

    • ∆tL : the time since the last mating

    Mental state function fear l.jpg
    Mental State Function (Fear)

    F(t) = min[Fi, 1],

    where Fi = min[D0/di(t), 1]

    • D0 : constant, D0 = 100

    • Fi : the fear to sighted predator i

    • di : the distance to sighted predator i

    Intention generator l.jpg
    Intention Generator

    Collision detection

    Danger of collision?

    Yes No

    It = avoid

    Predator detection

    F > f0 ?

    If It-1 avoid

    Push the memory



    Fm < f1 ?

    And likes schooling?

    No Yes

    Pop the memory





    IS = eat or mate?

    It = escape

    It = school


    Generate new intention It

    by checking the mental state

    and the habit string

    Go to the focuser

    Go to the next layer

    Behavior routines l.jpg
    Behavior Routines

    • Avoiding-static-obstacle

    • Avoiding-fish

    • Eating-food

    • Mating

    • Leaving

    • Wandering

    • Escaping

    • Schooling

    Schooling behavior routine l.jpg
    Schooling Behavior Routine

    >= 2 neighbours within one

    body length to the front?



    find the closest schoolmate

    in front and speed up towards it

    >= 2 neighbours within one

    body length to the sides?



    find the closest schoolmate to

    the sides and turn to it

    swim in generally the same

    direction as the neighbours?



    make appropriate turns to match

    the general orientation of the neighbours

    Swim at the standard speed

    Predators l.jpg

    Collision detection

    Danger of collision?

    Yes No

    It = avoid

    Pop the memory


    If It-1 avoid

    Push the memory




    Yes No


    IS = eat?


    It = eat

    It = wander

    Go to the focuser

    Go to the next layer

    Predators60 l.jpg

    • Chasing cost function of a prey k

      Ck= dk(1 + β1Sk+ β2Ek /π)

      Ck : chasing cost

      dk : thedistance between the mouth of the predator and the center of prey k’s body

      Sk: Sk=1 if prey k is in a school of fishes, otherwise Sk=0

      Ek : the turning cost

      β : parameters that tune the contributions of Sk and Ek

    Slide61 l.jpg

    • The intention generator of a prey fish is a specialization of the generic intention generator as follows:




    Generate new intention It

    by checking the mental state

    and the habit string

    It = eat

    likes schooling?

    or Is = school?



    It = wander

    It = school

    • Schooling and evading predators are two distinct behaviors of prey.

    Pacifists l.jpg

    • The intention generator of a pacifist included activated mate intention, and deactivated escape and school intention

    Artificial lives fishes and evolving creatures66 l.jpg
    Artificial Lives: Fishes and Evolving Creatures

    • Introduction to Artificial Lives

    • Artificial Fishes and Reinforcement Learned Behavior

    • Evolving Creatures and Genetic Algorithm

    Virtual creatures l.jpg
    Virtual Creatures

    • Complexity vs. Control

    • Genetic Algorithms

      • Darwin (fitness)

      • Differs from previous work

    Creature representation l.jpg
    Creature Representation

    • Genotype

    • Phenotype

    Creature representation69 l.jpg
    Creature Representation

    • Directed Graph

      • Nodes

        • Information

          • Dimensions

          • Joint-type

          • Joint-limits

          • Recursive-limit

          • Neurons

          • Connections

            • Child Node

            • Position

            • Orientation

    Creature control l.jpg
    Creature Control

    • Brain

      • A directed graph of “neurons”

    • Effectors

      • Applied at Joints as Forces or Torques

        • Muscle Pairs

    Creature control71 l.jpg
    Creature Control

    • Sensors

      • Joint Angle Sensors

      • Contact Sensors

        • Multiple Sensors per Unit

        • Self vs. Environment

      • Photosensors

        • Provides relative coordinates of light source

    Creature control72 l.jpg
    Creature Control

    • Neurons

      • Provide different functions

        • Sum, product, abs, max, sin, cos, oscillators, etc…

      • Output vs. Input

        • Number of inputs dependant on function

        • Output dependant on input and maybe previous state

    Creature control73 l.jpg
    Creature Control

    • Effectors

      • Connected to sensor or neuron

      • Control a single DOF of a joint

      • Scaled by Maximum-strength

    Combining control and representation l.jpg
    Combining Control and Representation

    • Nodes

      • Contain unit information

      • Contain neuron information

    • Also a global set of neurons

    Physical simulation l.jpg
    Physical Simulation

    • Collision Detection

      • Bounding Box + Pair Specific

    • Collision Response

      • Impulses + penalty springs

    • Friction

    • Viscosity

      • For simulating underwater

    • Bugs will be found and exploited!

    Behavior l.jpg

    • Evolution for a specific behavior

      • Swimming

      • Walking/Limping

      • Jumping

      • Following (Land/Water)

    • Fitness function evaluated at each step

      • Weights for more preferred methods

    Evolution l.jpg

    • Recipe for a successful evolution

      • Create initial genotypes

        • From scratch

        • From pre-evolved creatures

        • From the user

      • Calculate survival ratio

      • Evaluate fitness and kill off the weaklings

      • Reproduce the Alpha-males

        • Reproductive ability

      • Evolve, and proceed to step 3.

    Evolution79 l.jpg

    • Directed Graph Mutation

      • Node internal parameters

      • Adding random node

      • Parameters of each connection

      • Adding/removing random connections

      • Removing unconnected elements

    • Mutation frequency scales inversely with graph size

    Evolution80 l.jpg

    Mating Directed Graphs

    • Reproductive Method

      • 40% Asexual

      • 30% Crossover

      • 30% Grafting

    Evolution81 l.jpg

    • User Control

      • Choose genotypes to survive

      • Design initial aesthetic genotype

      • Evolve only the aesthetic phenotype, then add a brain.

    Performance l.jpg

    • CM-5 with 32 processors – 3 Hours

      • Population of 300

      • 100 Generations

    Results l.jpg

    • Homogeneity

    • Swimmers

      • Paddlers

      • Tail-waggers

    • Walkers

      • Lizard-like

      • Pushers/Pullers

      • Hoppers

    • Followers

      • Steering Fins

      • Paddlers

    Virtual creatures84 l.jpg
    Virtual Creatures

    • They are the one-armed, three-legged, limping, falling and drowning creatures of the virtual world.