Evolving Character Controllers for Collision Preparation

1. Evolving Character Controllersfor Collision Preparation Robert Rose 09/08/06 Oregon State University

2. Agenda • Motivation • Overview • Background • Character Model • Evolving Character Controllers for Collision Preparation • Results • Conclusion Oregon State University

3. Motivation • Video Games • Games have many collisions with human characters • Recent focus has been on collision response: • Ragdoll (pure physics) • Hybrid Response (Zordan05, Mandel04) • Little attention has been given to collision preparation • A complete preparation and response system would increase a game’s realism dramatically DOA4 - Tecmo NCAA 2003 - EA ABC Television Oregon State University

4. Motivation • Exploration of Evolutionary Robotics • Developing controllers for robots and physically simulated characters by hand is a difficult problem • Evolving Virtual Creatures (Sims94) • Evolved human locomotion (Smith98, Wolff03) • A strong sense… • Holy grail: complete dynamics-based control of human characters • Evolutionary techniques must be the key! Sims94 Smith98 Wolff03 Oregon State University

5. Motivation • Video Oregon State University

6. Overview • Physically simulate character model • Control system • Desired joint angles • Test environment • Throw a projectile at the character • Evaluation system • Measure “pain” perceived by character • Optimization system • Genetic algorithms Oregon State University

7. Background • Motion Capture vs. Physical Simulation • Physically Simulated Humans • Controlling Physically Simulated Humans • Special-purpose controllers • Bruderlin89, Raibert91, Laszlo96, Takashima90, Hodgins95 • Combining controllers • de Garis90, Faloutsos01 • Developing controllers by hand is not optimal • Time consuming • Re-targeting difficult (or not possible) Oregon State University

8. Background • Evolution of Human Character Controllers • Genetic Algorithms • Roberts03, Wyeth03 • Genetic Algorithms + Neural Networks • de Garis90, Smith98, Nolfi00 • Genetic Programming • Gritz95/97, Wolff03 Gritz95 Roberts03 Oregon State University

9. Background • Collision Response • Motion tracking - Zordan02 • Applies torques to character to track motion capture data • Weaken control after impact, then gradually strengthen • Fall control - Mandel04 • Transitions to fall controller after impact or tripping • Motion capture resumes after character is in “rest” state Oregon State University

10. Background • Collision Response • Hybrid Control • Dynamic Response for Motion Capture Data, Zordan05 • Use control system to bring character to nearest motion data • Blend out of control, into motion capture after a short period of time Oregon State University

11. Background • Collision Response • Endorphin - Natural Motion • Offline solution; “multi-pass simulation technique” for building “uncapturable” motion capture sequences • Contains 40-50 preprogrammed controllers that can be applied to your character. Ex: Stagger backwards and fall over • All controllers end in a rest state (ragdoll) Oregon State University

12. Character Model • Physically Simulated Character Model • 9 segments, 8 joints, 18 DOF • Uniform density segments • Simulated in Novodex Oregon State University

13. Character Model • 1 DOF Joint Control • PD-servos - “spring” controller • Ball and Socket Joint Control • Our solution is an extension of PD-servos to 3 dimensions • Controller Stability • Precision loss is our worst enemy • Causes high-frequency oscillations - smoothing outputs helps • Overpowered on the twist axis • Damping the twist axis torque helps Oregon State University

14. Character Model • Pain Mesh • Color codes areas of the character model according to “pain regions” Oregon State University

15. Evolving Character Controllers • Testing Controllers • Evolutionary process generates a lot of controllers • Fitness measures performance of controller during a single test • Multiple tests are necessary to ascertain “total fitness” • Evaluating Controllers • Fitness metrics: pain, energy, distance • Generating Controllers • Chromosome format • Selection process • Mutation and crossover Oregon State University

16. Testing Controllers • 36 tests per controller • 9 tests according to the threat grid • x 4 tests for each mode of joint failure • Threat grid • We desire “general” solutions rather than solutions that train for blocking a specific case • 9 tests with “jitter” Oregon State University

17. Testing Controllers • Joint Failure • Humans typically block with both hands and duck the head • We desire solutions that use all faculties of the character model • Joint failure forces the character to defend itself using: • Both arms • Only the right arm, then only the left arm • No arms at all Oregon State University

18. Evaluating Controllers • Evaluation of a test run requires a fitness function • Fitness = pain + energy + distance • Pain Metric • How effectively did the controller minimize the amount of pain perceived by the character? • Energy Metric • How much energy did the controller use to defend the character? • Distance Metric • How far away did the controller block the projectile from the character’s head? Oregon State University

19. Evaluating Controllers • Pain Metric • Goal: Find controller that cause the least amount of harm to come to the character • Premise: Getting hit in some parts of your body hurts more than others • Method: • Take the point of impact pi and use it to look up the “pain region” the character was hit in -- each region has a pain multiplier • Apply the pain multiplier to the force of the impact • Sum for every point of impact Oregon State University

20. Evaluating Controllers • Pain Metric • Pain multipliers developed ad hoc Oregon State University

21. Evaluating Controllers • Energy Metric • Goal: We want to find “realistic” solutions • Premise: Humans don’t “over-compensate” movement • Method: Penalize controller based on energy consumption • Picking a large value for ct prevents solutions that over-compensate • But, picking too large value has undesirable consequences… • Energy metric overtakes the pain metric • Converges on solutions that only minimize energy • ct = 0.001 Oregon State University

22. Evaluating Controllers • Distance Metric • Goal: We want to find different styles of poses • Premise: Humans sometimes block far from their head • Method: Penalize controllers that block close to the head • Take the distance from the first point of impact (p0) to the head (h) • Round this to the nearest integer • Apply distance multiplier cd • cd = 0.015 Oregon State University

23. Generating Controllers • To generate controllers, we use Genetic Algorithms • Encode desired joint angles as a chromosome • Tournament selection process • Perform mutation and crossover to generate new chromosomes • Slope of solutions over a window determines end Oregon State University

24. Results Oregon State University

25. Genetic Algorithm Parameters • Before we could begin testing our pain measurement theories we needed to stabilize the parameters to the GA • Tested four mutation schemes (below) • Tested two tournament sizes (4, 8) • Tested eight population sizes (100-800 increments of 100) Oregon State University

26. Genetic Algorithm Parameters • Mutation Schemes A-D Fitness vs. Time Oregon State University

27. Genetic Algorithm Parameters • Tournament Sizes 4,8 - Population Sizes 100-800 Fitness vs. Time Oregon State University

28. Genetic Algorithm Parameters • Conclusion: • Mutation Scheme A • Clear winner, others didn’t converge as reliably • Tournament size didn’t seem to matter • Others have observed the same behavior (Gritz97) • 4 was chosen arbitrarily • Population size not so clear • Population sizes of 400 and over always converged on a solution • Population sizes under 400 converged, but more sporadically • Ultimately, convergence time was the decider: 400 seemed optimal • Footnote: • Early in this work I used culling as a selection process… • Simulations took overnight to converge on a solution! Oregon State University

29. Results • Evolution in progress Oregon State University

30. Results • Results at various angles Oregon State University

31. Results • Hand emphasis: hands-only collision Oregon State University

32. Results • Video Oregon State University

33. Conclusion • Collision Preparatory Poses • We successfully generated preparatory poses automatically • Tweaking the fitness function and collision space gave us different styles of results • Our system occasionally gave us some odd solutions • Blocking with the back of the hands - Ow! • The mysterious “inverted elbow” pose - I wish I could do that! • Opportunities • Energy metric enhancements • Consider the direction torque is being applied in - no back of hands • Joint limit penalties • Penalize the system for placing the character in positions that would fracture a bone on impact Oregon State University

34. Conclusion • Evolving Character Controllers • Automated development of human character controllers is a hard problem! • Developing realistic controllers requires a large search space • Evaluation of human movement requires codifying what is “human” • Our problem domain works well because of its simplicity • Ultimately, it’s about what the artist wants • Artist codification of a problem, combined with pain measurement could be a powerful tool Oregon State University