Retargetting Motion to New Characters

1. Retargetting Motion to New Characters Michael Gleicher

2. Homework • Keep working on next homework • Due on Tuesday • Build a mocap data previewer • mocap.cs.cmu.edu (see Tools and Resources) • Load data file • Create favorite data structures • Use DOFs to animate a simple character

3. Homework • Grading • Demo your IK solution Sunday or Monday • Room 002a this time! • Sign up sheet up front

4. Readings • Interactive Control of Avatars Animted with Human Motion Data (Lee et al.) • Motion Graphs (Kovar et al.)

5. Spacetime Constraints • Homeworks look good • Let’s discuss for a moment • Sequential Quadratic Programming • aka Newton’s method and Newton-Lagrange • Wilson (1963) and Beale’s SOLVER (1967) • A member of the descent family of constrained optimization techniques • Makes critical assumption that approximating constraints with linear functions is meaningful • Cost function must be quadratic

6. Spacetime Constraints • This is just a search method • Penalty Methods permit constraints to be violated when searching for optimality • Non-linear Least Squares used by Gleicher • Conjugate gradient • Simulated annealing loses benefit of derivatives • What about GA?

7. Spacetime Constraints • The building blocks define SQP • The implementation of SQP can vary • Matrix inversion • Special routines for handling sparse or positive definite matrices

8. Motivation: Why Do We Care? • Have one animated motion, want another • For characters with identical structure, but different proportions • Manual tweaking is slow and difficult • Want to preserve high-level properties that are hard to define

9. Simple Try 1 • Just reuse parameters of original motion in new motion with different character hmm … something’s not quite right …

10. Simple Try 2 • Need to meet constraints, for instance foot touches floor • No problem, just use inverse kinematics to establish constraints each frame … better, but not very smooth …

11. What’s the problem? • No temoporal notion of constraints • Solving locally for each frame generates unwanted artifacts, can’t plan ahead • Problem: introduces high frequency motions

12. Try again with Filtering • Apply a low-pass filter to remove unwanted high frequencies • But, may violate constraints • Want to preserve high frequencies in original motion, without introducing new ones Original IK Smoothed IK Retargetted

13. Enough Failed Attempts • Need a way to solve the constrained optimization problem that takes the whole motion into account • Where have we seen this before? • Spacetime constraints • Through-the-lens camera control • Neuroanimator • Virutal Creatures

14. Spacetime Constraints • Mathematically encode all constraints and objectives • Not so easy to capture desirable aspects of original motion mathematically • More constraints and more complicated objective functions take longer to solve • Don’t use physics constraints on every step

15. Tools for Traveling through Space and Time • Constraints • Identify features that must be present

16. Constraints qti = parameters of motion (joint angles and root position) at time ti • Define constraint as: f(qti) = c • c is a constant • Constraint examples • Parameter’s value is in a certain range (joint limits) • Point on character is in specific location or within a range • Two points are a certain distance apart • Vector between points has a certain orientation

17. Tools for Traveling through Space and Time • Objective Functions • Stay close to the original (especially high frequency motion) • Use constraints to keep objective function from being too simple minimize g(x)subject to f(x) = c

18. Objective Functions • Goal: Minimize noticeable change • Hard to define, choose something simple instead • Minimize differences between new motion and original motions • Define functions of motion parameters • Original motion: • New (retargeted) Motion: • Difference between the two: • Objective Function:

19. Tools for Traveling through Space and Time • Initial Solution for Solver • Some basic scaling of original data to get a good guess

20. Starting Point • Good initial motion can speed numerical solver • Start just by scaling motion to match scaled character • If needed, translate motion to get as close as possible to satisfying constraints

21. Representation • Want to minimize introduction of high frequencies • Choose representation of to achieve this • Cubic B-splines • Control point spacing determines frequency limit • Try having control points every 2, 4 or 8 frames (uniform spacing)

22. Retargeting Procedure • Take original motion and identify constraints • Scale and translate to find initial estimate for solution, m1(t) • Choose representation for d(t) – what is the control point spacing • Solve non-linear constraint problem: find d(t) that will satisfy constraints when added to motion estimate from step 2 (use spacetime) • If constraints are not well satisfied, solve again using m1(t) + d(t) as initial motion in step 2 and choose a denser spacing in step 3

23. Solving the Constrained Optimization Problem • Want to solve for control point values (call the vector of these x) • Define constraints and objective functions in terms of these control points • minimize g(x) subject to f(x) = c • g(x) = ½ xMx (a weighted sum of squares, diagonal matrix M gives weights) • In addition to SQP could use a non-linear least squares solver to minimize constraint residuals (distance from meeting constraints)

24. The examples • 120 Hz mocap data downsampled to 30 Hz (one example from text was rotoscoped) • Maker positions converted to articulated character parameters • Euler angles for joints (3 DOF) • Except elbow, knee, ankles • Frequently, data for hand and feet is unavailable

25. Getting 60% shorter • Gray character results in foot skate • Left shows strides getting longer (black) • Right shows adjusting foot plants (black)

26. 2-D Walking • 82 frames / 15 fps • 14 DOFs (2 pos + 12 ang) • 146 constraints on heel/toe • 328 inequality constraintsto keep feet above ground • 1968 joint limit constraints

27. 3-D Walking • 34 DOF, 112 Frames • 4193 scalar constraints (~354 active at any one time… computation in seconds

28. Ladder • Their custom SQP, LMULT permits some constraints to be violated by ½ in. • Least squares optimization results in errors less than ¼ in.

29. Dancing • Woman mightlift off ground • Male permittedto move upperbody • 1200 constraints • 14 seconds to solve w/ 1/8 in.errors

30. Extensions • Characters with different structure • First use standard retargeting to a character whose dimensions roughly match those of the new character • Find correspondences (manually) between features of original and new character, make these constraints • Use retargeting with spacetime constraints again, to find a new motion that satisfies these constraints

31. Conclusions • Basic retargeting works for characters with same structure, different limb lengths • Control point spacing is important for preserving desired frequencies • Enforcing uniformity across time is a limitation • Enforcing certain features of original motion may not produce natural retargeted motion

32. Morphing simulated characters • Hodgins and Pollard • Adapting Simulated Behaviors For New Characters • Siggraph 97

33. Morphing simulated characters • We know how character has changed in size and mass • We have no control of joint positions explicitly • We adapt character through control algorithms • Timing parameters • Muscle stiffnesses and damping

34. Morphing simulated characters

35. We know a lot about scaling • Animation of Dynamic Legged Locomotion – SIGGRAPH 91, Raibert and Hodgins • Scale kangaroo by L • It will have to jump L times as high • Because gravity is unchanged • more time will pass before landing (for L >1) • Have cadence 1/sqrt(L) of original • But because limbs are longer (L>1) • travel sqrt (L) as fast

36. We know a lot about scaling

37. Motion Graphs • Introduction to the paper by Kovar, Gleicher, and Pighin • Motion can be “sampled” just as videoclips were inVideoTextures

38. Sampling mocap • Transitioningfrom frame i toframe j • White are goodtransitions • Green arelocal min

39. One change vis. Video Textures • Multiple timesteps are grouped into a clip • The clips are connected to one another in a graph • Transitioning from one clip to another is not as easy as cross-fade or morph from image domain

40. General outline • Build motion graph • Remove dead ends (find strongly connected components) • Search for paths that produce desired motion (branch and bound search) • Blend clips to form a smooth motion