1 / 61

Learning Human Pose and Motion Models for Animation

Learning Human Pose and Motion Models for Animation. Aaron Hertzmann University of Toronto. Animation is maturing …. … but it’s still hard to create. Keyframe animation. Keyframe animation. q 1. q 2. q 3. q (t). q (t). http://www.cadtutor.net/dd/bryce/anim/anim.html.

kura
Download Presentation

Learning Human Pose and Motion Models for Animation

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Learning Human Pose and Motion Models for Animation Aaron Hertzmann University of Toronto

  2. Animation is maturing … … but it’s still hard to create

  3. Keyframe animation

  4. Keyframe animation q1 q2 q3 q(t) q(t) http://www.cadtutor.net/dd/bryce/anim/anim.html

  5. Characters are very complex • Woody: • 200 facial controls • 700 controls in his body http://www.pbs.org/wgbh/nova/specialfx2/mcqueen.html

  6. Motion capture [Images from NYU and UW]

  7. Motion capture

  8. Mocap is not a panacea

  9. Goal: model human motion What motions are likely? Applications: • Computer animation • Computer vision

  10. Related work: physical models • Accurate, in principle • Too complex to work with • (but see [Liu, Hertzmann, Popović 2005]) • Computationally expensive

  11. Related work: motion graphs Input: raw motion capture “Motion graph” (slide from J. Lee)

  12. Approach: statistical models of motions Learn a PDF over motions, and synthesize from this PDF [Brand and Hertzmann 1999] What PDF do we use?

  13. Style-Based Inverse Kinematics with: Keith Grochow, Steve Martin, Zoran Popović

  14. Motivation

  15. Body parameterization Pose at time t: qt Root pos./orientation (6 DOFs) Joint angles (29 DOFs) Motion X = [q1, …, qT]

  16. Forward kinematics Pose to 3D positions: FK qt [xi,yi,zi]t

  17. Degrees of freedom (DOFs) q Constraints Problem Statement Generate a character pose based on a chosen style subject to constraints

  18. Style Learning Real-time Pose Synthesis Approach Off-Line Learning Motion Constraints Synthesis Pose

  19. Features y(q) = q orientation(q) velocity(q)[ q0 q1 q2 …… r0 r1 r2 v0 v1 v2 … ]

  20. Goals for the PDF • Learn PDF from any data • Smooth and descriptive • Minimal parameter tuning • Real-time synthesis

  21. Mixtures-of-Gaussians

  22. x2 x1 Latent Space GPLVM Gaussian Process Latent Variable Model [Lawrence 2004] GP y2 -1 x ~ N(0,I) y ~ GP(x; ) y3 y1 Feature Space Learning: arg max p(X, | Y) = arg max p(Y | X, ) p(X)

  23. Scaled Outputs Different DOFs have different “importances” Solution: RBF kernel function k(x,x’) ki(x,x’) = k(x,x’)/wi2 Equivalently: learn x  Wy where W = diag(w1, w2, … wD)

  24. Precision in Latent Space 2(x)

  25. SGPLVM Objective Function x2 y2 y3 y1 x1

  26. Baseball Pitch

  27. Track Start

  28. Jump Shot

  29. Style interpolation Given two styles q1 and q2, can we “interpolate” them? Approach: interpolate in log-domain

  30. (1-s) s Style interpolation

  31. s (1-s) Style interpolation in log space

  32. Interactive Posing

  33. Interactive Posing

  34. Interactive Posing

  35. Multiple motion style

  36. Realtime Motion Capture

  37. Style Interpolation

  38. Trajectory Keyframing

  39. Posing from an Image

  40. Modeling motion GPLVM doesn’t model motions • Velocity features are a hack How do we model and learn dynamics?

  41. Gaussian Process Dynamical Models with: David Fleet, Jack Wang

  42. Dynamical models xt+1 xt

  43. Hidden Markov Model (HMM) Linear Dynamical Systems (LDS) [van Overschee et al ‘94; Doretto et al ‘01] Switching LDS [Ghahramani and Hinton ’98; Pavlovic et al ‘00; Li et al ‘02] Nonlinear Dynamical Systems [e.g., Ghahramani and Roweis ‘00] Dynamical models

  44. Latent dynamical model: latent dynamics pose reconstruction Assume IID Gaussian noise, and with Gaussian priors on and Marginalize out , and then optimize the latent positions to simultaneously minimize pose reconstruction error and (dynamic) prediction error on training data . Gaussian Process Dynamical Model (GPDM)

  45. where • is a kernel matrix defined by kernel function • with hyperparameters Dynamics The latent dynamic process on has a similar form:

  46. Subspace dynamical model: Markov Property Remark: Conditioned on , the dynamical model is 1st-order Markov, but the marginalization introduces longer temporal dependence.

  47. reconstruction likelihood dynamics likelihood priors training motions latent trajectories hyperparameters To estimate the latent coordinates & kernel parameters we minimize with respect to and . Learning GPDM posterior:

  48. Motion Capture Data ~2.5 gait cycles (157 frames) Learned latent coordinates (1st-order prediction, RBF kernel) 56 joint angles + 3 global translational velocity + 3 global orientation from CMU motion capture database

  49. large “jumps’ in latent space 3D GPLVM Latent Coordinates

  50. Volume visualization of . (1st-order prediction, RBF kernel) Reconstruction Variance

More Related