1 / 24

Brett Allen 1,2 , Brian Curless 1 , Zoran Popovi ć 1 , and Aaron Hertzmann 3

Learning a correlated model of identity and pose-dependent body shape variation for real-time synthesis. Brett Allen 1,2 , Brian Curless 1 , Zoran Popovi ć 1 , and Aaron Hertzmann 3 1 University of Washington 2 Industrial Light & Magic 3 University of Toronto. movies. games. telepresence.

sezja
Download Presentation

Brett Allen 1,2 , Brian Curless 1 , Zoran Popovi ć 1 , and Aaron Hertzmann 3

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Learning a correlated model of identity and pose-dependent body shape variation for real-time synthesis Brett Allen1,2, Brian Curless1, Zoran Popović1, and Aaron Hertzmann3 1 University of Washington 2 Industrial Light & Magic 3 University of Toronto

  2. movies games telepresence design Motivation

  3. + Identity pose shape Goal • We would like to be able to generate body models of any individual in any pose. - want to synthesize models in real-time - model should be learnable from real data

  4. Data • CAESAR data set: 44 subjects in 2 poses • Multi-pose data set: 5 subjects in 16 poses • Dense-pose data set: 1 subject in 69 poses [Anguelov et al. 2005]

  5. Related work

  6. Anatomical methods Chadwick et al. 1989 Turner and Thalmann 1993 Scheepers et al. 1997 Wilhelms and Van Gelder 1997 … Aubel 2002

  7. v = shape vector p = example parameters  = function parameters Example-based methods Given: n examples v0…vn and n sets of parameters p0…pn (optional) find .

  8. 1 2 2 1 3 4 4 3 Allen et al. 2002 Scattered data interpolation columns of  = key shapes i = reconstruction weights from applying k-nearest neighbors or RBFs on p Lewis et al. 2000 Sloan et al. 2001 Kry et al. 2002

  9. An aside on enveloping Enveloping + scattered data interpolation = “corrective enveloping”

  10. Blanz & Vetter 1999 Latent variable models x = latent variable (component weights) W = components in columns v = average shape Allen et al 2003 Seo et al 2003 Anguelov et al 2005 …

  11. Pose variation vs body variation Sloan et al. 2001

  12. Pose variation vs body variation Anguelov et al. 2005

  13. Intrinsic skeleton parameters: bone lengths and carrying angles Our approach c = “character vector”: all information needed to put a character in any pose v = shape in a particular pose

  14. Two Problems 1. Scans might not be at “key” poses 2. Scans are not complete Maximize: p(c | {e()}) …actually, we don’t know the pose or skinning weights either: Maximize: p(c, s, q{} | {e()})

  15. Maximum a posteriori estimation

  16. Results

  17. Going to multiple characters • One possibility: Learn several character vectors separately, then run PCA. • Two problems: • the character vector contains values that have different scales (rest positions, offsets, bone lengths) • we don’t have enough data!

  18. Identity variation ~ ~ c is the character vector of the th example person g(c,s,q) applies skinning and pose space deformations

  19. Alternating optimization • We initialize the {x} with the weights from running PCA on estimated skeleton parameters. • We then optimize W, c, s, q. • Then we optimize for {x}. • Repeat…

  20. Results

  21. Results (video)

  22. Conclusions We present a flexible approach for learning body shape variation between individuals and between poses, including the interrelationship between the two. + very general: can handle irregular and incomplete sampling in regard to both the poses/identities scanned, and in the surfaces themselves + the learned model can generate body shapes very quickly (over 75 fps)

  23. Limitations • You need a lot of data! Our data set was too sparse in some areas. • Some poses are hard to capture. • It’s very hard to compensate for the skinning artifacts. • The shape matching could be improved (high-frequency details are lost if the matching is poor).

  24. Acknowledgements • UW Animation Research Labs • Washington Research Foundation • National Science Foundation, NSERC, CFI • Microsoft Research, Electronic Arts, Sony, Pixar • Kathleen Robinette and the AFRL lab • Dragomir Anguelov • Domi Pitturo

More Related