1 / 25

Video Motion Capture

Video Motion Capture. Christoph Bregler Jitendra Malik UC Berkley 1997. Overview. Challenges Review Method Results Conclusions. Challenges. High Accuracy Frequent Inter-part Occlusion Low Contrast. Review. Review . Motion capture on synthetic images O’Rouke and Balder, 1980

mosleyd
Download Presentation

Video Motion Capture

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Video Motion Capture Christoph Bregler Jitendra Malik UC Berkley 1997 Phillip Saltzman

  2. Overview • Challenges • Review • Method • Results • Conclusions Phillip Saltzman

  3. Challenges • High Accuracy • Frequent Inter-part Occlusion • Low Contrast Phillip Saltzman

  4. Review Phillip Saltzman

  5. Review • Motion capture on synthetic images • O’Rouke and Balder, 1980 • 1 DOF marker free tracking • Hogg, 1983. Rohr, 1993 • Higher DOF full body tracking • Gravrila and Davis, 1995 Phillip Saltzman

  6. Review About the previous work • All in controlled environments with high contrast and clear edge bounries • Most use skintight suits or markers • Camera calibration needed Phillip Saltzman

  7. Method Phillip Saltzman

  8. Method Basic Assumptions • From frame to frame, all intensity pixel intensity changes are local: • u is motion model and is written as a matrix equation: Phillip Saltzman

  9. Method Finding Gradients • Gradient form of the first equation: • Find a least squares solution to f • Warp image I(t+1) based on f • Find new gradients • Repeat to minimize Phillip Saltzman

  10. Method Motion as twists • Standard pose matrix to move from object space to camera space (3D) • Scaled orthographic projection moves to image space • Requires knowing something about the 3D model of the image. Approximated as ellipsoids. Phillip Saltzman

  11. Method Motion as twists • Any motion can be represented as a rotation about an axis, and a translation about that axis • For example, to make this motion: Phillip Saltzman

  12. Method Motion as twists You make this motion: Phillip Saltzman

  13. Method Motion as twists • Twists can be represented as small vector or matrix • Can be made to a pose by • Encode the motion of a pixel between two frames Phillip Saltzman

  14. Method Motion as twists • Linear algebra manipulation allows using the twist vector to write a motion equation for each pixel • Those equations are put in a vector and used to find a global f parameter for that object Phillip Saltzman

  15. Method Kinematic chains • Body parts represented as multiple connected objects • Each object can be found by the top pose and an angle and twist for each object down the chain • More linear algebra is used to find a f for each body part Phillip Saltzman

  16. Method Multiple cameras • Adds accuracy because change of fully occluded parts drop with each view • Normal motion equation is: • H is system of equations for each pixel • f is global parameter vector for each object • z is initial position of the pixel Phillip Saltzman

  17. Method Multiple cameras • Adding synchronized cameras: • H becomes a matrix where each column represents a view • The f vector gets a term Wfor each view that represents the pose seen from that view • z becomes a vector with an initial position for each view. Phillip Saltzman

  18. Method Support maps • Limits pixel search to area defined by map for speed • Value for each pixel in range [0,1], where 1 means pixel is in the region • Method for finding starts as an elliptical guess, but refining it is not described Phillip Saltzman

  19. Method Algorithm review Input: Image I(t), I(t+1), pose and IK angles Output: Pose and IK angles for I(t+1) Find 3D points for each pixel in image Compute support map for each segment Set poses and IK angles for I(t+1) = I(t) Iterate: Compute gradients Estimate f Update poses and IK angles Warp image based on the pose and support map Phillip Saltzman

  20. Method Initialization • Algorithm depends on known positions for the first frame • For multiple views, each first frame must be initialized • User clicks joint positions, and 3D estimations and joint angles are computed • Values like symmetry can be enforced Phillip Saltzman

  21. Results Phillip Saltzman

  22. Results In Lab Movie • Single angle • 53 frames with decent results • Upper leg hard to track, so IK chain compensates with lower leg and torso Phillip Saltzman

  23. Results Oblique Lab Movie • Oblique angle • Tracking over 45 frames • Algorithm could track change in scale due to perspective changes Phillip Saltzman

  24. Results Digital Muybridge • Oldest known “movie” • High noise and low contrast • Low framerate • Multiple views Phillip Saltzman

  25. Conclusions Future Work/Shortcomings • May break with large movements • Fixed camera only • Did not show tracking of back limbs • No timing data • Few results Phillip Saltzman

More Related