chap 6 motion capture mocap n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Chap 6 Motion Capture (Mocap) PowerPoint Presentation
Download Presentation
Chap 6 Motion Capture (Mocap)

Loading in 2 Seconds...

play fullscreen
1 / 62
abel-dunlap

Chap 6 Motion Capture (Mocap) - PowerPoint PPT Presentation

161 Views
Download Presentation
Chap 6 Motion Capture (Mocap)
An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

  1. Chap 6Motion Capture (Mocap)

  2. What is motion capture? • Key-frames, forward kinematics, inverse kinematics • A daunting task for creating physically realistic motion • Requires a large amount of animation talent • Record the motion and map it into a synthetic object • Much easier • Realistic motion CS, NCTU, J. H. Chuang

  3. What is motion capture? • track motion of reference points • convert to joint angles to drive • an articulated 3D model • a deformable surface CS, NCTU, J. H. Chuang

  4. What is motion capture? • Objective • Reconstruct the 3D motion of a physical object and apply it to a synthetic object • Recording 3D live action, including • whole body • face • hands • animals CS, NCTU, J. H. Chuang

  5. Applications • Animation • Special effects • Robot control • Interactive characters • Games CS, NCTU, J. H. Chuang

  6. Computer Puppetry Shin et al., “Computer puppetry” CS, NCTU, J. H. Chuang

  7. VideoOptical Motion Capture in Games CS, NCTU, J. H. Chuang

  8. Video: Lord of the rings: Gollum CS, NCTU, J. H. Chuang

  9. Video • Facial Mocap in King Kong CS, NCTU, J. H. Chuang

  10. Video: Irobot CS, NCTU, J. H. Chuang

  11. Pros and Cons of Motion Capture • Pros • All fine details of human motion will be recorded-- if they can be captured • Cons • Not so easy to • Edit • Generalize • Control • Not cheap CS, NCTU, J. H. Chuang

  12. What is captured? • What do we need to know? • X,Y,Z • roll, pitch, yaw • Errors cause • Joints to come apart • Links to grow/shrink • Bad contact points CS, NCTU, J. H. Chuang

  13. What is captured? • Large and small scale CS, NCTU, J. H. Chuang

  14. How to use the data? • Off-line • Processed by filtering, inverse kinemtics • Produce libraries of motion trajectories • Choose among them • Blend between them • Modify on the fly • On-line (performance animation) • Driving character directly based on what actor does in real time CS, NCTU, J. H. Chuang

  15. History of the technology • Recording motion for biomechanics • High accuracy • Fewer recorded pints • Hand digitizing film • Supplement with force plate, muscle activity • Computer animation • Rotoscoping • VR tracking technology • Less accuracy required • Fewer sensors CS, NCTU, J. H. Chuang

  16. Optical Motion Capture (Passive) • Passive reflection • Camera • Infrared, visible, or near infrared strobes • High resolution (1 to 4 million pixels) • 120-240 frames/sec (max 2000 frames/sec) • Not outdoors • No glossy or reflective materials • Tight clothing • Occlusion of markers by limbs or props CS, NCTU, J. H. Chuang

  17. Optical Motion Capture (Active) • Active output of the LED • No marker confusion problem • Outdoor capture • 120 frames/sec (128 markers or four persons) • 480 frames/sec (32 markers or single person) • 1/3 the cost of passive systems CS, NCTU, J. H. Chuang

  18. Magnetic Motion Capture • Electromechanical transducer • Heavier sensors • Wires on body (wireless back to base station) • Limited accuracy (~10x less accuracy than optical) CS, NCTU, J. H. Chuang

  19. Magnetic Motion Capture (cont.) • Smaller workspace • Sensors are the cost • Sensitive to EMI/metal • Relatively cheaper than optical device Ascension MotionStar Wireless CS, NCTU, J. H. Chuang

  20. Mechanical Motion Capture • Subject wears an exoskeleton • No interference from light or magnetic field • No marker confusions • No range limit • Some restriction of movement • Absolute position is unknown CS, NCTU, J. H. Chuang

  21. Mechanical Motion Capture • Data glove • Bend sensor + optical tracking • 6 DOF • video http://www.vrealities.com/glove.html CS, NCTU, J. H. Chuang

  22. Technology Issues • Resolution/range of motion • Calibration • Accuracy • Marker movement • Sensor noise • Restrictions on the environment • Capture rate • Occlusion/correspondence CS, NCTU, J. H. Chuang

  23. Markers - Examples CS, NCTU, J. H. Chuang

  24. Skin Motion CapturePark & Hodgins, SIGGRAPH’06 • Uses a conventional optical motion capture system 40-60 markers CS, NCTU, J. H. Chuang

  25. A dense set of markers Skin Motion Capture • Uses a conventional optical motion capture system CS, NCTU, J. H. Chuang

  26. A dense set of markers Skin Motion Capture • Uses a conventional optical motion capture system CS, NCTU, J. H. Chuang

  27. A dense set of markers Detailed surface model Skin Motion Capture • Uses a conventional optical motion capture system CS, NCTU, J. H. Chuang

  28. A dense set of markers Detailed surface model Skin Motion Capture • Uses a conventional optical motion capture system Data collection and cleaning CS, NCTU, J. H. Chuang

  29. A dense set of markers Detailed surface model Skin Motion Capture • Uses a conventional optical motion capture system Data collection and cleaning Skin Animation CS, NCTU, J. H. Chuang

  30. What is motion capture?Steps after motion is captured • Process 2D images to locate, identify, and correlated the markers in multiple video streams • Requires image processing techniques • Reconstruct the 3D locations of the markers • Requires camera calibration • Overcome numerical inaccuracies • Constrain the 3D marker locations to a model of physical system whose motion is being captured • Require satisfying constraints between relative marker positions CS, NCTU, J. H. Chuang

  31. Camera calibration • Find the locations and orientations of cameras in world space and the intrinsic properties of the camera such as focal length, image center, and aspect ratio • Use a simple pinhole camera model • An idealized model • Usually sufficient for graphics and animation CS, NCTU, J. H. Chuang

  32. Camera calibration Pinhole camera model CS, NCTU, J. H. Chuang

  33. 3D position reconstruction • Locate a marker’s position in at least two views relative to known camera positions CS, NCTU, J. H. Chuang

  34. 3D position reconstruction • Locate a marker’s position in at least two views relative to known camera positions CS, NCTU, J. H. Chuang

  35. Fitting to the skeleton • After the motion of the individual markers looks smooth and reasonable, • Attach markers to the underlying skeletal structure that is to be controlled by the digitized motion • The position of each marker is used to absolutely position a specific joint • Is not straight due to noise, smoothing, and inaccuracy • Change in bone length can be significant • Markers are located outside the joints at the surface CS, NCTU, J. H. Chuang

  36. Fitting to the skeleton • Given marker location and the relative distance from marker to the joint is not sufficient since the direction of displacement is not known • Put markers on both side of the joint • Work fine for simple joints, but not the complex joints (such as shoulder and spine) • Use normal of a plane formed by 3 markers • Wrist-elbow-shoulder markers plane normal • Offset the elbow marker in the direction of plane normal by the amount measured from the performer CS, NCTU, J. H. Chuang

  37. Manipulating motion capture data • Processing the signal • Motion signal processing and Motion warping • Considers how frequency components capture various qualities of the motion. And warps the signal in order to satisfy user-supplied key-frame-like constraints • Retargeting the motion • Map the motion onto the mismatched synthetic character and modify it to satisfy important constraints • Combining the motion • Assemble motion segments into longer sequence CS, NCTU, J. H. Chuang

  38. Processing the signal • Motion parameter values as a time-varying signal • Consider values of an individual parameter of the captured motion as a time-varying signal • Signal can be decomposed into frequencies, time-warped, interpolated with other signals • Multidimensional signal (vector-valued signal) • A function of time m(t): R -> Rn • Not really in Rn • 3 DOF in translation, 3 DOF in absolute orientation, many DOF in relative orientations CS, NCTU, J. H. Chuang

  39. Processing the signal • Motion signal processing • Considers how frequency components capture various qualities of the motion. • Lower frequencies – represent base activities (walking) • Higher frequencies – idiosyncratic movements (walking with limp) • Motion manipulation/editing/warping • Warps the signal in order to satisfy user-supplied key-frame-like constraints. CS, NCTU, J. H. Chuang

  40. Processing the signalMotion signal processing • Treat motion as a multi-dimensional signal • Low pass filtering • noise removal • High pass filtering • style change • High frequency component can be motion details, not just noise • Modify a motion through filtering is not easy • Physical constraints (joint limit, ground contact)? • Naturalness? CS, NCTU, J. H. Chuang

  41. Processing the signalMotion signal processing • Frequency bands can be extracted, manipulated, and reassembled to allow the user to modify certain qualities of the signal while leaving others undisturbed. • Signal is successively convolved with expanded versions of a filter kernel. • Gains of each band are adjusted by the user and can be summed to reconstruct a motion signal. CS, NCTU, J. H. Chuang

  42. Motion signal processing • PROBLEM: High frequencies can be important! • Getting rid of them makes motion look soggy • ANSWER: Do not over-apply LPF • Small amounts of Low-Pass Filtering • Noise modeling • Non-linear filters ? CS, NCTU, J. H. Chuang

  43. Motion signal processing High frequencies are important! • Don’t occur often • Always significant • Impact • Rapid, sudden movement • Emphasis • Sensitivity of perception • Eye is sensitive to high frequencies CS, NCTU, J. H. Chuang

  44. Motion manipulation/editing/warpingWhy? • What you get is not what you want • You get observations of the performance • Specific performer (a real human) • Specific motion • With the noise and “realism” of real sensors • You want animation • A character • Doing something • Or something similar but not the same CS, NCTU, J. H. Chuang

  45. Motion manipulation/editing/warping • Manipulate time • Time warp / Speed control • m(t) = m0( f(t) ) • f : R  R • Manipulate value • m(t) = f(m0(t) ) • f : Rn Rn CS, NCTU, J. H. Chuang

  46. Motion manipulation/editing/warpingManipulating time m(t) = m0( f (t) ) • Time scaling f(t) = k t • Time shifting f(t) = t + k • Time warping • Interpolate a table • Align events • Speed control • Ease in/Ease out CS, NCTU, J. H. Chuang

  47. Motion manipulation/editing/warpingManipulating values • Scale • Shift • Blending • Filtering • Transition between motions • Cyclification • Change style • Constraints on the motion • Concatenation CS, NCTU, J. H. Chuang

  48. Motion manipulation/editing/warpingMotion warping • Witkin and Popovic, “Motion Warping,” SIGGRAPH’95 • Keyframes as constraints in smooth deformation • Keyframe placing the ball on the racket at impact CS, NCTU, J. H. Chuang

  49. Motion manipulation/editing/warpingMotion warping CS, NCTU, J. H. Chuang

  50. Manipulating motion capture dataRetargeting the motion • Map the captured motion onto the mismatched synthetic character and then modify it to satisfy important constraints • Constraints: • avoid foot penetration of the floor, avoid self-penetration, no feet sliding when walking • A new motion is constructed, as close to the original motion as possible, while enforcing the constraints. • New motion is formulated as a space-time, non-linear constrained optimization problem. CS, NCTU, J. H. Chuang