1 / 23

Motion Capture: Recent Trends

Motion Capture: Recent Trends. Rama Hoetzlein, 2011 Lecture Notes Aalborg University at Copenhagen. Facial Capture. Frederick I. Parke, University of Utah Computer Generated Animation of Faces 1972. Facial Capture.

lydia
Download Presentation

Motion Capture: Recent Trends

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Motion Capture:Recent Trends Rama Hoetzlein, 2011Lecture NotesAalborg University at Copenhagen

  2. Facial Capture Frederick I. Parke, University of UtahComputer Generated Animation of Faces1972

  3. Facial Capture 1988 B. Robertson, Mike the Talking Head, Computer Graphics World 11 (7):57 Method 1 – Phonemes When a particular type of sound is spoken: phonemes Specific shapes of the whole face are captured. (top down) Phonemes – the sounds that make up a word, not letters balloon b – ah – l – oo – n Method 2 – Feature Tracking Parts of the face are tracked separately.Each part contributes to overall motion. (bottom up) Motion is the sum of many features. Works for speech and other facial expressions

  4. Trend – Markerless Facial Capture Surface is tracked based on image distortion rather than markers. Emily, Image Metrics, 2010 http://www.youtube.com/watch?v=UYgLFt5wfP4&feature=player_embedded

  5. Problem:Motion capture records the body over volumes up to: 10 x 10 sq. meters (30 sq. ft)Facial capture records subtle details over space of: 30 x 30 cm (1 sq. ft) How to capture both the large-scale motion of the body and subtle motion of the face during a single performance?

  6. Trend –Performance captureis a collection of techniques that combine torecord the totalmotion of an actor. Avater (2010),James Cameron Solution: Block off the face using individual, head-mounted cameras, which record only the face. Use motion cameras and passive markers for the body.Allows for both large volumes and small details.

  7. Markers include: Body capture Green lines, white dots Facial capture Head-mounted device, /w camera booms Hair capture Blue and red ropes

  8. Hardware Trends

  9. Trend – Markerless capture:Origins in 3D laser scanning 3D Lego Digitizer http://www.rchoetzlein.com/project/digitizer/

  10. Trend – Markerless capture: Structured Light Q: High frequency gives details about height of point. But how do we tell if the point is on left or right side of obj? Faster: Do all lines at onceProjector with structured light mapped onto the object. Use two cameras to determine object structure.Structured light can be linear, binary coded, gray coded, or color coded. The encoding allows you to uniquely identify points.Light may be infrared (Kinetic). A: Low frequency gives overall characteristics of pixels.

  11. Pointcloud Volume construction Fittorse Fitextremities No markers.Structured light creates a point cloud.Skeleton is fit inside point cloud from root joints to extremities.Torso defines primary orientation,and also constraints placement of next joint layer in hierarchy.

  12. Trend – Markerless capture: Direct-to-3D models Performance Capture from Sparse Multi-view Video, SIGGRAPH 2010 Christian Theobalt, Stanford University http://www.youtube.com/watch?v=dTisU4dibSc&playnext=1&list=PLD31C3C36D294EEDB http://www.stanford.edu/group/biomotion/Markerless.html

  13. Trend – Monocular capture One camera, without depth, is under-constrained.However, the human body has fixed limb lengths and ratios.Use the body ratios as an additional constraint. Fabio Remondino, Andreas Roditakis Institute for Geodesy and Photogrammetry - ETH Zurich, Switzerland 3D Reconstruction of Human Skeleton from Single Images or Monocular Video Sequences2003, 25th Pattern Recognition Symposium

  14. Trend – Low Cost Systems Cheap hardware: Microsoft Kinect, Web cameras. Open source software: OpenKinect open kinect drivers libfreenect open kinect drivers OpenNI skeleton fitting FaceAPI facial trackingMain challenges: 1) Integration into existing frameworks, 2) Usually requires programming experience 3) Can be difficult to modify for research

  15. Software Trends

  16. Motion Graphs Motion Graph:A database of motion capture clips, connected to one another to represent transitionsbetween actions.Motion graphs can be represented by a finite state machine,a set of states with edges representing state transitions. Stand Run Jump

  17. Trend – Motion Graphs in Gaming Planning and Directing Motion Capture For GamesMelianthe Kines, Gamasutra. January 19, 2000http://www.gamasutra.com/view/feature/3420/planning_and_directing_motion_.php

  18. What are the advantages and disadvantages ofmotion graphs for gaming? Advantages1. Fast. Motion is simply played back from pre-recorded data.2. Interactive. Motion can be changed immediately by transitioning to a different state.3. Modular. Different motions can easily be swapped in.4. Extensible. More states can be added to the graph.Disadvantages1. Jump transitions between capture clips2. Motion may not match scene exactly. e.g. jump over chasm3. Cannot grasp objects accurately. No inverse kinematics.4. Cannot move in any direction5. Interruptions from outside forces not easily handled

  19. Trend – Motion Blendingin Gaming “In order to create streams of high-quality motion, current applications [games] assemble static clips of motion created with traditional animation techniques such as motion capture or keyframing. The assembly process requires making transitions between motions. These transitions may be difficult to create, such as a transition between a running clip and one where the character is lying down, or trivial, if the end of one clip is identical to the beginning of the next. In practice, simple techniques such as linear blends are capable of creating transitions in cases where the motions are similar.” Michael Gleicher, Hyun Joon Shin, Lucas Kovar, Andrew Jepsen Snap-Together Motion: Assembling Run-Time AnimationsInteractive 3D Graphics 2003

  20. Common solutions in Gaming:1. Jump transitions Linear blending between motion clips2. Motion may not match Blend with scene constraints scene exactly (extend jump over river)3. Cannot grasp objects Add inverse kinematics to arms in game characters. 4. Cannot move in any Add steering. direction Simple: re-orient, then play walk cycle Advanced: add IK to legs5. Interruptions from Use a rag-doll physics switch. outside forces When object hits.. Turn on physics, apply force.

  21. Trend – Motion Graphs How would you instruct a character to follow an arbitrary pathusing a set of pre-recorded captured motion? Lucas Kovar, Michael Gleicher, Frederic Pighin. Motion Graphs, SIGGRAPH 2002.

  22. Trend – Motion Graphs How do we make energy optimal motion based on several, arbitrary constraints?Uses motion capture data, butin arbitrary, non-acted scenarios. Alla Safonova Jessica K. Hodgins, Carnegie-Melon University. Construction and optimal search of interpolated motion graphs SIGGRAPH 2007

  23. INPUT Overview OUTPUT Post processing (cleaning) Physicalcapture /Haptics Jointdata Skinning Animation Secondary motion Markerdata BlendingOptimization Re-targeting Motioncapture Motion graphs (e.g Gaming) Sequencing Monoccularvideo Skeletonfitting Performancecapture Pointclouds Facialcapture 3D modelcapture

More Related