1 / 85

Motion Capture from Body-Mounted Cameras

Motion Capture from Body-Mounted Cameras. Takaaki Shiratori Hyun Soo Park Leonid Sigal Yaser Sheikh Jessica K. Hodgins. Takaaki Shiratori Postdoctoral Associate Disney Research, Pittsburgh. Yaser Sheikh Assistant Professor Robotics Institute School of Computer Science CMU.

olympe
Download Presentation

Motion Capture from Body-Mounted Cameras

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Motion Capture from Body-Mounted Cameras TakaakiShiratori Hyun Soo Park Leonid Sigal Yaser Sheikh Jessica K. Hodgins

  2. Takaaki ShiratoriPostdoctoral AssociateDisney Research, Pittsburgh Yaser Sheikh Assistant ProfessorRobotics InstituteSchool of Computer ScienceCMU Jessica K. Hodgins Professor Computer Science and RoboticsCMU Hyun Soo Park Ph.D. studentMechanical Engineering, CMU Leonid Sigal Research Scientist Disney Research, Pittsburgh

  3. Related work • Optical motion capture system: Vicon • Marker-less method: Organic Motion • Wearable electro-mechanical system: Gypsy • Inertial measurement unit: Xsens

  4. Contributions: • The first to employ camera sensors for Mocap; • Outdoor setting; • Not intrusive and easy to wear; • Cheaper.

  5. Hardware setup • 16 sport action cameras: HD Hero

  6. Pipeline:

  7. Reconstructing human motion: • Initializing camera poses using SFM • 3D reconstruction of reference images; SIFT: key point; FLANN: match; RANSAC : fundamental matrix; Direct Linear Transform: 3D feature points; SBA: bundle adjustment; PnP: reconstruct camera pose.

  8. Reconstructing human motion: • Initializing camera poses using SFM • Absolute camera registration; • Relative camera registration; • Homographies for unregistered camera.

  9. Reconstructing human motion: • Mapping cameras to a skeleton O’BRIEN, J. F., BODENHEIMER, R. E., BROSTOW, G. J., AND HODGINS, J. K. 2000. Automatic joint parameter estimation from magnetic motion capture data. In Proc. Graphics Interface, 53–60.

  10. Reconstructing human motion: • Estimating body poses with global optimization: Levenberg-Marquardt method

  11. Experiments:

  12. Experiments:

  13. Discussion: • Motion blur, automatic white balancing rollong shutter effect, occlusion; • Computation cost.

  14. Motion Reconstruction Using Sparse Accelerometer Data Jochen Tautges , Arno Zinke , Björn Krüger, Jan Baumann, Andreas Weber Thomas Helten, Meinard Müller, Hans-Peter Seidel, Bernd Eberhardt.

  15. Diplom-Informatiker Jochen Tautges Researcher WG Weber Diplom-Informatiker Jan Baumann Researcher WG Weber Professor Dr. Andreas Weber Head of Multimedia, Simulation and Virtual Reality Group Dr. Arno Zinke Researcher WG Weber Diplom-Informatiker Björn Krüger Researcher WG Weber

  16. Thomas Helten PHD student MPI, Germany Meinard Müller Senior Researcher MPI, Germany Prof. Dr. Hans-Peter Seidel Director, MPI Affiliated Professor, Saarland University Prof. Dr. Bernhard Eberhardt Stuttgart Media University (HdM) Germany

  17. Introdution(video): Low-dimensional control signals Xsens MTx Data HDM05 Low-dimensional control signals

  18. Contributions: • Online framework for full body mocap based on sparse accelerometer input; • Flexibly deal with temporal and spatial variations; • Online motion retrieval .-. OLNG (EG 2010) • Generality.

  19. Pipeline:

  20. Pipeline: • Control input and knowledge base;

  21. Pipeline: • Online lazy neighborhood graph

  22. Pipeline: • Motion reconstruction

  23. Pipeline: • Motion reconstruction

  24. Pipeline: • Energy minimization The lsqnonlinfunction of MATLAB

  25. Limitation: • Database; • Occasionally jumps between poses may occur; • Relative motion reconstruction: root motion is • only approximate.

  26. Future work: Build device for various tasks requiring motion puppetry; Code optimization; Other animation tasks;

  27. Physically-Valid Statistical Models for Human Motion Generation Xiaolin Wei Jianyuan Min JinxiangChai Texas A&M University

  28. Xiaolin WeiGraduate StudentTexas A&M University Jianyuan Min Jinxiang Chai Assistant Professor Computer Science and Engineering Texas A&M University

  29. Introduction(video):

  30. Related work • Statistical motion modeling • Physics-based motion modeling

  31. Contributions: • Combining statistical motion prior and physical constraints for motion generations • Nonlinear probabilistic force field.

  32. Pipeline: • Physics-based dynamics models • Newtonian dynamics • Contact mechanics: Coulomb’s friction model

  33. Pipeline: • Force field function modeling

  34. Pipeline: • Force field function modeling • Motion data preprocessing: Physics-based trajectory optimization WITKIN, A., AND KASS, M. 1988. Spacetime Constraints. In Proceedings of ACM SIGGRAPH 1998. 159–168. COHEN, M. F. 1992. Interactive Spacetime Control for Animation. In Proceedings of ACM SIGGRAPH 1992. 293–302.

  35. Pipeline: • Force field function modeling • Gaussian Processes (GP) modeling of force fields. QUINONERO-CANDELA, J., AND RASMUSSEN, C. E. 2005. A unifying view of sparse approximate gaussian process regression. Journal of Machine Learning Research. 6:1935–1959.

  36. Pipeline: • Human motion modeling and synthesis • Combining physics with statistical prior

  37. Pipeline: • Human motion modeling and synthesis • Constraint-based motion synthesis CHAI, J., AND HODGINS, J. 2007. Constraint-based Motion Optimization Using A Statistical Dynamic Model. In ACM Transactions on Graphics. 26(3): Article No.8. • Optimization: sequential quadratic programming

  38. Experiments:

  39. Limitations: GP modeling’s computational demands; Data-dependence; Unable to handle arbitrary external forces; Force field is simple; Local minima in the optimization; User interface.

  40. Illumination Decomposition for Material Recoloringwith Consistent Interreflections Robert Carroll Ravi Ramamoorthi ManeeshAgrawala University of California, Berkeley

  41. Robert CarrollGraduate StudentUniversity of California, BerkeleyEECS Department Ravi Ramamoorthi Associate Professor University of California, Berkeley Computer Graphics and Computer Vision Groups EECS Department Maneesh Agrawala Associate ProfessorUniversity of California, Berkeley Visualization Lab EECS Department

  42. Related work • Intrinsic images BARROW, H., AND TENENBAUM, J. 1978. Recovering intrinsic scene characteristics from images. Computer Vision Systems 27, 9, 3–26. BOUSSEAU, A., PARIS, S., AND DURAND, F. 2009. User-assisted intrinsic images. ACM Trans. Graph. 28, 130:1–130:10.

  43. Related work • Illumination decomposition SEITZ, S., MATSUSHITA, Y., AND KUTULAKOS, K. 2005. A theory of inverse light transport. In ICCV ’05, vol. 2, 1440–1447. BAI, J., CHANDRAKER, M., NG, T.-T., AND RAMAMOORTHI, R. 2010. A dual theory of inverse and forward light transport. In ECCV ’10, 294–307. • Material and lighting editing OH, B. M., CHEN, M., DORSEY, J., AND DURAND, F. 2001. Image-based modeling and photo editing. In Proc. SIGGRAPH, 433–442. FINLAYSON, G., HORDLEY, S., LU, C., AND DREW, M. 2006. On the removal of shadows from images. IEEE Trans. PAMI 28, 1, 59 –68. SHOR, Y., AND LISCHINSKI, D. 2008. The shadow meets the mask: Pyramid-based shadow removal. Comput. Graph. Forum 27, 2, 577–586.

  44. Contributions: • The only one to address diffuse reflection; • Traditional image editing tools can not handle;

  45. Assumptions: • Illumination is locally a mix of a small set of independent colors; • There is only a single direct light color; • There is only a single bounce of indirect illumination; • The scene is Lambertian.

  46. Pipeline: • White balance • Computing intrinsic image

  47. Pipeline: • Illumination decomposition

  48. Pipeline: • Energy Minimization Data Fidelity: Non-negativity:

More Related