1 / 29

Choreographer-mathematician collaboration

Choreographer-mathematician collaboration. developing machine segmentation techniques for motion capture analysis of dance. Associate Professor Kim Vincs Dr Wai Kuan Yip Dr Vicky Mak Ms Kim Barbour. Motion capture & dance analysis.

goldy
Download Presentation

Choreographer-mathematician collaboration

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Choreographer-mathematician collaboration developing machine segmentation techniques for motion capture analysis of dance

  2. Associate Professor Kim Vincs • Dr Wai Kuan Yip • Dr Vicky Mak • Ms Kim Barbour

  3. Motion capture & dance analysis • Motion capture offers precision and true 3D tracking of dance movement • Recent development over the last 10 years • Relatively few motion capture labs with dance focus • A new gold standard for quantitative dance analysis?

  4. Challenges in motion capture dance analysis • Analysis needs to be culturally and artistically relevant (deLahunta) • Pattern recognition in dance crosses multiple frames of reference both between and within genres (?ref) • Approach to capture protocol, marker set design and feature extraction need to be appropriate to the needs of end-users, i.e. dance artists (Norman)

  5. ‘Capturing Dance’ project at Deakin Motion.Lab • Aims to isolate features of motion capture data that are artistically useful to choreographers • Features may be different for different dance genres and different dance artists • Will work with artists across 3 dance genres; contemporary dance, ballet and Australian indigenous dance over 3 years

  6. Project team Associate Professor Kim Vincs Dr Vick Mak Associate Professor Richard Smith Dr Wai Kuan Yip Ms Kim Barbour Mr Daniel Skovli Mr Peter Driver Ms Lisa Bolte Ms Carlee Mellow Ms Phoebe Robinson Ms Mee Young Yuk

  7. This paper • Analysis of a single, complex contemporary dance phrase • 20 repeats of the phrase by a single dancer • 64 marker-set adapted to capture spine-arm-foot relationships important to dance • Approaches to segmentation into dance-meaningful chunks for analysis • Necessity of choreographer-mathematician collaboration in developing appropriate analysis techniques

  8. Pattern recognition process

  9. Segmentation • Manual segmentation (human/expert) • Unsupervised machine segmentation based on kinematic assumptions • Supervised machine learning • Unsupervised machine learning

  10. Challenges in segmenting dance • Dance phrases tend to overlap • Different body segments need different levels of smoothing • Both large and small movements may be artistically significant • Different combinations of body segments may initiate new movements • Dance artists themselves don’t agree on how phrases are segmented (deLahunta 2005)

  11. Some previous approaches to segmenting dance movement • Inter-limb correlation (Nakata 2007) • Laban Movement Analysis, (Bouchard & Badler 2007) • Minimum velocity (Hachimura & Nakamura 2001) • Triple minima; force, kinetic energy & momentum (Kahol, Tripathi & Panchanathan 2004)

  12. Preliminary data – manual segmentation of a ‘tendu’ • Expert segmentation problematic as depends on weighting conflicting factors • Surprising variability within samples (5 samples, elite ballet dancer) • Highlights difference between dancers’ conceptual map of the movement and the detailed ‘motion capture view’

  13. Cortex file – tendu segment

  14. Comparison of segmentation methods

  15. Our approach • Primary question is what segmentation is artistically meaningful? • We used a collaborative, practice-based approach to develop segmentation ‘schema’ for the phrase • Choreographer, 2 dancers and 2 mathematicians developed definition of segments based on kinematics • Aimed to create an automated system based on the artistic schema

  16. Problems identified • Dancers used multiple conceptual frameworks, e.g. velocities, heights, correlations of body parts • Dancers’ framework is ‘procedural’ – segment definitions only make sense in relation to the preceding movement • Segmentation is phrase-specific and cannot be generalized

  17. Our schema

  18. Segmentation steps • Conversion from positional information to hierarchical (parent-child) translation and rotation format to make analysis scale & position invariant • Smoothing to remove noise – Butterworth 6Hz, moving average 20-50 window, Gaussian smoothing

  19. Segmentation steps • Polygonal approximation to estimate gradients of consecutive points to find local minima • 3 possible scenarios; • Negative gradient positive gradient • Zero-gradient positive gradient • Positive gradient zero gradient

  20. Examples of polygonal approximation Important to select the right threshold to estimate local minima Above: Maximum tolerable threshold 5*SD (just nice) Above Right: 15*SD (too loose) Right: 30*SD (way too loose)

  21. Local minimum segmentation Local minimum segmentation Root location Absolute rotational velocity Right elbow location Absolute rotational velocity

  22. Local minimum segmentation Example of compliant sample (#11) Example of non-compliant sample (#2)

  23. Local minimum segmentation Example of compliant sample (#17) Example of non-compliant sample (#12)

  24. Accuracy

  25. Comparison of methods

  26. Insert cortex file of Carlee Bitter

  27. Summary • Method works very well for movements that rely on body-shape change, but breaks down for large loco-motor and turning movements • Large variation in ‘dancer compliance’ • Variation or ‘dancer compliance’ does not correlate with dancer/choreographer ratings – variation in inherent in the style

  28. Conclusions • Dancer-mathematician collaboration useful for identifying recognition problems that need to be solved • Dance recognition process needs to be able to deal with a high level of intra-performer variability • Different types of movement may require different approaches to segmentation • Further techniques for optimizing machine segmentation (eliminating false segmentation points) needed to supplement dance-generated phrase descriptions

  29. References • Bouchard, D & Badler, N 2007, 'Semantic Segmentation of Motion Capture Using Laban Movement Analysis', in C. Pelachaud et al. (ed.), Intelligent Virtual Agents, Springer-Verlag, Berlin Heidelberg, vol. 4722, pp. 37-44. • DeLahunta, S & Barnard, P 2005, 'What's in a Phrase?', in J Birringer & J Fenger (eds), Tanz im Kopf: Jarbuch 15 der Gesellschaft für Tanzforschung, LIT Verlag, Münster. • Hachimura, K & Nakamura, M 2001, 'Method of generating coded description of human body motion from motion-captured data', Robot and Human Interactive Communication, 2001. Proceedings. 10th IEEE International Workshop on, pp. 122-7. • Kahol, K, Tripathi, P & Panchanathan, S 2004, 'Automated gesture segmentation from dance sequences', Proceedings, Sixth IEEE International Conference on Automatic Face and Gesture Recognition, 17-19 May pp. 883-8. • Nakata 2007, 'Temporal Segmentation and Recognition of Body Motion Data Based on Inter-Limb Correlation Analysis', Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2007), Oct. 29 2007-Nov. 2 2007 pp. 1383 - 8 • Norman, SJ 2006, 'Generic Versus Idiosycratic Expression in Live Performance Using Digital Tools', Performance Research, vol. 11, no. 4, pp. 23-9.

More Related