1 / 16

Functional Data Analysis of Continuous Judgments in Music Cognition

Functional Data Analysis of Continuous Judgments in Music Cognition. Gesture in Musical Performance. What role do a musician’s gestures play in a performance? Do they convey emotion? Do gestures convey the same things as does the music? Popular study in field of music psychology

lorna
Download Presentation

Functional Data Analysis of Continuous Judgments in Music Cognition

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Functional Data Analysis of Continuous Judgments in Music Cognition

  2. Gesture in Musical Performance • What role do a musician’s gestures play in a performance? • Do they convey emotion? • Do gestures convey the same things as does the music? • Popular study in field of music psychology • Data on listener’s emotions are collected in real-time • With a real-time Optotrak slider, which • Measures location of the slider 10 times per second. Continuous Judgments of Music

  3. The Tension & Gesture Experiment • Musical performance recorded on video • Stravinsky’s 2nd Piece for Solo Clarinet • 30 musically-trained participants either • Watched & Listened to the recording (Natural) • Watched the silent recording (Video Only), or • Listened to the recording (Audio Only). • And reported their continuous “level of tension [emotion]”.

  4. The Data • Vectors of length 800 • 10 “Audio Only,” 10 “Video Only,” 10 “Audio + Video” • Scaled to [0,1] interval

  5. The Functional Objects • 150 order 6 B-splines, using FDA software in Matlab. • Then smoothed.

  6. Functional Principal Components Analysis • 25 s – 65 s: crucial separation • Effect of amplification / attenuation: how strong are the changes of emotion?

  7. Functional Linear Model Emotion(t) = µ(t) + β0(t){AudRemoved} + β1(t){VidRemoved} + ε(t)

  8. Functional Linear Model Emotion(t) = µ(t) + β0(t){AudRemoved} + β1(t){VidRemoved} + ε(t)

  9. Derivatives describe music dynamics: Tension • These judgments are really measures of musical emotion. • ‘Tension/Resolution’ is rate of change of emotion (velocity). • When emotion is rapidly increasing, music has strong tension. • When emotion is rapidly decreasing, music has strong resolution.

  10. Derivatives describe music dynamics: Force/Release • ‘Force/Release’ is rate of change of tension (acceleration). • When tension is rapidly increasing, we feel a musical force. • When tension is rapidly decreasing, we feel a musical release. • When both derivatives are near zero, music is inert (the “new age/massage music” effect).

  11. Phase-Plane Plots • Can examine dynamics with plot of acceleration vs velocity. • Purely harmonic behavior gives a circle. • The larger the radius, the more musical energy transfer.

  12. 25 – 33 s: • AUDIO: high volume, note density, and pitch; end of musical phrase • tension then strong resolution; big energy transfer • VIDEO: routine gestures; then dramatic flourish • low tension and small resolution; moderate pull with flourish; small energy transfer 33 – 65 s: • AUDIO: decrease in volume (mezzo forte to pianissimo), note density, and pitch • continued resolution; push from new phrase; back to inertness • VIDEO: eyebrow and body movements • push from new phrase, moderate tension from movements

  13. What have we learned? • New applications • PCA as exploration • Derivatives have physical (and scientific) meaning • Phase-Plane plots highlight relationships

More Related