Functional data analysis of continuous judgments in music cognition
This presentation is the property of its rightful owner.
Sponsored Links
1 / 16

Functional Data Analysis of Continuous Judgments in Music Cognition PowerPoint PPT Presentation


  • 80 Views
  • Uploaded on
  • Presentation posted in: General

Functional Data Analysis of Continuous Judgments in Music Cognition. Gesture in Musical Performance. What role do a musician’s gestures play in a performance? Do they convey emotion? Do gestures convey the same things as does the music? Popular study in field of music psychology

Download Presentation

Functional Data Analysis of Continuous Judgments in Music Cognition

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Functional data analysis of continuous judgments in music cognition

Functional Data Analysis of Continuous Judgments in Music Cognition


Gesture in musical performance

Gesture in Musical Performance

  • What role do a musician’s gestures play in a performance?

  • Do they convey emotion?

  • Do gestures convey the same things as does the music?

  • Popular study in field of music psychology

  • Data on listener’s emotions are collected in real-time

    • With a real-time Optotrak slider, which

    • Measures location of the slider 10 times per second.

Continuous Judgments of Music


The tension gesture experiment

The Tension & Gesture Experiment

  • Musical performance recorded on video

    • Stravinsky’s 2nd Piece for Solo Clarinet

  • 30 musically-trained participants either

    • Watched & Listened to the recording (Natural)

    • Watched the silent recording (Video Only), or

    • Listened to the recording (Audio Only).

  • And reported their continuous “level of tension [emotion]”.


The data

The Data

  • Vectors of length 800

    • 10 “Audio Only,” 10 “Video Only,” 10 “Audio + Video”

  • Scaled to [0,1] interval


The functional objects

The Functional Objects

  • 150 order 6 B-splines, using FDA software in Matlab.

  • Then smoothed.


Functional principal components analysis

Functional Principal Components Analysis

  • 25 s – 65 s: crucial separation

  • Effect of amplification / attenuation: how strong are the changes of emotion?


Functional data analysis of continuous judgments in music cognition

Functional Linear Model

Emotion(t) = µ(t) + β0(t){AudRemoved} + β1(t){VidRemoved} + ε(t)


Functional linear model

Functional Linear Model

Emotion(t) = µ(t) + β0(t){AudRemoved} + β1(t){VidRemoved} + ε(t)


Derivatives describe music dynamics tension

Derivatives describe music dynamics: Tension

  • These judgments are really measures of musical emotion.

  • ‘Tension/Resolution’ is rate of change of emotion (velocity).

    • When emotion is rapidly increasing, music has strong tension.

    • When emotion is rapidly decreasing, music has strong resolution.


Derivatives describe music dynamics force release

Derivatives describe music dynamics: Force/Release

  • ‘Force/Release’ is rate of change of tension (acceleration).

    • When tension is rapidly increasing, we feel a musical force.

    • When tension is rapidly decreasing, we feel a musical release.

  • When both derivatives are near zero, music is inert (the “new age/massage music” effect).


Phase plane plots

Phase-Plane Plots

  • Can examine dynamics with plot of acceleration vs velocity.

  • Purely harmonic behavior gives a circle.

  • The larger the radius, the more musical energy transfer.


Functional data analysis of continuous judgments in music cognition

25 – 33 s:

  • AUDIO: high volume, note density, and pitch; end of musical phrase

    • tension then strong resolution; big energy transfer

  • VIDEO: routine gestures; then dramatic flourish

    • low tension and small resolution; moderate pull with flourish; small energy transfer

      33 – 65 s:

  • AUDIO: decrease in volume (mezzo forte to pianissimo), note density, and pitch

    • continued resolution; push from new phrase; back to inertness

  • VIDEO: eyebrow and body movements

    • push from new phrase, moderate tension from movements


What have we learned

What have we learned?

  • New applications

  • PCA as exploration

  • Derivatives have physical (and scientific) meaning

  • Phase-Plane plots highlight relationships


  • Login