Skip this Video
Download Presentation
‘Joining in’ spontaneous conversation and improvisational music-making

Loading in 2 Seconds...

play fullscreen
1 / 21

‘Joining in’ spontaneous conversation and improvisational music-making - PowerPoint PPT Presentation

  • Uploaded on

‘Joining in’ spontaneous conversation and improvisational music-making. Sarah Hawkins 1 , Richard Ogden 2 , Ian Cross 1. 1 Centre for Music & Science University of Cambridge. 2 Dept. of Language and Linguistic Science, University of York.

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about ' ‘Joining in’ spontaneous conversation and improvisational music-making' - vevina

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

‘Joining in’

spontaneous conversation and

improvisational music-making

Sarah Hawkins1, Richard Ogden2, Ian Cross1

1Centre for Music & ScienceUniversity of Cambridge

2Dept. of Language and Linguistic Science,University of York

{sh110, ic108} [email protected]

Language, Music and Interaction, Philological Society,QMUL, Nov. 2012

what processes underpin interaction in music making and speech
What processes underpin interaction in music-making and speech?

with unfamiliar instruments / objects

Spontaneous interaction:conversation and musical improvisation

  • A controlled environment in which to elicit (relatively!) natural joint (inter)action in unrehearsed
    • talking
    • music-making
    • non-musical play
  • Initial observations and hypotheses
speaker listener neural coupling underlies successful communication
Speaker–listener neural coupling underlies successful communication

Stephens, Silbert, and Hasson (2010) PNAS 107(32) 14425-14430

Neural: Charles Schroeder group; Edward Large group e.g.

Schroeder et al. (2008)TICS 12(3), 106-113

Fujioka, Trainor, Large & Ross (2012) J. Neurosci. 32(5), 1791-1802

Sociophonetics: Garrod & Pickering (2004) TICS 8(1), 8-11

Stephens, Silbert, and Hasson (2010)

fMRI as speaker tells a story; another P listens.

a speaker’s brain activity is spatially and temporally coupled with the listener’s activity

but only when the listener understands the speaker

most correlated patterns in listener are delayed relative to speaker’s; but some are anticipatory

greater anticipatory speaker–listener coupling→ greater understanding (independent comp. measure)


Hypothesis: if coupling of neural oscillations underpins successful communication, then we should find:

similar processes in music and speech differing only by

demands of the medium

function of the particular interaction

our aim: find a set of comparable tasks & measures in music-making and conversing

CA framework: alignment and disalignment

alignment and disalignment in talk
Alignment and disalignment in talk

hasn’t he got nice eyes?

In music, we can expect similar patterns (Turino)

Superordinate, multidimensional terms

We don’t expect binary classification every time


demographic questions; initial consent



≥ 5

  • How did you get here?
  • What do you think of the room?
  • 9/11; Princess Diana’s death…
  • important event you shared
  • xylophone, kalimba
  • drums, claves
  • card houses
  • tallest tower: blocks
  • market stall: playdough

at least 2


non-musical play


musical play

no shakers!



detailed musical questions; final consent; £8



5 dyads,

  • 3 musician pairs
  • 2 non-musician
  • various tasks

“Experiment 1”

6 dyads,

  • 3 musician pairs
  • 3non-musician
  • 2-3 prescribed tasks


  • friends
  • same-sex
  • native speakers of English
  • 18-30
  • university educated
  • both musicians,or bothnon-musicians
starting set up familiarisation
Starting set-up: familiarisation
  • card house
  • tallest tower of blocks
  • playdough market stall
  • Sri Lankan drums (2 types, one not shown)
  • claves
  • circular xylophone
  • kalimba(mbira)


  • ≥ 10 years’ formal training
  • currently actively engaged in music at least once a month
  • ≤ 7 years’ formal training
  • no active music-making in past 4 years


  • 4 video cameras
  • overhead omni mike
  • stereo pair (music)
  • 2 close-talking mikes
  • 4 video cameras
  • overhead omni mike
  • stereo pair (music)
  • 2 close-talking mikes
looking for co ocurrences
Looking for co-ocurrences
  • Focus: alignment and disalignment
    • rhythmically: entrainment and failure to entrain
  • Body movement is well established as marking important events (beats) in both speech and music
  • What happens when such beats carry across the two modalities?
  • A single framework for labelling events
  • tracking beats in speech and music: currently, Cummins
example s of what we found placeholder
Example(s) of what we found placeholder
  • Alignment “magic”: E1_MF1 29:43-29:51
    • beat continuation across modalities, and between participants
    • perfect coordination
    • unscripted (music not ‘counted in’)
  • Disalignment: E1_MF1 26:31-26:47
alignment beat continuation across modalities and between participants
Alignment: beat continuation across modalities, and between participants
  • The criterion is (for alignment): there is speech before or after where
  • the music changes, and the music \'works\'.
  • The questions are: to what extent is the speech and music beat coordinated?
  • and how does this compare when there is breakdown or less successful interaction?
  • what happens with body movements and eye gaze?
tentative hypotheses to be completed
Tentative Hypotheses (to be completed)
  • look at the effector:
    • hands when playing
    • faces when speaking
    • presumably faces when singing together
  • look at times of uncertainty….
next steps
Next steps
  • Quantify: proportion of positive instances of categories
  • Theory: which? is there only one?
    • Why? (Causes) do people entrain willy nilly or element of prediction from one or other
    • if we can’t tell bottom up from top down, and there’s not a clear listener vs clear talker, what are we dealing with – the holy spirit?
  • Theory: top down and bottom expectancies mesh: me, Narmour, Pearce/Wiggins….we need to work actively to get this working for a general theory – and using sp and music as our test bed seems an exciting way forward. Form a working group???

Satinder Gill

Thanks to!

David Greatrex

Rein OveSikveland

Daniel Halford

Hannah Leach

Newton Trust, U. Cambridge; BA/Leverhulme Foundation Small Grant

so why do music and speech seem fairly different but we feel they are the same
So why do music and speech seem fairly different, but we feel they are the same?
  • music and speech typically differ in the relative balance accorded to conveying phatic vs referential meaning—but this is a very loose difference
    • functions of both modalities dictate what is important, and where we should look for guiding principles
    • languages will differ – as will musics (structure)
  • these different balances in large part dictate the greater predictability of rhythm in music than in speech
some things that music and speech share
some things that music and speech share
  • wide range of rates, affected by many factors
  • phrase-final lengthening
  • predictable tonal endings: cadence, nuclear tone
  • internal/local rate change: rubato, asides, emotion, floor holding
  • deviation from rhythmicity indicates
    • emotion
    • phrasing
    • in speech, the demands of the actual words used
  • regular rhythms are constructed, in both speech and music
    • consistent with most other perceptual approaches
summary of a video clip not included
Summary of a video clip (not included)
  • the background speaker, L, maintains a beat of about 460 ms in speaking; and appears to lead the transition into music:
  • though R talks more, and is talking very casually, she seems to entrain to L’s speaking beat
  • they start playing about on ‘the current beat’: c. 800-900 ms
  • gradually increase tempo to c. 700 ms
  • look at the effector:
    • hands when playing
    • faces when speaking
    • presumably faces when singing together