Auditory Scene Analysis

1. Auditory Scene Analysis

2. Sequential vs. Simultaneous Organisation • Sequential grouping involves connecting components over time to form streams • Simultaneous grouping involves grouping the components present at a given moment in time to create one or more perceived sounds • This lecture - how these two types of grouping compete with one another in the formation of auditory streams

3. Simultaneous vs. Sequential Organisation • Organisation at a point in time may conflict with organisation over time • Competition to establish the identity of sounds • The relationship between two simultaneous tones can be affected by the relationship between one of the tones and other previous and/or successive tones • Vertical and horizontal relationships seem to compete with each other

4. Simultaneous vs. Sequential Organisation • The relationships are influenced by: • The differences (e.g. in pitch) between the simultaneous components and the sequential components • The degree of synchrony of the simultaneous components • The number of components involved

5. Simultaneous vs. Sequential Organisation • Asa demo 25 Capturing a tonal component out of a mixture: Part 1 • 3 pure tones: A, B and C. A is played first followed by B and C play simultaneously • A could form one stream (A- A…) • The simultaneous tones B and C could form another (BC- BC…) • OR • A and B could form a stream of pure tones (A B A B…)

6. Simultaneous vs. Sequential Organisation • An important determinant of which of these groupings will emerge in perception is the frequency separation between the tones A and B. • Large difference in frequency, A segregates from B and B fuses with C • If A and B are close in frequency they group toegether to form a pure tone stream breaking the tendency of B to fuse with C • When A captures B – sequential integration is stronger than simultaneous integration

7. Sequential vs. Simultaneous Organisation • Asa demo 26 – Capturing a tonal component out of a mixture: Part 2 • Demonstrate that the fusion of components depends on their synchronisation • Pure tones A, BC as previous • Frequency separation between A and B held constant • Vary asychrony of BC • When B and C are synchronous they fuse into a single sound – A forms a separate stream

8. Sequential vs. Simultaneous Organisation • B groups with A when B is asynchronous with C • Asa demo 27 – Competition of sequential and simultaneous grouping • Demonstrates that if another component is added that is close to one of the simultaneous components – may also work against their fusion • 3 tone cycle A, BC as before • Tone D that is close in frequency to C is added to the cycle – A, BC, D

9. Sequential vs. Simultaneous Organisation • The sequential grouping of C with D competes and weakens the fusion of C with B • C groups with D forming a C-D pure tone stream – difficult to hear C as part of the complex BC – • weakening of BC fusion has the effect of releasing B to group more strongly with A in an A-B pure tone stream • C-D grouping affects A-B grouping, even though neither A nor B themselves were changed

10. Sequential vs. Simultaneous Organisation • Asa demo 33 – Creation of a high-pitched residual by capturing some harmonics from a complex tone • Tone B, with a rich spectrum, alternates with tone A (captor) that contains only a few lower harmonics • Under some conditions tone B will split - tone A will capture out of B the lower harmonics it shares with it - a perceptual stream of low sound • The harmonics that are unique to B will be left behind - heard as a separate higher sounding tone

11. Sequential vs. Simultaneous Organisation • The strength of the decomposition of B depends on the length of the silent gap between A and B • 1st hear tone B • 2nd – A B alternate with a 200 msec silent gap • 3rd – A B alternate with no silent gap • Capturing of B harmonics by A is best when there is no silent gap between A and B

12. Sequential vs. simultaneous Organisation • Asa demo 37 – Changing a vowel’s quality by capturing a harmonic • hear a synthetic “i” vowel whose third harmonic (a pure tone) is kept on all the time while the rest of the vowel goes on and off • Idea: because the pure tone does not turn on and off with the vowel, it does not integrate with it perceptually, and the vowel sounds as if all or some of the energy of its third harmonic is missing

13. Sequential vs. Simultaneous Organisation • Pure tone – sequentially capturing its counterpart (the third harmonic) from the vowel. • The “i” vowel minus its third harmonic sounds like a different vowel – “e” or “en” (as in bed or bend) • 1st - hear 4 cycles of the “i” sound, then 4 of the “en” sound • Third harmonic present all the time, “i” vowel going on and off 8 times, followed by a comparison vowel.

14. Sequential vs. Simultaneous Organisation • when the “i” vowel comes on, some of the energy of the third harmonic does not combine with the rest of the vowel – change in the vowel’s quality - “en” • The quality of the comparison vowel gradually changes from “i” to “en” during the course of the trials

15. Memory • All musical activities involve memory • E.g. sight reading involves the memory of the relationship between specific symbols and pitches and the set of body and finger movements needed to execute them • Most knowledge about music has to be acquired through learning – can be formal or informal • Memory – a capacity that can be improved – not a fixed quantity

16. Memory • Memory – a practical requirement – such as in the performance of complex music without a score • memory often involves the ability to make sense of incoming material in terms of previously learnt information - when material cannot be assimilated into familiar structures, memory performance declines • Importance of familiar structures for musical memory

17. Memory • Studies have indicated that listeners can discriminate better between two short consecutive pitch sequences when these sequences are drawn from familiar tonal materials than from unfamiliar materials • Lifelong exposure to tonal music – tuning of our cognitive systems to the structures and regularities in this music • Distinguish between two types of memory:

18. Memory • Memory for specific pieces of music (episodic, veridical) • Memory for norms and prototypes which may be a shared attribute of many pieces of music (semantic, schematic) • If a familiar piece of music is interrupted before its close, episodic memory – reconstruct the actual continuation, whereas schematic memory – allows a guess of a likely continuation

19. Memory • A good schematic memory is required for episodic memory – easier to memorise a piece of music once the structural interrelationships within the music are understood • Memory has two basic forms – recognition and recall • Recognition – listening • Recall - performance

20. Recognition • Recognition – a process that operates in perception to match incoming information to previously stored information • One form of recognition is in the perception of similarity – something heard is perceived as close to or the same as something heard before • The ability to recognise similarity between structures in a piece – crucial in the perception of form within a piece of music

21. Recognition • Recognition of themes under various transformations – listeners will recognise a melody, with its interval relationships preserved, as the same irrespective of its starting pitch – importance of pitch contour • evidence that even trained musicians find it hard to keep the starting-key of a heard composition in memory if it modulates several times

22. Recognition • Some transformations make recogniton more difficult – e.g. changing the rhythm of a melody can significantly disrupt recognition even though the pitch pattern may be unchanged • Recognition ability – not static - can improve with experience and training • Second form of recognition – identification or naming – retrieval of a verbal label e.g. for a note, chord, or the musical piece.

23. Recognition • Identification – involve recognition of similarity – although recognition of a similarity may not lead to identification • e.g. recognise a familiar chord but unable to name / identify it

24. Recall • Recall – the reproduction of a previously experienced sequence • Requires more mental resources than recognition - a listener will generally recognise much more than can be recalled - recall may require some form of cue to trigger it. • Cued recall - performing learnt music from a score - an experienced performer will rarely look at every note.

25. Recall • Can be regarded as either an unintended by-product of other activities or as the result of deliberate memorizing efforts. • Most everyday examples of musical recall are involuntary and unintended results of other mental processes. – e.g. due to repeated hearings of radio / tv themes and jingles – reproducible without any particular effort • However, recall of more complex performance music – through a process of deliberate and conscious memorization

26. Recall • Characteristics of expert memorizers: • Long time experience in memorizing • Possess the ability to represent the material to be learnt in terms of patterns and structures that have rich interconnections with each other and with previously learnt material. • develop multiple interlocking levels of representation (e.g. visual, auditory, motor movements), such that if any one of them is temporarily lost it can be re-cued from another level.

27. Recall • Representations are flexible – e.g. an experienced performer could transpose a piece learnt in one key to another with not much extra effort