Music and Memory

1. Music and Memory Perfecto Herrera Music Perception and Cognition

2. Multi-storage view • Echoic Memory • Short-term or Working Memory • Long-term Memory (or memories: declarative, non-declarative, episodic, semantic, procedural, etc.)

3. Functional view Encoding operations • Echoic Memory: • feature extraction & perceptual binding • STM: • segmentation & chunking <-> Decay of traces <-> Interference of traces • LTM: • explicit learning rehearsal • mnemonics • implicit learning (learning by exposure) • consolidation (hypocampus, sleep effect) <-> Interference and Reconstruction

4. Retrieval operations • Recognition: acknowledgement that a pattern in STM is stored in LTM (did you hear this song before? Does it sound familiar?) • Recall or Recollection: activation of a LTM encoded pattern by a “diffuse” effort of will (what is the title of this song? Who was the composer?) • Reminding: activation of a LTM encoded pattern by a pattern in STM (which memories are activated by listening to this song?) • See http://www.magicalmemorytour.com/

5. Top-down (knowledge based) influences

6. STM-LTM

7. Echoic or Sensory Memory • Trace that stimuli leave just after the transduction • Probably it is accounted by the connections up to the thalamic centres • It accounts for the basic feature extraction (pitch, intensity, onset, noise, harmonic pattern, spatial position…) • Extracted features are “bounded” (what goes with what, e. g. a given partial with another one and with a fundamental coming all of them from the same direction) • The visual counterpart (iconic memory) is easier to experience (for a while look at a highly contrasted window scene, then close your eyes and keep on looking using your mind’s eye). • Active up to 4 seconds, very fast degradation • Sensory coding (no conceptual or categorical coding is available)

8. Short-term Memory • Temporal storage for helping the permanent encoding (aka “Working Memory”) • It is the grounding of our sensation of “present”, a shifting focus of awareness selects what/how much is it processed • Has limitations: capacity (7+/-2 items) and time (around 10-20 seconds) • Chunking and rehearsal help to overcome these limitations • The type of processing done to the items affect the recoverability of them (i.e. “deep” or semantic processing helps to robustly retrieve them whereas “surface” processing does not) • Has “sensory-specific” sub-blocks (for dealing with visual-spatial, phonological, pitch information –parallel processing of them) • Attentional mechanisms “modulate” what is stored in short-term and which type of processing is devoted to that

9. (Long-term) Memory systems of the brain(from Squire, 2004)

10. Memory structures

11. Memory anatomical structures Forget about the names, just look at the complex interaction pattern!!!

12. Declarative-Non Declarative Memories • Declarative (~ explicit memory): • Consciously available • Fast learning, even “single-trial” • Non-Declarative (~ implicit memory): • Contents consciously “unavailable” or interferring its development when trying to make them conscious (playing an instrument and thinking on which movements are required to keep on playing -> Disaster) • Slow learning • Automatic once learning has happened (“compiled knowledge”) • Often modality-specific

13. Declarative memory • What is this note name? What’s the name of the piece? • Episodic memory: memory for specific event in time (e.g. “that” performance of Don Giovanni that we saw in Vienna) ~ autobiographical memory • Semantic memory: memory about things of the world, common-sense (e.g., Don Giovanni is an Opera by Mozart) • Both are associative and distributed • Both are reconstructive (“extra info”, not encoded in the original event, can be generated at retrieval time)

14. Non-declarative, Procedural memory • How does the melody go? How should I play this phrase with this instrument? • Procedural knowledge is slowly acquired, usually “by doing”, it is very long-lasting • Usually related to “sequences” of sensory/motor representations (e.g., finger movements, notes, etc.) -> Grammars (they define “correct sequences”) • Difficult to retrieve verbally, often causing interference • After some amount of practice knowledge is “compiled” into big chunks that are not accessible to introspection (how the chess expert knows the best movement? -> nothing to do with Deep Blue!!!

15. Operations in STM: Segmenting The continuous musical texture is broken into shorter sequences using “segmentation cues” -> “Closure” and “Change” detection • Pauses, silences, stretching of notes • Instrument changes • Cadences • Accents and other metrical elements • Tendency changes (up-down melody, long-short notes, etc…)

16. Operations in STM: Grouping

17. Operations in STM: Grouping • Lerdahl & Jackendoff “grouping well-formedness rules” and “grouping preference rules” (Generative Theory of Tonal Music) • Well-formedness rules define (abstract) structural descriptions that can be derived from the surface structure (a series of hypotheses for organizing the musical structure) • Grouping preference rules define the conditions that allow a listener to choose the preferred interpretation of the structure from all of the possible ones that conform to the well-formedness rules (connected to real perceptual events)

18. Operations in STM: Chunking • Encoding or consolidation of small groups of elements into a compact larger or more abstract element, which is then encoded, recognized or remembered • Example: Memorize this series of letters: • F-B-I-C-I-A-U-S-A-C-N-N-I-B-M • You create chunks of 3 letters that “go togheter” in an existing memory • Musical scales and chords can be used as elements that facilitate chunking (you usually do not code the individual notes) • C-E-G-B -> C major 7 • Cadences • Similarity and relatedness between the sequential elements facilitates chunking; not only a “top-down” (knowledge-based) effect

19. Pitch-specific short-term memory Deutsch experiments

20. In which condition should we expect more errors? Pitch-specific short-term memory What if the interference was like “A, B, D, F, A, C” ?

21. Serial position effect Long-term memory effect, if you do not allow enough time to encode between each item, then the segment is flat Recency Primacy Probability of recall Short-term memory effect, if you wait 30” for retrieving the items, the segment turns flat Position

22. The “dynamics” of information storage Substitute “visual” by “auditory”

23. Interlude (I) “The problem of perception is initially a problem of taxonomy in which the individual animal must “classify” the things of its world (…) The internal taxonomy of perception is adaptive but is not necessarily veridical in the sense that it is concordant with the descriptions of physics” Edelman, Neural Darwinism, p. 26

24. Interlude (II) The animals are divided into: (a) belonging to the emperor, (b) embalmed, (c) tame, (d) sucking pigs, (e) sirens, (f) fabulous, (g) stray dogs, (h) included in the present classification, (i) frenzied, (j) innumerable, (k) drawn with a very fine camelhair brush, (l) et cetera, (m) having just broken the water pitcher, (n) that from along way off look like flies.Jorge Luis Borges, The analytical language of John Wilkins

25. Interlude (II) "It was not only difficult for him to understand that the generic term dog embraced so many unlike specimens of differing sizes and different forms; he was disturbed by the fact that a dog at three-fourteen (seen in profile) should have the same name as the dog at three-fifteen (seen from the front)". Jorge Luis Borges Funes the memorious

26. Categorization • The act of assigning a single response/reaction to a series of different inputs from the external world -> A device to cope with the physical variability of our “analog” world • It cannot be a process of one small portion of the nervous system -> Distributed memory (linking sensory, motor, planning, emotion and reward subsystems) • Does it need “words”? • Animals show stimulus generalization, which requires some form of categorization • Pre-linguistic babies show stimulus generalization too • Language may act as a second code, in parallel to perceptual codes, boosting learning by providing a “constant” feature vector for objects that perceptually are not totally identical • Maybe useful distinction between Perceptual learning (automatic) and Conceptual learning (linked to language, will, etc.)

27. Knowledge structures: Categories • A group of nonidentical objects or events that an individual treats as equivalent • Equivalent could mean: • Their internal representations are similar or close • They generate similar behaviours (avoidance / approach) • They generate similar internal states (e.g. emotions) (pleasure / pain) • Categorization reduces the overwhelming complexity of the natural world (frequencies -> notes, timbres -> instruments) • “Static” knowledge structure for representing facts and hierarchical relationships between them

28. Knowledge structures: Categories • Categories can be “perceptual” (learned implicitly, culture-independent) or “conceptual” (learnt explicitly, hence “verbalizable”) • Categories have “properties” or “features” that define the belongingness of an object to them • Musical categories: note (as event vs. other sound events), note (as pitched event), source, rhythm pattern (waltz, march…), notated duration (black, quaver…), key, mode, chord, genre… • Different categorization models (prototype-based, exemplar-based, etc.)

29. Models of categorization • Classical theory: Categories are defined by enumerable properties (rule-based classification) • Example-based: Categories are defined (implicitly) by the exemplars belonging to them (K-NN classification) • Prototype-based: Categories are represented by a prototype that averages the properties of all the instances belonging to the category (Gaussian mixture models) • Boundary-based: Categories are represented by the boundaries that separate their respective instances (Support Vector Machines)

30. Black and white dots are different sounds, defined by 2 formants • P is the prototype of white category, NP the prototype of the black; • Consider how the marked dots would be classified according to prototype and instance-based models ? ?

31. Categorical perception Tendency to perceive stimuli as falling into discrete categories rather than in terms of smooth gradients • Color, Speech sounds • Music: pitches, interval sizes, rhythmic categories

32. Category centres • ~ familiar tuning (equal temperament) Categorical perception of musical intervals(Burns & Campbell, 1974) Task: tell if the following 2 tones are a second, fifth, seventh, etc. Heightened discrimination near category boundary –abrupt transitions of %- Stimuli: Melodic intervals of complex tones Participants: Musicians Question: Which of 12 categories does the interval belong to? Percent of correct judgments • Just noticeable difference (JND) is smaller at boundary: a small change changes the category

33. Melodic categories: tuning systems, scales, pitches • Tuning systems reduce the variety of audible frequencies into a small amount of classes to be discriminated • Scales are “subpopulations” of a tuning system that allow pitch information to be managed under the constraints of our memories (=<7 pitches considered) • Scales provide a framework to admit “pitch nuances” and small mistunings as instances of the learned categories (categorical perception at play here!) -> but poor recall of them as they are usually not encoded as such!!!

34. Knowledge structures: Schemata 3 different melodic schemata, based on Meyer (1958) • Dynamic knowledge structures • Organized set of knowledge about event sequences, spatial and temporal combinations of elements • Help to recognize and to code a series of events or objects • They also guide the performance of musical behaviors • Basis for the elaboration of prediction of musical events (e.g., tonal schemata, predicting the appearance of the tonic in certain metrical positions, for longer durations, etc.)

35. Recovering declarative knowledge from memory (I) • Recollection: cueing of a memory intentionally (tell me names of dodecaphonic composers) • Reminding: cueing of a memory unintentionally (something we perceive makes other memory to pop up, e.g. thinking on Beethoven reminds you about Vienna –by means of having visited his house there-) • Recognition: an item acts as its own cue (we “realize” that the item is in our long term memory, e.g. listening to a pattern of 4 specific notes (G-G-G-E) we recognize Beethoven’s Fifth Symphony)

36. Recovering declarative knowledge from memory (II) Priming is a way of implicitly testing LTM (because of the associative character of LTM): • Reading “She sings a song” makes the word “music” more readily available that the word “socks” (i.e. reaction time to recognition of “music” as a word is faster) • Listening to a melody using 4 notes from a major scale makes another note from that scale more readily available or detectable than those that are not (less errors, shorter reaction times)