Norbert Kopčo

1. Click vs. click-click vs. blink-click: Factors influencing human sound localization in the horizontal plane • Norbert Kopčo • TU Košice Dept. of Cybernetics and AIBoston University Hearing Research CenterDartmouth College Center for Cognitive Neuroscience

2. Intro: Sound localization 3-dimmensional: azimuth, elevation, distance depends on: • stimulus type: spectrum, temporal aspects • environment: anechoic, reverberant • source movement: static, dynamic • presence of other stimuli (auditory or visual) • a priori knowledge / expectations about the scene

3. Effect of additional stimuli • The extra sound can act as a: • Masker – localization worse • Adaptor – localization biased (Attraction/Repulsion) • Real sound (of which the target is a reflection) – localization worse/suppressed • Perceptual stream of which the target is or is not a part • Cue – localization better (doesn’t have to be auditory) • Anchor – change localization strategy

4. Effect of additional sounds • Temporal relations studied previously: • Extra sound precedes target by: • 10 secs to mins Adaptation/Repulsion • 50 msecs to 1 sec Adaptation/ reflections • 4 – 40 msecs Precedence • Concurrent sounds Adaptation/Repulsion • Inverse order Backward masking

5. Auditory Pathway and Spatial Hearing • Cochlea – peripheral filtering and neural coding • Olivary complex – processing of binaural information • Thalamus (Inferior Colliculus) – integration, modulation detection • Auditory Cortex – auditory object formation, figure/ground separation, ASA • Posterior Parietal Cortex – supramodal spatial representation & attentional modulation

6. Current goal • Begin to understand auditory localization in a more complex scene: • when target is preceded by another identical sound/s from a known location that the listener should ignore (Exp 1) • when target is preceded by visual or auditory cue that allows the listener to direct spatial attention (Exp 2) • when a concurrent visual stimulus induces a shift in auditory perception / ventriloquism (Exp 3)

7. Experiment 1Perceptual and central effects in sound localization with a preceding distractor (aka Click vs. Click-click vs. Click-click-click-click...) • Collaborators • Barbara Shinn-Cunningham, Virginia Best • Hearing Research Center • Boston University

8. Exp 1 - Preceding distractor: Intro • Several preceding studies indicated that preceding stimulus influences localization at SOAs of several hundreds milliseconds (Kopco et al., 2001, Perrott and Pacheco, 1989) • Goal: • Characterize this influence (bias and std.dev. in responses) • Determine its cause. Candidates: • short-term adaptation in brainstem representations • reverberation suppression and acoustics • strategy • perceptual organization • attention: focused away from distractor location

9. Exp 1 - Preceding distractor: Hypotheses • Peripheral factors will have short-term effects • Central factors will influence results at longer separations • Effect of reverberation can be separated by comparing performance in anechoic and echoic rooms • Effect of perceptual organization can be addressed by modifying the stimuli

10. Exp 1 - Preceding distractor: Methods • Anechoic room or a classroom • Blocks of trials with fixed distractor location • Trials with SOAs of 25,50,100,200 or 400 ms interleaved w/ no distractor trials • Seven subjects

11. Exp 1 - Preceding distractor: Results • Complex pattern of biases and standard deviation effects observed • Four main effects in terms of bias discussed • Bias 1: Lateral bias for frontal targets and lateral distractor in room

12. Exp 1 - Preceding distractor: Results – Bias 1 • ROOM • Largest effect • Strongest at • short SOAs • No comparable • effect of frontal • distractor

13. Exp 1 - Preceding distractor: Results – Bias 1 • ANECHOIC • ROOM • Effect eliminated in anechoic room  has to do with reverberation. • Acoustic or neural interaction?

14. Exp 1 – Bias 1: Perceptual organization • ROOM: Click-click • ROOM: click-click-click-click … click • Effect not due to acoustics because correct representation is available

15. Exp 1 – Bias 1: Standard deviation • The largest increase in standard deviation corresponds with the largest bias  • Neural suppression along with reflections • BUT: Why only lateral distractor?

16. Exp 1 - Preceding distractor: Results – Bias 2 • Targets in the middle of the range are attracted by the distractor, independent of: • Environment • Distractor location • Only at short SOAs •  Interactions in low-level spatial maps (brainstem)

17. Exp 1 - Preceding distractor: Results – Bias 3 • Lateral targets are repulsed by lateral distractors • Independent of SOA • Independent of environment • Probably central effect: e.g., change in response strategy, using distractor as an anchor w/ known location • Not in front because of higher resolution.

18. Exp 1 - Preceding distractor: Context • There is bias also in the no-distractor responses • The bias is always away from the non-present distractor • Because the runs were interleaved, this bias had to build up anew during each run

19. Exp 1 - Preceding distractor: Context • Difference in no-distractor responses in the frontal and lateral distractor context • Is independent of azimuth • Grows over time • Slightly stronger for the 8-click train context • Contextual plasticity on time scale of minutes • Similar to effects of long-term exposure • Either due to bottom-up factors (distribution of stimuli) or top-down factors (focusing away from distractor) Contextual bias

20. Exp 1 - Preceding distractor: Summary • A preceding distractor coming from a known location • Induces a complex pattern of biases • Over a range of time scales • Probably caused at different stages in the spatial auditory processing pathway

21. Experiment 2Modality-dependant attentional control in human sound localization(aka Click vs. Beep-click vs. Blink-click) • In collaboration w/ students • Beáta Tomoriová, Rudolf Andoga, Martin Bernát • Perception and Cognition Lab • Technical University, Košice

22. Exp 2 – Uni-/Cross-modal attention: Intro • Several studies explored the question whether directing automatic or strategic attention by an auditory cue can improve sound localization (Spence & Driver, 1994; Sach, 2000; Kopco & Shinn-Cunningham, 2003) • Results: improvements in RTs (Spence&Driver), but small (Sach) or no (Kopco) improvements in performance • Possible reason: the SOAs too short to orient attention • Goal: • determine whether attentional effects occur at longer SOAs • compare the effect of a visual and auditory cue

23. Exp 2 – Uni-/Cross-modal attention: Hypotheses • No effect of automatic attention (previous studies) • Strategic attention will affect performance at long SOAs • Effect modality-independent because spatial cuing very coarse (only left vs. right)

24. Exp 2 – Uni-/Cross-modal attention: Methods • Virtual auditory environment • Target – broadband click • Cue indicates side of target: • visual (arrow on a computer screen) • auditory (monaural tone) • SOA: 400, 800, 1600 ms • Informative: 100%, 80%, 50% validity • analysis: mean and s.d. in responses

25. Exp 2 – Uni-/Cross-modal attn: Results - bias • Mean effect of auditory cue (averaged across target azimuth): • Invalid cues cause medial bias, fairly independent of SOA • Valid cues cause similar medial bias

26. Exp 2 – Uni-/Cross-modal attn: Results - bias • When cue modality is visual: • Invalid cues cause medial bias, similar to the auditory cues • Valid cues cause lateral bias that grows with SOA • Modality through • which expectation of • the target location is • controlled influences • the perceived location

27. Exp 2 – Uni-/Cross-modal attn: Results – s.d. • Effect in terms of standard deviation: • No effect of auditory cue • Visual cue never improves performance, but invalid cue at 1600 ms increases s.d. • Summary: • Cuing doesn’t improve performance • Expectation of side of stimulus induces bias in a modality dependent way • Might have something to do with the coordinate systems in which visual and auditory space are represented

28. Experiment 3Behavioral examination of the auditory spatial coordinate system using the ventriloquism effect • Collaboration • Jennifer Groh • Center for Cognitive Neuroscience, Dept of Psychological and Brian Sciences, Dartmouth College • Barbara Shinn-Cunningham, I-Fan Lin • Boston University

29. Exp 3 – Coordinate system of auditory space: Intro • Mullette-Gillman et al. (2005): • Does the visual and auditory spatial coding have the same reference frame in the monkey parietal cortex? • Is the frame head-centered (as in auditory periphery) or eye-centered (as in visual periphery)? • This is an issue only for primates and animals that can move their eyes (not barn owls) • Result: some neurons in PPC A-only, some V-only, some AV, some head-centered, some eye-centered

30. Exp 3 – Coordinate system of auditory space: Intro • Here, use the ventriloquism effect to address a similar question behaviorally in monkeys and in humans: • Is the coordinate system at which the auditory behavioral responses are determined head- or eye-centered? • Method: • Induce a local shift in the auditory spatial map for a fixed eye position. • Move eyes to a new position. • If the region of the shift doesn’t change  head-centric • Otherwise, eye-centric coordinate system

31. Exp 3 – Coordinate system of auditory space: Method • Study performed in humans and in monkeys • Monkey data here

32. Exp 3 – Coordinate system of auditory space: Results (preliminary) • Difference between positive (rightward) and negative (leftward) shifts induced in the central region with left fixation point and generalization testedwith right fixation point • Result: • Induced shift generalizes • on the right side • No shift in bias due to change • in fixation point  • Head-centric coordinate system

33. Overall summary • Three experiments explored various aspects of horizontal sound localization • Understanding is limited even in the simple auditory scenes studied • Need follow-ups to clarify results

34. Acknowledgements • US National Institutes of Health (PIs: Shinn-Cunningham and Groh) • US National Academy of Sciences (Shinn-Cunningham, Kopčo) • Slovak Scientific Grant Agency (Kopčo)

35. Distance perception in reverberant environments- is consistent experience necessary for accurate distance perception?- also, studies looking at other parameters (mono- vs. binaural, anechoic vs. reverberant, real vs. simulated environments)“Room learning” and calibration to its acoustic properties- is localization accuracy and “room learning” affected by changes in listener position in a room?- do speech perception mechanisms calibrate to different acoustic environments?Spatial release from masking- effect of signal and masker location on detectability/intelligibility of pure tones, broadband non-speech stimuli, and speech in anechoic and reverberant environments Overview of recent studies of binaural and spatial hearing