Understanding speech in the background of other sounds

1. Understanding speech in the background of other sounds It's complicated!

3. Why is this interesting? • Speech is the major form of communication between people and most is not heard in quiet … • Even in a classroom • People of every age vary a lot in how well they can understand speech in the presence of ‘noise’ • Even when they have normal hearing … • And impaired hearing makes everything worse! • Lots of developmental disorders seem to have an impact on this ability • Language impairment • Autism spectrum disorders • Auditory processing disorder (APD)?

4. Essential terminology • signalor target • what you are trying to listen to • typically speech or music or … • ‘noise’or masker • what you are trying to ignore • can be noise like from a hoover, but also other speech

5. There are lots of different kinds of ‘noises’ ‘noise’ alone speech + ‘noise’

6. Another kind of ‘noise’ ‘noise’ alone speech + ‘noise’ ‘show’ starts at t≈0.65 ms

7. One major way that noises interfere • By the energy in the masker interacting directly with the speech in the inner ear… • making it inaudible, or • disrupting the information in the speech • Energetic (EM) and Modulation Masking (MM) • like the noise of a hoover • MM is the disruptive effect that modulations in the masker have on the modulations in the target • So it’s not the energy in the masker that is so important • Similar to EM, in happening at the periphery (needing to be in the same time/frequency region)

8. Another way to interfere:Informational Masking • Everything else! • signal and masker ‘are both audible but the listener is unable to disentangle the elements of the target speech from a similar-sounding distracter’

9. Two aspects of Informational Masking • Difficulties in auditory scene analysis • forming disparate acoustic elements into distinct and coherent sources of sound • Difficulties in attention • Example: your own name • Both abilities are more in the brain than in the ear • although they depend upon the quality of the signal coming from the ear Shinn-Cunningham, B. G. (2008). Object-based auditory and visual attention. Trends In Cognitive Sciences, 12, 182-186.

10. More essential terminology Signal-to-Noise Ratio (SNR)

11. Signal-to-Noise Ratio (SNR) The amount of energy in the signal divided by the amount of energy in the noise

12. modulations dual tasking cognitive load concurrent task informational masking pupillometry cocktail party non-native listening vocal folds McGurk effect

13. I’m a decibel.Kids love me. SNRs are usually expressed in decibels (dB)

14. Getting to grips with SNRs SNR = +20 dB SNR = +10 dB SNR = 0 dB SNR = -10 dB SNR = -20 dB

15. The Children’s Coordinate Response Measure (CCRM) aka ‘Doggy’ • Bouncy • in quiet in steady noise against another talker

16. Performance measured adaptively Vary the: Signal-to-Noise Ratio (SNR) to find the: Speech Reception Threshold (SRT) [the SNR which leads to a fixed % correct, typically 50% or 79%] So the lower the better!

17. Children can’t ignore another talker  better performance -1.7 -0.7 slopes

18. Specific Language Impairment (SLI) • A specific deficit • occurs in the context of other cognitive abilities that are more or less normal. • SLI — Late-developing and severely impaired language abilities alongside more-or-less normal cognitive abilities.

19. Email message from AZ when 15-years old Subject: RE: A New Warriors Date: Fri, 8 Aug 1997 11:07:40 -0400 Can you make a new warriors? (e.g I have planed a new space marine for the Blood Angels and I have called him The Blood Hurter. He got a jump pack on him and a rip arm from the tyranids and a orks head on his belt.)

20. AZ: a child with SLI • Late to develop language, using 3 words at age 5: Mummy, Daddy, Gangan • Errors in plural forms: Two mens. Two foots. • Errors in verb tense: My dad go to work. This is what they ated. My mum make the breakfast. • Errors in the use of embedded phrases: Which cat Mrs White stroked? What did Mrs Brown dropped. • Nonverbal IQ  120-130

21. Abook for parents http://www.ican.org.uk/

22. Children with SLI find it hard ignore another talker  better performance – SRT (dB) 14 SLI & 17 TD children aged 6-11 years CCRM sentences PhD work of CsabaRedey-Nagy

23. Autism Spectrum Disorders (ASD) • Primary problems in aspects of social interactions • Often accompanied by ‘sensory’ symptoms, e.g. hyperacusis

24. A personal report from someone with ASD Being with a group of friends, and sitting in the middle, one conversation going on to my left and one to my right, no excessive background noise … when two are going at once, I can't follow either, so I usually just sit in silence.

25. Some people with autism find it hard ignore another talker  better performance CCRM sentences PhD work of Katharine Mair

26. Increased IM in older listeners • speech-shaped noise  better performance • 8-talker babble • 20s 30s 40s 50s 60s • age cohort • Rajan & Cainer (2008)

27. So, what about hearing impairment?!

28. Back to basics:A steady-state noise

29. Much is understood about what makes a steady noise a good or bad masker • spectral shape • SNR

30. Energetic & Modulation Masking • Noises interfere with speech to the extent that have energy in the same frequency regions • Can be quantified in the AI (articulation index) • or SII (speech intelligibility index) • Reflects direct interaction of masker and speech in the cochlea, which acts as a frequency analyser • Hearing impaired listeners are more affected by steady noises … • because they typically have impaired frequency selectivity (wider auditory filters).

31. Better frequency selectivity keeps noise in its place

32. But noises are typically not steady …

33. Fluctuating maskers afford ‘glimpses’ of the target signal target glimpses masker a.k.a. ‘dip listening’ masker

34. Hearing impaired listeners ‘glimpse’ less better performance → Performance in the SPIN task as a function of SNR for modulated and unmodulated noises (not an effect of ageing) Takahashi & Bacon (1992)

35. Results from a special kind of auditory prosthesis:A cochlear implant

36. The electrode array

38. CI simulations

39. CI users are greatly affected by noise SRT (dB)  better performance Bimodallistener } CI NH male target sentences Cullington & Zeng (2008)

40. Final remarks Different kinds of background sounds affect speech intelligibility in different ways …

41. Final remarks Even people with normal hearing can have an unusual degree of difficulty hearing speech in noisy backgrounds • young children • older people with normal hearing • children with language impairment • about half the people with High Functioning Autism

42. Final remarks And the situation is worse with impaired hearing because of … • increased energetic and modulation masking • which also interferes with auditory scene analysis

43. Thanks!