Langston psycholinguistics lecture 3
Download
1 / 65

Speech perception - PowerPoint PPT Presentation


  • 557 Views
  • Updated On :

Langston Psycholinguistics Lecture 3. Speech perception. Plan. Top-down Comprehension Bottom-up. Plan. Our goal is to start with the input and see how far we can take it. Constraint satisfaction problem. We will introduce top-down influences when the situation demands it.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Speech perception' - libitha


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Langston psycholinguistics lecture 3 l.jpg

Langston

Psycholinguistics

Lecture 3

Speech perception


Slide2 l.jpg
Plan

  • Top-down

  • Comprehension

  • Bottom-up


Slide3 l.jpg
Plan

  • Our goal is to start with the input and see how far we can take it.

    • Constraint satisfaction problem.

  • We will introduce top-down influences when the situation demands it.


What is speech l.jpg
What is speech?

  • Levels of analysis:

    • Acoustic: The physical speech signal.

    • Articulatory: How it's made.

      • Phones: Individual sounds (approximately 4000 available, about 869 in some language, about 100 account for most, Kluender, 1994).

      • Phonemes: Mental representation of sounds or sounds that affect meaning. Often made up of several phones treated as alike (keep cool).


What is speech5 l.jpg
What is speech?

  • Levels of analysis:

    • Phonemes: Not all differences in sounds are phonemic (pin spin). Allophones: Set of phones treated as identical by a language.

    • Changing a phoneme will change the meaning (bit pit).

    • You can map a language's phonemes by looking for minimal pairs.


What is speech6 l.jpg
What is speech?

  • Levels of analysis:

    • Phonemes: Languages seem to choose phonemes to maximize distinctiveness:

    • (Kluender, 1994)


What is speech7 l.jpg
What is speech?

  • Levels of analysis:

    • Morphemes: Units that actually have meaning (we'll come to these later).


Articulatory phonetics l.jpg
Articulatory Phonetics

  • Based on how sounds are produced.

  • A consonant is:

    • Air +

    • Voicing (on or off) +

    • Manner (some form of disruption) +

    • Place (where the disruption happens)



Articulatory phonetics10 l.jpg
Articulatory Phonetics

  • Here's a link to a map with a clickable glossary: http://www.sil.org/mexico/ling/glosario/E005ci-PlacesArt.htm


Articulatory phonetics11 l.jpg
Articulatory Phonetics

  • The manners:

    • Plosive (stop): Completely stop the air flow.

    • Fricative: Interrupt the air flow and create friction.

    • Affricate: Stop released to a fricative.

    • Nasal: Stop with sound coming out the nasal passages.

    • Flap: Brief stoppage.

    • Trill: Hold it in place and let it vibrate.


Articulatory phonetics12 l.jpg
Articulatory Phonetics

  • The manners:

    • Approximant: Like a fricative, little obstruction.

      • Liquids: Central (flow over the middle of the tongue) or lateral (flow around the sides of the tongue).

      • Glides: Similar to a vowel but with the tongue creating a small amount of turbulence (also called semivowels).




Articulatory phonetics15 l.jpg
Articulatory Phonetics

  • English:

    • Bilabial:

      • Stop: voiced bin, unvoiced pin

      • Nasal: man

      • Approximants: wind

    • Labiodental:

      • Fricative: voiced vat, unvoiced fat

    • Dental:

      • Fricative: voiced then, unvoiced thin


Articulatory phonetics16 l.jpg
Articulatory Phonetics

  • English:

    • Alveolar:

      • Stop: voiced dip, unvoiced tip

      • Nasal: nap

      • Flap: city

      • Fricative: voiced zap, unvoiced sap

      • Approximants: central rip, lateral lip

    • Post-alveolar: (palatal?)

      • Fricative: voiced azure, unvoiced sure

      • Affricate: voiced jug, unvoiced chug


Articulatory phonetics17 l.jpg
Articulatory Phonetics

  • English:

    • Palatal:

      • Approximant: your

    • Velar:

      • Stop: voiced got, unvoiced cot

      • Nasal: sing

    • Glottal:

      • Stop: satin

      • Fricative: hen


Articulatory phonetics18 l.jpg
Articulatory Phonetics

  • A vowel is:

    • Part: front, center, back +

    • Height: High, mid, low



Articulatory phonetics20 l.jpg
Articulatory Phonetics

  • English:

    • Front: beet, bit, baby, bet, bat

    • Central: hut, sofa, bird, heater

    • Back: boot, book, bode, bought, hot


Articulatory phonetics21 l.jpg
Articulatory Phonetics

  • English:

    • Also dipthongs: cute, bite, bough, boy

    • Also suprasegmentals (added on to the vowels):

      • Stress: blackbird, blackbird

      • Length

      • Tone contour


Acoustic phonetics l.jpg
Acoustic Phonetics

  • You can use a spectrograph to produce a spectrogram. This is a graphic representation of speech.


Acoustic phonetics23 l.jpg
Acoustic Phonetics

  • If you download Praat you can produce your own spectrograms relatively easily. Get Praat here: http://www.fon.hum.uva.nl/praat/


Acoustic phonetics24 l.jpg
Acoustic Phonetics

  • The acoustic approach is to analyze the physical speech signal without making reference to how it was produced.



Acoustic phonetics26 l.jpg
Acoustic Phonetics

  • Formant: “a concentration of acoustic energy around a particular frequency in the speech wave” (Praat Tutorial, see next page for link).


Acoustic phonetics27 l.jpg
Acoustic Phonetics

  • You can learn more about formants in the Praat tutorial here: http://person2.sol.lu.se/SidneyWood/praate/whatform.html


Acoustic phonetics28 l.jpg
Acoustic Phonetics

  • Formant transition: A sharp rise or fall in a formant. Usually a consonant.


Acoustic phonetics29 l.jpg
Acoustic Phonetics

  • Steady state: Part of a formant with little or no change. Generally vowels.


Acoustic phonetics30 l.jpg
Acoustic Phonetics

  • The darker the band the more energy there is there.

  • You can see sounds change over time by going from left to right.


Acoustic phonetics31 l.jpg
Acoustic Phonetics

  • Problems for perception:

    • Parallel transmission: You do not produce phonemes like beads on a necklace. Instead, you are transmitting overlapping parts of phonemes in parallel (Easter eggs).


Acoustic phonetics32 l.jpg
Acoustic Phonetics

  • Problems for perception:

    • Parallel transmission:


Acoustic phonetics33 l.jpg
Acoustic Phonetics

  • Problems for perception:

    • Context conditioned variation: Each phoneme is affected by surrounding phonemes (lack of invariance).


Acoustic phonetics34 l.jpg
Acoustic Phonetics

  • Problems for perception:

    • Context conditioned variation:


How does perception work l.jpg
How Does Perception Work?

  • From Kerzel & Bekkering (2000; doi:10.1037/0096-1523.26.2.634):

    • Direct realism: “listeners to speech recover information about the articulatory activities of the vocal tract from various sources of information” (p. 635).

    • But: Not motor based. The articulators structure the “informational medium.”


How does perception work36 l.jpg
How Does Perception Work?

  • From Kerzel & Bekkering (2000):

    • Direct realism: “when the ear of the listener is stimulated by the acoustic medium, the structure is imparted and the listener perceives the speaker's gestures” (p. 635).

    • Can also come from structuring of optic medium.

    • Direct perception.


Example 1 l.jpg
Example 1

http://sunburn.stanford.edu/~nick/compdocs/,

click on Practical HI Examples.pdf


Examples 4 5 l.jpg
Examples 4 & 5

http://www.baddesigns.com/file.html

http://www.baddesigns.com/sidewalk.html


How does perception work39 l.jpg
How Does Perception Work?

  • Direct realism:

    • Carello, Anderson, & Kunkler-Peck (1998; doi:10.1111/1467-9280.00040): Information in the auditory signal can be used to recover information about lengths of dowels (I'll be dropping some dowels).


How does perception work40 l.jpg
How Does Perception Work?

Carello, Anderson, & Kunkler-Peck (1998, p. 212)


How does perception work41 l.jpg
How Does Perception Work?

Carello, Anderson, & Kunkler-Peck (1998, p. 212)


How does perception work42 l.jpg
How Does Perception Work?

  • Direct realism:

    • Kunkler-Peck & Turvey (2000; doi:10.1037/0096-1523.26.1.279): Auditory information can also be used to recover information about an object's shape.


How does perception work43 l.jpg
How Does Perception Work?

  • Direct realism:

    • To sum up: The signal contains sufficient structure to recover a distal property (shape). Speech could work the same way (the distal property is phonetic gesture).


How does perception work44 l.jpg
How Does Perception Work?

  • From Kerzel & Bekkering (2000):

    • Fuzzy logical model of perception (FLMP):

      • “features are evaluated in terms of prototypes of syllables” (p. 635).

      • “degree of correspondence to the prototype is determined” (p. 635).

      • “the relative goodness of match of each prototype is evaluated, and the prototype with the best match is selected” (p. 635).


How does perception work45 l.jpg
How Does Perception Work?

  • From Kerzel & Bekkering (2000):

    • Fuzzy logical model of perception (FLMP):

      • “speech perception is explained by a best-match procedure” (p. 635).


How does perception work47 l.jpg
How Does Perception Work?

  • From Galantucci, Fowler, & Turvey (2006): Motor theory of speech perception. 3 parts:

    • “speech processing is special” (p. 361)

    • “perceiving speech is perceiving vocal tract gestures” (p. 361)

    • “speech perception involves access to the speech motor system” (p. 361)


How does perception work48 l.jpg
How Does Perception Work?

  • “speech processing is special”

    • Perception of distal properties unique to speech. No. (See the shape stuff above.)

    • Recruitment of the motor system unique to speech. No.

    • Special neural hardware. Not enough evidence to tell, but probably no.


How does perception work49 l.jpg
How Does Perception Work?

  • “perceiving speech is perceiving vocal tract gestures”

    • “the objects of speech perception are the speakers' vocal tract gestures and not the acoustic patterns that the gestures generate in the air” (p. 365)


How does perception work50 l.jpg
How Does Perception Work?

  • “perceiving speech is perceiving vocal tract gestures”

    • When articulation and sound go their separate ways, which way does perception go? With articulation (di-du).


How does perception work51 l.jpg
How Does Perception Work?

  • “perceiving speech is perceiving vocal tract gestures”

    • “gestures may be specified by information other than that in air pressure waves” (p. 365)

      • McGurk effect (http://psiexp.ss.uci.edu/research/teachingP140C/demos/McGurk_large.mov).

      • Perception of speech in a noisy environment facilitated by seeing the speaker (Sumby & Pollack, 1954).


How does perception work52 l.jpg
How Does Perception Work?

  • “perceiving speech is perceiving vocal tract gestures”

    • Speech imitation is fast.

    • In choice tasks, there is not the increase in reaction time expected.


How does perception work53 l.jpg
How Does Perception Work?

  • “perceiving speech is perceiving vocal tract gestures”

    • “the signal should be processed so that acoustic information for a given gesture is used as information for that gesture even when its acoustic consequences overlap with the acoustic consequences of another gesture” (p. 366).

    • Evidence is yes.


How does perception work54 l.jpg
How Does Perception Work?

  • “speech perception involves access to the speech motor system”

    • Bell-Berti, Raphael, Pisoni, & Sawusch (1979): The way people produce vowels is related to the way they perceive them.

      • http://www.jango.com/music/Merle+Haggard?l=0 (go to 34 seconds).

      • Or: http://www.youtube.com/watch?v=vfcv74S1Zmo

      • Artist: Merle Haggard; Song: I Think I'll Just Stay Here And Drink Take all the money in the bank.Think 'll just stay here and drink.


How does perception work55 l.jpg
How Does Perception Work?

  • “speech perception involves access to the speech motor system”

    • Kerzel & Bekkering (2000; doi:10.1037/0096-1523.26.2.634): Seeing a face make a syllable affects responses to written stimulus even though face is irrelevant.


How does perception work56 l.jpg
How Does Perception Work?

  • “speech perception involves access to the speech motor system”

    • In sum, there is reason to believe that perception is particularly attuned to the general anatomical and dynamical constraints on biological movements.... In other words, the same conclusion that Liberman and colleagues…drew specifically for speech, that speech motor competence must inform speech perception, can be drawn…for motor competence and perception quite generally. (p. 371).


How does perception work57 l.jpg
How Does Perception Work?

  • Also:

    • Mirror neurons: Neurons that fire during action also fire when seeing someone else do the action. Perhaps action recognition comes from neural simulation of action. (But, mirror neurons in humans?)


How does perception work58 l.jpg
How Does Perception Work?

  • Also:

    • Canonical neurons: Respond during grasping an object and when seeing the object. “responsive to the actions that an object potentially affords, even when acting on the object is not required” (Galantucci, et al., 2006, p. 371).


How does perception work59 l.jpg
How Does Perception Work?

  • Also:

    • “the perceptual relationship between an animal and its surrounding physical world is reflected in the nervous system in a way that is intimately intertwined with the neural means for preparing to produce compatible actions” (Galantucci, et al., 2006, p. 372).


Questions l.jpg
Questions

  • What about phonemic restoration?

    A shorter one for class: http://www.youtube.com/watch?v=Or1Xg4NXRws

  • What about categorical perception? http://www.ling.gu.se/~anders/KatPer/Applet/index.eng.html


Questions61 l.jpg
Questions

  • What about phonetic symbolism? (size-sound)


Questions62 l.jpg
Questions

  • What can be learned from statistical regularities in the input? Aslin, Saffran, & Newport (1998; doi:10.1111/1467-9280.00063)

    • Can learners use statistical information to discover word boundaries?

    • Boundaries “not marked by any consistent acoustic cues” (p. 321).


Questions63 l.jpg
Questions

  • Aslin, Saffran, & Newport (1998):

    • Presented infants with words from an artificial language (3 syllables, 4 words in each of two languages).

    • Put into a 3-minute stream:

      • Pabikugolatudaropitibudodaropigolatu…

    • Infants listen longer to part-words (crossing boundaries) than words.


Questions64 l.jpg
Questions

  • Aslin, Saffran, & Newport (1998):

    • Stimuli were constructed specifically not to have information from prosody, pauses, or phonotactics. These would provide more sources of constraint in real inputs.