Understanding Spoken Language
This presentation is the property of its rightful owner.
Sponsored Links
1 / 105

Understanding Spoken Language using Statistical and Computational Methods Steven Greenberg International Computer Science Institute PowerPoint PPT Presentation


  • 73 Views
  • Uploaded on
  • Presentation posted in: General

Understanding Spoken Language using Statistical and Computational Methods Steven Greenberg International Computer Science Institute 1947 Center Street, Berkeley, CA 94704 http://www.icsi.berkeley.edu/~steveng (contains electronic versions of papers and links to data)

Download Presentation

Understanding Spoken Language using Statistical and Computational Methods Steven Greenberg International Computer Science Institute

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Understanding spoken language using statistical and computational methods steven greenberg international computer scie

Understanding Spoken Languageusing

Statistical and Computational Methods

Steven Greenberg

International Computer Science Institute

1947 Center Street, Berkeley, CA 94704

http://www.icsi.berkeley.edu/~steveng

(contains electronic versions of papers and links to data)

Patterns of Speech Sounds in Unscripted Communication -

Production, Perception, Phonology. Akademie Sankelmark, October 8-11, 2000


Understanding spoken language using statistical and computational methods steven greenberg international computer scie

OR ….


Understanding spoken language using statistical and computational methods steven greenberg international computer scie

How I Learned to Stop Worrying

and Use

The Canonical Form


Understanding spoken language using statistical and computational methods steven greenberg international computer scie

Disclaimer

I am a Phonetician - NOT!

(many thanks for the invite)


Understanding spoken language using statistical and computational methods steven greenberg international computer scie

No Scientist is an Island …

IMPORTANT COLLEAGUES

PHONETIC TRANSCRIPTION OF SPONTANEOUS SPEECH (SWITCHBOARD)

Candace Cardinal, Rachel Coulston, Dan Ellis, Eric Fosler, Joy Holllenback, John Ohala, Colleen Richey

STATISTICAL ANALYSIS OF PRONUNCIATION VARIATION

Eric Fosler, Leah Hitchcock, Joy Hollenback

ARTICULATORY-ACOUSTIC BASIS OF CONSONANT RECOGNITION

Leah Hitchcock, Rosaria Silipo

AUTOMATIC PHONETIC TRANSCRIPTION OF SPONTANEOUS SPEECH

Shawn Chang, Lokendra Shastri


Understanding spoken language using statistical and computational methods steven greenberg international computer scie

Germane Publications

STATISTICAL PROPERTIES OF SPOKEN LANGUAGE AND PRONUNCIATION MODELING

Fosler-Lussier, E., Greenberg, S. and Morgan, N. (1999) Incorporating contextual phonetics into automatic speech recognition. Proceedings of the International Congress of Phonetic Sciences, San Francisco.

Greenberg, S. and Fosler-Lussier, E. (2000) The uninvited guest: Information's role in guiding the production of spontaneous speech, in the Proceedings of the Crest Workshop on Models of Speech Production: Motor Planning and Articulatory Modelling, Kloster Seeon, Germany .

Greenberg, S. (1999) Speaking in shorthand - A syllable-centric perspective for understanding pronunciation variation, Speech Communication, 29, 159-176.

Greenberg, S. (1997) On the origins of speech intelligibility in the real world. Proceedings of the ESCA Workshop on Robust Speech Recognition for Unknown Communication Channels, Pont-a-Mousson, France, pp. 23-32.

Greenberg, S., Hollenback, J. and Ellis, D. (1996) Insights into spoken language gleaned from phonetic transcription of the Switchboard corpus, in Proc. Intern. Conf. Spoken Lang. (ICSLP), Philadelphia, pp. S24-27.

PERCEPTUAL BASES OF SPEECH INTELLIGIBILITY

Greenberg, S., Arai, T. and Silipo, R. (1998) Speech intelligibility derived from exceedingly sparse spectral information, Proceedingss of the International Conference on Spoken Language Processing, Sydney, pp. 74-77.

Greenberg, S. (1996) Understanding speech understanding - towards a unified theory of speech perception. Proceedings of the ESCA Tutorial and Advanced Research Workshop on the Auditory Basis of Speech Perception, Keele, England, p. 1-8.

Silipo, R., Greenberg, S. and Arai, T. (1999) Temporal Constraints on Speech Intelligibility as Deduced from Exceedingly Sparse Spectral Representations, Proceedings of Eurospeech, Budapest

AUTOMATIC PHONETIC TRANSCRIPTION AND SEGMENTATION

Chang, S., Shastri, L. and Greenberg, S. (2000) Automatic phonetic transcription of spontaneous speech (American English). Proc. Int. Conf. Spoken Lang. Proc., Beijing.

Shastri, L. Chang, S. and Greenberg, S. (1999) Syllable detection and segmentation using temporal flow neural networks. Proceedings of the International Congress of Phonetic Sciences, San Francisco, pp. 1721-1724.

http://www.icsi.berkeley.edu/~steveng


Understanding spoken language using statistical and computational methods steven greenberg international computer scie

Prologue


Understanding spoken language using statistical and computational methods steven greenberg international computer scie

Language - The Traditional Perspective

The “classical” view of spoken language posits a quasi-arbitrary relation between the lower and higher tiers of linguistic organization

Phonetic orthography


Understanding spoken language using statistical and computational methods steven greenberg international computer scie

Language - A Syllable-Centric Perspective

A more empirical perspective of spoken language focuses on the syllable as the interface between “sound” and “meaning”

Within this framework the relationship between the syllable and the higher and lower tiers is non-arbitrary and systematic statistically


Understanding spoken language using statistical and computational methods steven greenberg international computer scie

Take Home Messages

  • SPONTANEOUS SPEECH DIFFERS IN CERTAIN SYSTEMATIC WAYS   FROM CANONICAL DESCRIPTIONS OF LANGUAGE AND MORE FORMAL   SPEAKING STYLES    


Understanding spoken language using statistical and computational methods steven greenberg international computer scie

Take Home Messages

  • SPONTANEOUS SPEECH DIFFERS IN CERTAIN SYSTEMATIC WAYS   FROM CANONICAL DESCRIPTIONS OF LANGUAGE AND MORE FORMAL   SPEAKING STYLES    

    • Such insights can only be obtained at present with large amounts of phonetically labeled material (in this instance, four hours of telephone dialogues recorded in the United States - the Switchboard corpus)


Understanding spoken language using statistical and computational methods steven greenberg international computer scie

Take Home Messages

  • SPONTANEOUS SPEECH DIFFERS IN CERTAIN SYSTEMATIC WAYS   FROM CANONICAL DESCRIPTIONS OF LANGUAGE AND MORE FORMAL   SPEAKING STYLES    

    • Such insights can only be obtained at present with large amounts of phonetically labeled material (in this instance, four hours of telephone dialogues recorded in the United States - the Switchboard corpus)

  • THE PHONETIC PROPERTIES OF SPOKEN LANGUAGE APPEAR TO BE    ORGANIZED AT THE SYLLABIC RATHER THAN AT THE PHONE LEVEL


Understanding spoken language using statistical and computational methods steven greenberg international computer scie

Take Home Messages

  • SPONTANEOUS SPEECH DIFFERS IN CERTAIN SYSTEMATIC WAYS   FROM CANONICAL DESCRIPTIONS OF LANGUAGE AND MORE FORMAL   SPEAKING STYLES    

    • Such insights can only be obtained at present with large amounts of phonetically labeled material (in this instance, four hours of telephone dialogues recorded in the United States - the Switchboard corpus)

  • THE PHONETIC PROPERTIES OF SPOKEN LANGUAGE APPEAR TO BE    ORGANIZED AT THE SYLLABIC RATHER THAN AT THE PHONE LEVEL

    • Onsets are pronounced in canonical (i. e., dictionary) fashion 80-90% of the time


Understanding spoken language using statistical and computational methods steven greenberg international computer scie

Take Home Messages

  • SPONTANEOUS SPEECH DIFFERS IN CERTAIN SYSTEMATIC WAYS   FROM CANONICAL DESCRIPTIONS OF LANGUAGE AND MORE FORMAL   SPEAKING STYLES    

    • Such insights can only be obtained at present with large amounts of phonetically labeled material (in this instance, four hours of telephone dialogues recorded in the United States - the Switchboard corpus)

  • THE PHONETIC PROPERTIES OF SPOKEN LANGUAGE APPEAR TO BE    ORGANIZED AT THE SYLLABIC RATHER THAN AT THE PHONE LEVEL

    • Onsets are pronounced in canonical (i. e., dictionary) fashion 80-90% of the time

    • Nuclei and codas are expressed canonically only 60% of the time


Understanding spoken language using statistical and computational methods steven greenberg international computer scie

Take Home Messages

  • SPONTANEOUS SPEECH DIFFERS IN CERTAIN SYSTEMATIC WAYS   FROM CANONICAL DESCRIPTIONS OF LANGUAGE AND MORE FORMAL   SPEAKING STYLES    

    • Such insights can only be obtained at present with large amounts of phonetically labeled material (in this instance, four hours of telephone dialogues recorded in the United States - the Switchboard corpus)

  • THE PHONETIC PROPERTIES OF SPOKEN LANGUAGE APPEAR TO BE    ORGANIZED AT THE SYLLABIC RATHER THAN AT THE PHONE LEVEL

    • Onsets are pronounced in canonical (i. e., dictionary) fashion 80-90% of the time

    • Nuclei and codas are expressed canonically only 60% of the time

    • Nuclei tend to be realized as vowels different from the canonical form


Understanding spoken language using statistical and computational methods steven greenberg international computer scie

Take Home Messages

  • SPONTANEOUS SPEECH DIFFERS IN CERTAIN SYSTEMATIC WAYS   FROM CANONICAL DESCRIPTIONS OF LANGUAGE AND MORE FORMAL   SPEAKING STYLES    

    • Such insights can only be obtained at present with large amounts of phonetically labeled material (in this instance, four hours of telephone dialogues recorded in the United States - the Switchboard corpus)

  • THE PHONETIC PROPERTIES OF SPOKEN LANGUAGE APPEAR TO BE    ORGANIZED AT THE SYLLABIC RATHER THAN AT THE PHONE LEVEL

    • Onsets are pronounced in canonical (i. e., dictionary) fashion 80-90% of the time

    • Nuclei and codas are expressed canonically only 60% of the time

    • Nuclei tend to be realized as vowels different from the canonical form

    • Codas are often deleted entirely


Understanding spoken language using statistical and computational methods steven greenberg international computer scie

Take Home Messages

  • SPONTANEOUS SPEECH DIFFERS IN CERTAIN SYSTEMATIC WAYS   FROM CANONICAL DESCRIPTIONS OF LANGUAGE AND MORE FORMAL   SPEAKING STYLES    

    • Such insights can only be obtained at present with large amounts of phonetically labeled material (in this instance, four hours of telephone dialogues recorded in the United States - the Switchboard corpus)

  • THE PHONETIC PROPERTIES OF SPOKEN LANGUAGE APPEAR TO BE    ORGANIZED AT THE SYLLABIC RATHER THAN AT THE PHONE LEVEL

    • Onsets are pronounced in canonical (i. e., dictionary) fashion 80-90% of the time

    • Nuclei and codas are expressed canonically only 60% of the time

    • Nuclei tend to be realized as vowels different from the canonical form

    • Codas are often deleted entirely

    • Articulatory-acoustic features are also organized in systematic fashion with respect to syllabic position


Understanding spoken language using statistical and computational methods steven greenberg international computer scie

Take Home Messages

  • SPONTANEOUS SPEECH DIFFERS IN CERTAIN SYSTEMATIC WAYS   FROM CANONICAL DESCRIPTIONS OF LANGUAGE AND MORE FORMAL   SPEAKING STYLES    

    • Such insights can only be obtained at present with large amounts of phonetically labeled material (in this instance, four hours of telephone dialogues recorded in the United States - the Switchboard corpus)

  • THE PHONETIC PROPERTIES OF SPOKEN LANGUAGE APPEAR TO BE    ORGANIZED AT THE SYLLABIC RATHER THAN AT THE PHONE LEVEL

    • Onsets are pronounced in canonical (i. e., dictionary) fashion 80-90% of the time

    • Nuclei and codas are expressed canonically only 60% of the time

    • Nuclei tend to be realized as vowels different from the canonical form

    • Codas are often deleted entirely

    • Articulatory-acoustic features are also organized in systematic fashion with respect to syllabic position

    • Therefore, it is important to model spoken language at the syllabic level


Understanding spoken language using statistical and computational methods steven greenberg international computer scie

Take Home Messages

  • SPONTANEOUS SPEECH DIFFERS IN CERTAIN SYSTEMATIC WAYS   FROM CANONICAL DESCRIPTIONS OF LANGUAGE AND MORE FORMAL   SPEAKING STYLES    

    • Such insights can only be obtained at present with large amounts of phonetically labeled material (in this instance, four hours of telephone dialogues recorded in the United States - the Switchboard corpus)

  • THE PHONETIC PROPERTIES OF SPOKEN LANGUAGE APPEAR TO BE    ORGANIZED AT THE SYLLABIC RATHER THAN AT THE PHONE LEVEL

    • Onsets are pronounced in canonical (i. e., dictionary) fashion 80-90% of the time

    • Nuclei and codas are expressed canonically only 60% of the time

    • Nuclei tend to be realized as vowels different from the canonical form

    • Codas are often deleted entirely

    • Articulatory-acoustic features are also organized in systematic fashion with respect to syllabic position

    • Therefore, it is important to model spoken language at the syllabic level

  • THE SYLLABIC ORGANIZATION OF SPONTANEOUS SPEECH CALLS INTO QUESTION SEGEMENTALLY BASED PHONETIC ORTHOGRAPHY


Understanding spoken language using statistical and computational methods steven greenberg international computer scie

Take Home Messages

  • SPONTANEOUS SPEECH DIFFERS IN CERTAIN SYSTEMATIC WAYS   FROM CANONICAL DESCRIPTIONS OF LANGUAGE AND MORE FORMAL   SPEAKING STYLES    

    • Such insights can only be obtained at present with large amounts of phonetically labeled material (in this instance, four hours of telephone dialogues recorded in the United States - the Switchboard corpus)

  • THE PHONETIC PROPERTIES OF SPOKEN LANGUAGE APPEAR TO BE    ORGANIZED AT THE SYLLABIC RATHER THAN AT THE PHONE LEVEL

    • Onsets are pronounced in canonical (i. e., dictionary) fashion 80-90% of the time

    • Nuclei and codas are expressed canonically only 60% of the time

    • Nuclei tend to be realized as vowels different from the canonical form

    • Codas are often deleted entirely

    • Articulatory-acoustic features are also organized in systematic fashion with respect to syllabic position

    • Therefore, it is important to model spoken language at the syllabic level

  • THE SYLLABIC ORGANIZATION OF SPONTANEOUS SPEECH CALLS INTO QUESTION SEGEMENTALLY BASED PHONETIC ORTHOGRAPHY

    • It may be unrealistic to assume that any phonetic transcription based exclusively on segments (such as the IPA) is truly capable of capturing the important phonetic detail of spontaneous material


Understanding spoken language using statistical and computational methods steven greenberg international computer scie

Take Home Messages

  • SPONTANEOUS SPEECH DIFFERS IN CERTAIN SYSTEMATIC WAYS   FROM CANONICAL DESCRIPTIONS OF LANGUAGE AND MORE FORMAL   SPEAKING STYLES    

    • Such insights can only be obtained at present with large amounts of phonetically labeled material (in this instance, four hours of telephone dialogues recorded in the United States - the Switchboard corpus)

  • THE PHONETIC PROPERTIES OF SPOKEN LANGUAGE APPEAR TO BE    ORGANIZED AT THE SYLLABIC RATHER THAN AT THE PHONE LEVEL

    • Onsets are pronounced in canonical (i. e., dictionary) fashion 80-90% of the time

    • Nuclei and codas are expressed canonically only 60% of the time

    • Nuclei tend to be realized as vowels different from the canonical form

    • Codas are often deleted entirely

    • Articulatory-acoustic features are also organized in systematic fashion with respect to syllabic position

    • Therefore, it is important to model spoken language at the syllabic level

  • THE SYLLABIC ORGANIZATION OF SPONTANEOUS SPEECH CALLS INTO QUESTION SEGEMENTALLY BASED PHONETIC ORTHOGRAPHY

    • It may be unrealistic to assume that any phonetic transcription based exclusively on segments (such as the IPA) is truly capable of capturing the important phonetic detail of spontaneous material


Understanding spoken language using statistical and computational methods steven greenberg international computer scie

Take Home Messages

  • PHONETIC PROPERTIES OF SPONTANEOUS SPEECH REFLECT INFORMATION CONTENT

Greenberg, S. and Fosler-Lussier, E. (2000) The uninvited guest: Information's role in guiding the production of spontaneous speech, in the Proceedings of the Crest Workshop on Models of Speech Production: Motor Planning and Articulatory Modelling, Kloster Seeon, Germany .


Understanding spoken language using statistical and computational methods steven greenberg international computer scie

Road Map

  • PHONETIC TRANSCRIPTION OF SPONTANEOUS AMERICAN ENGLISH


Understanding spoken language using statistical and computational methods steven greenberg international computer scie

Road Map

  • PHONETIC TRANSCRIPTION OF SPONTANEOUS AMERICAN ENGLISH

    • Provides the basis for the statistical analyses of spontaneous material


Understanding spoken language using statistical and computational methods steven greenberg international computer scie

Road Map

  • PHONETIC TRANSCRIPTION OF SPONTANEOUS AMERICAN ENGLISH

    • Provides the basis for the statistical analyses of spontaneous material

  • A BRIEF TOUR OF PRONUNCIATION VARIATION FROM THE PERSPECTIVE OF:

    • Phonetic segments


Understanding spoken language using statistical and computational methods steven greenberg international computer scie

Road Map

  • PHONETIC TRANSCRIPTION OF SPONTANEOUS AMERICAN ENGLISH

    • Provides the basis for the statistical analyses of spontaneous material

  • A BRIEF TOUR OF PRONUNCIATION VARIATION FROM THE PERSPECTIVE OF:

    • Phonetic segments

    • Words


Understanding spoken language using statistical and computational methods steven greenberg international computer scie

Road Map

  • PHONETIC TRANSCRIPTION OF SPONTANEOUS AMERICAN ENGLISH

    • Provides the basis for the statistical analyses of spontaneous material

  • A BRIEF TOUR OF PRONUNCIATION VARIATION FROM THE PERSPECTIVE OF:

    • Phonetic segments

    • Words

    • Syllables


Understanding spoken language using statistical and computational methods steven greenberg international computer scie

Road Map

  • PHONETIC TRANSCRIPTION OF SPONTANEOUS AMERICAN ENGLISH

    • Provides the basis for the statistical analyses of spontaneous material

  • A BRIEF TOUR OF PRONUNCIATION VARIATION FROM THE PERSPECTIVE OF:

    • Phonetic segments

    • Words

    • Syllables

    • Articulatory-acoustic features


Understanding spoken language using statistical and computational methods steven greenberg international computer scie

Road Map

  • PHONETIC TRANSCRIPTION OF SPONTANEOUS AMERICAN ENGLISH

    • Provides the basis for the statistical analyses of spontaneous material

  • A BRIEF TOUR OF PRONUNCIATION VARIATION FROM THE PERSPECTIVE OF:

    • Phonetic segments

    • Words

    • Syllables

    • Articulatory-acoustic features

  • PERCEPTUAL EVIDENCE

    • The articulatory-acoustic basis of consonant recognition


Understanding spoken language using statistical and computational methods steven greenberg international computer scie

Road Map

  • PHONETIC TRANSCRIPTION OF SPONTANEOUS AMERICAN ENGLISH

    • Provides the basis for the statistical analyses of spontaneous material

  • A BRIEF TOUR OF PRONUNCIATION VARIATION FROM THE PERSPECTIVE OF:

    • Phonetic segments

    • Words

    • Syllables

    • Articulatory-acoustic features

  • PERCEPTUAL EVIDENCE

    • The articulatory-acoustic basis of consonant recognition

    • Not all articulatory-acoustic features are created equal - place-of-articulation cues appear to be most important for consonant recognition


Understanding spoken language using statistical and computational methods steven greenberg international computer scie

Road Map

  • PHONETIC TRANSCRIPTION OF SPONTANEOUS AMERICAN ENGLISH

    • Provides the basis for the statistical analyses of spontaneous material

  • A BRIEF TOUR OF PRONUNCIATION VARIATION FROM THE PERSPECTIVE OF:

    • Phonetic segments

    • Words

    • Syllables

    • Articulatory-acoustic features

  • PERCEPTUAL EVIDENCE

    • The articulatory-acoustic basis of consonant recognition

    • Not all articulatory-acoustic features are created equal - place-of-articulation cues appear to be most important for consonant recognition

  • COMPUTATIONAL METHODS

    • Automatic methods for phonetic transcription based on articulatory-acoustic features


Understanding spoken language using statistical and computational methods steven greenberg international computer scie

Road Map

  • PHONETIC TRANSCRIPTION OF SPONTANEOUS AMERICAN ENGLISH

    • Provides the basis for the statistical analyses of spontaneous material

  • A BRIEF TOUR OF PRONUNCIATION VARIATION FROM THE PERSPECTIVE OF:

    • Phonetic segments

    • Words

    • Syllables

    • Articulatory-acoustic features

  • PERCEPTUAL EVIDENCE

    • The articulatory-acoustic basis of consonant recognition

    • Not all articulatory-acoustic features are created equal - place-of-articulation cues appear to be most important for consonant recognition

  • COMPUTATIONAL METHODS

    • Automatic methods for phonetic transcription based on articulatory-acoustic features

    • Is the most likely means through which it will be possible to generate sufficient empirical data with which to rigorously test hypotheses germane to spoken language


Understanding spoken language using statistical and computational methods steven greenberg international computer scie

Phonetic Transcription of Spontaneous (American) English


Understanding spoken language using statistical and computational methods steven greenberg international computer scie

Phonetic Transcription of Spontaneous English

  • TELEPHONE DIALOGUES OF 5-10 MINUTES DURATION - SWITCHBOARD

  • AMOUNT OF MATERIAL MANUALLY TRANSCRIBED    

    • 3 hours labeled at the phone level and segmented at the syllabic level (this material was later phonetically segmented by automatic methods)

    • 1 hour labeled and segmented at the phonetic-segment level

  • DIVERSITY OF MATERIAL TRANSCRIBED

    • Spans speech of both genders (ca. 50/50%) reflecting a wide range of American dialectal variation (6 regions + “army brat”), speaking rate and voice quality

  • TRANSCRIBED BY WHOM?

    • 7 undergraduates and 1 graduate student, all enrolled at UC-Berkeley. Most of the corpus was transcribed by three individuals out of the original eight

    • Supervised by Steven Greenberg and John Ohala

  • TRANSCRIPTION SYSTEM

    • A variant of Arpabet, with phonetic diacritics such as:_gl,_cr, _fr, _n, _vl, _vd

  • HOW LONG DOES TRANSCRIPTION TAKE? (Don’t Ask!)

    • 388 times real time for labeling and segmentation at the phonetic-segment level

    • 150 times real time for labeling phonetic segments and segmenting syllables

  • HOW WAS LABELING AND SEGMENTATION PERFORMED?

    • Using a display of the signal waveform, spectrogram, word transcription and “forced alignments” (estimates of phones and boundaries) + audio (listening at multiple time scales - phone, word, utterance) on Sun workstations

  • DATA AVAILABLE AT - http://www.icsi/berkeley.edu/real/stp


Understanding spoken language using statistical and computational methods steven greenberg international computer scie

A Brief Tour of

Pronunciation Variation

in

Spontaneous American English


Understanding spoken language using statistical and computational methods steven greenberg international computer scie

Cumulative Word Frequency in English

Focus on 100 most common words

The 10 most common words account for 27% of the corpus

The 100 most common words account for 67% of the corpus

The 1000 most common words account for 92% of the corpus

Thus, most informal dialogues are composed of a relatively small number of common words.

However, it is the infrequent words that typically provide the precision and detail required for complex information transfer

92%

67%

27%

Computed from the Switchboard corpus (American English telephone dialogues)


Understanding spoken language using statistical and computational methods steven greenberg international computer scie

N

Pronunciation

N

Pronunciation

How Many Pronunciations of “And”?


Understanding spoken language using statistical and computational methods steven greenberg international computer scie

N

Pronunciation

N

Pronunciation

How Many Pronunciations of “And”?


Understanding spoken language using statistical and computational methods steven greenberg international computer scie

MCP

%Total

Most Common

Pronunciation

Rank

Word

N

#Pron

How Many Different Pronunciations?


Understanding spoken language using statistical and computational methods steven greenberg international computer scie

MCP

%Total

Most Common

Pronunciation

Rank

Word

N

#Pron

How Many Different Pronunciations?


Understanding spoken language using statistical and computational methods steven greenberg international computer scie

MCP

%Total

Most Common

Pronunciation

Rank

Word

N

#Pron

How Many Different Pronunciations?


Understanding spoken language using statistical and computational methods steven greenberg international computer scie

MCP

%Total

Most Common

Pronunciation

Rank

Word

N

#Pron

How Many Different Pronunciations?


Understanding spoken language using statistical and computational methods steven greenberg international computer scie

MCP

%Total

Most Common

Pronunciation

Rank

Word

N

#Pron

How Many Different Pronunciations?


Understanding spoken language using statistical and computational methods steven greenberg international computer scie

English is (sort of) like Chinese ….

95% of the words contain just ONE or TWO syllables ….

81% of the word tokens are monosyllabic

Of the 100 most common words, 90 are one syllable

in length

Only 22% of the words in the lexicon are one syllable long

Hence, there is a decided preference for monosyllablic words in informal discourse


Understanding spoken language using statistical and computational methods steven greenberg international computer scie

Syllable and. Word Frequencies are Similar

Words and syllables exhibit similar distributions over the 300 most common elements, accounting for 80% of the corpus

The similarity of their distributions is a consequence of most words consisting of just a single syllable


Understanding spoken language using statistical and computational methods steven greenberg international computer scie

Word Frequency in Spontaneous English

Word frequency as a function of word rank approximates a 1/f distribution, particularly after rank-order 10

Word frequency is logarithmically related to rank order in the corpus (I.e., the 10th most common word occurs ca. 10 times more frequently than the 100th most common word, etc.

Computed from the Switchboard corpus (American English telephone dialogues)


Understanding spoken language using statistical and computational methods steven greenberg international computer scie

Information Affects Pronunciation

The faster the speaking rate the more likely that the pronunciation deviates from canonical

However, the effect is much more pronounced for the 100 most common words than for more infrequent words

From Fosler, Greenberg and Morgan (1999); Greenberg and Fosler (2000)


Understanding spoken language using statistical and computational methods steven greenberg international computer scie

English Syllable Structure is (sort of) Like Japanese

Most syllables are simple in form (no consonant clusters)

87% of the pronunciations are simple syllabic forms

84% of the canonical corpus is composed of simple syllabic forms

n= 103, 054


Understanding spoken language using statistical and computational methods steven greenberg international computer scie

Complex Syllables are Important, Though

Thus, despite English’s reputation for complex syllabic forms, only ca. 15% of the syllable tokens are actually complex

There are many “complex” syllable forms (consonant clusters, but all occur relatively infrequently

Complex codas are not as frequently realized in actual pronunciation as their canonical representation

Complex onsets tend to preserve the canonical pronunciation in realize their canonical representation

n= 17,760


Understanding spoken language using statistical and computational methods steven greenberg international computer scie

Syllable-Centric Pronunciation

Codas tend to be pronounced canonically more frequently in formal speech than in spontaneous dialogues

Onsets are pronounced

canonically far more often than nuclei or codas

Percent Canonically Pronounced

(Read Sentences)

“Cat” [k ae t]

[k] = onset

[ae] = nucleus

[t] = coda

Syllable Position

(Spontaneous speech)

n= 120,814


Understanding spoken language using statistical and computational methods steven greenberg international computer scie

Complex Onsets are Highly Canonical

Complex onsets are pronounced more canonically than simple onsets despite the greater potential for deviation from the standard pronunciation

Percent Canonically Pronounced

(Read Sentences)

(Spontaneous speech)

Syllable Onset Type


Understanding spoken language using statistical and computational methods steven greenberg international computer scie

Speaking Style Affects Codas

Codas are much more likely to be realized canonically in formal than in spontaneous speech

Percent Canonically Pronounced

Syllable Coda Type


Understanding spoken language using statistical and computational methods steven greenberg international computer scie

Onsets (but not Codas) Affect Nuclei

The presence of a syllable onset has a substantial impact on the realization of the nucleus

Percent Canonically Pronounced


Understanding spoken language using statistical and computational methods steven greenberg international computer scie

Syllable-Centric Feature Analysis

  • Place of articulation deviates most in nucleus position

  • Manner of articulation deviates most in onset and coda position

  • Voicing deviates most in coda position

Phonetic deviation

along a SINGLE feature

Place is VERY unstable in nucleus position

Place deviates very little from canonical form in the onset and coda. It is a STABLE AF in these positions


Understanding spoken language using statistical and computational methods steven greenberg international computer scie

Articulatory PLACE Feature Analysis

  • Place of articulation is a “dominant” feature in nucleus position only

  • Drives the feature deviation in the nucleus for manner and rounding

Phonetic deviation

across SEVERAL features

Place “carries” manner and rounding in the nucleus


Understanding spoken language using statistical and computational methods steven greenberg international computer scie

Articulatory MANNER Feature Analysis

  • Manner of articulation is a “dominant” feature in onset and coda position

  • Drives the feature deviation in onsets and codas for place and voicing

Phonetic deviation

across SEVERAL features

Manner drives place and voicing deviations in the onset and coda

Manner is less stable in the coda than in the onset


Understanding spoken language using statistical and computational methods steven greenberg international computer scie

Articulatory VOICING Feature Analysis

  • Voicing is a subordinate feature in all syllable positions

  • Its deviation pattern is controlled by manner in onset and coda positions

Phonetic deviation

across SEVERAL features

Voicing is unstable in coda position and is dominated by manner


Understanding spoken language using statistical and computational methods steven greenberg international computer scie

LIP-ROUNDING Feature Analysis

  • Lip-rounding is a subordinate feature

  • Its deviation pattern is driven by the place feature in nucleus position

Phonetic deviation

across SEVERAL features

Rounding is stable everywhere except in the nucleus where its deviation pattern is driven by place


Understanding spoken language using statistical and computational methods steven greenberg international computer scie

Perceptual Evidence

for the

Importance of Place (and Manner) of Articulation Features


Understanding spoken language using statistical and computational methods steven greenberg international computer scie

Spectral Slit Paradigm


Understanding spoken language using statistical and computational methods steven greenberg international computer scie

Consonant Recognition - Single Slits


Understanding spoken language using statistical and computational methods steven greenberg international computer scie

Consonant Recognition - 1 Slit


Understanding spoken language using statistical and computational methods steven greenberg international computer scie

Consonant Recognition - 2 Slits


Understanding spoken language using statistical and computational methods steven greenberg international computer scie

Consonant Recognition - 3 Slits


Understanding spoken language using statistical and computational methods steven greenberg international computer scie

Consonant Recognition - 4 Slits


Understanding spoken language using statistical and computational methods steven greenberg international computer scie

Consonant Recognition - 5 Slits


Understanding spoken language using statistical and computational methods steven greenberg international computer scie

Consonant Recognition - 2 Slits


Understanding spoken language using statistical and computational methods steven greenberg international computer scie

Consonant Recognition - 2 Slits


Understanding spoken language using statistical and computational methods steven greenberg international computer scie

Consonant Recognition - 2 Slits


Understanding spoken language using statistical and computational methods steven greenberg international computer scie

Consonant Recognition - 2 Slits


Understanding spoken language using statistical and computational methods steven greenberg international computer scie

Consonant Recognition - 2 Slits


Understanding spoken language using statistical and computational methods steven greenberg international computer scie

Consonant Recognition - 2 Slits


Understanding spoken language using statistical and computational methods steven greenberg international computer scie

Consonant Recognition - 3 Slits


Understanding spoken language using statistical and computational methods steven greenberg international computer scie

Consonant Recognition - 3 Slits


Understanding spoken language using statistical and computational methods steven greenberg international computer scie

Consonant Recognition - 3 Slits


Understanding spoken language using statistical and computational methods steven greenberg international computer scie

Consonant Recognition - 4 Slits


Understanding spoken language using statistical and computational methods steven greenberg international computer scie

Consonant Recognition - 5 Slits


Understanding spoken language using statistical and computational methods steven greenberg international computer scie

Correlation - AFs/Consonant Recognition

Consonant recognition is almost perfectly correlated with place of articulation performance

This correlation suggests that the place feature is based on cues distributed across the entire speech bandwidth, in contrast to other features

Manner is also highly correlated with consonant recognition, voicing and rounding less so


Understanding spoken language using statistical and computational methods steven greenberg international computer scie

Automatic Phonetic Transcription

of Spontaneous Speech


Understanding spoken language using statistical and computational methods steven greenberg international computer scie

Automatic Phonetic Transcription

  • MAINSTREAM ASR SYSTEMS USE MOSTLY AUTOMATIC    ALIGNMENT DATA TO TRAIN NEW SYSTEMS


Understanding spoken language using statistical and computational methods steven greenberg international computer scie

Automatic Phonetic Transcription

  • MAINSTREAM ASR SYSTEMS USE MOSTLY AUTOMATIC    ALIGNMENT DATA TO TRAIN NEW SYSTEMS

    • These materials are highly inaccurate (35-50% incorrect labeling of phonetic segments and an average of 32 ms (40% of the mean phone duration) off in terms of segment boundaries


Understanding spoken language using statistical and computational methods steven greenberg international computer scie

Automatic Phonetic Transcription

  • MAINSTREAM ASR SYSTEMS USE MOSTLY AUTOMATIC    ALIGNMENT DATA TO TRAIN NEW SYSTEMS

    • These materials are highly inaccurate (35-50% incorrect labeling of phonetic segments and an average of 32 ms (40% of the mean phone duration) off in terms of segment boundaries

  • IT IS BOTH TIME-CONSUMING AND EXPENSIVE TO MANUALLY LABEL    AND SEGMENT PHONETIC MATERIAL FOR SPONTANEOUS CORPORA


Understanding spoken language using statistical and computational methods steven greenberg international computer scie

Automatic Phonetic Transcription

  • MAINSTREAM ASR SYSTEMS USE MOSTLY AUTOMATIC    ALIGNMENT DATA TO TRAIN NEW SYSTEMS

    • These materials are highly inaccurate (35-50% incorrect labeling of phonetic segments and an average of 32 ms (40% of the mean phone duration) off in terms of segment boundaries

  • IT IS BOTH TIME-CONSUMING AND EXPENSIVE TO MANUALLY LABEL    AND SEGMENT PHONETIC MATERIAL FOR SPONTANEOUS CORPORA

    • Manual labeling and segmentation typically requires 150-400 times real time to perform


Understanding spoken language using statistical and computational methods steven greenberg international computer scie

Automatic Phonetic Transcription

  • MAINSTREAM ASR SYSTEMS USE MOSTLY AUTOMATIC    ALIGNMENT DATA TO TRAIN NEW SYSTEMS

    • These materials are highly inaccurate (35-50% incorrect labeling of phonetic segments and an average of 32 ms (40% of the mean phone duration) off in terms of segment boundaries

  • IT IS BOTH TIME-CONSUMING AND EXPENSIVE TO MANUALLY LABEL    AND SEGMENT PHONETIC MATERIAL FOR SPONTANEOUS CORPORA

    • Manual labeling and segmentation typically requires 150-400 times real time to perform

  • WE HAVE DEVELOPED AN AUTOMATIC LABELING OF PHONETIC    SEGMENTS (ALPS) SYSTEM TO PROVIDE TRAINING MATERIALS FOR    ASR AND TO ENABLE RAPID DEPLOYMENT TO NEW CORPORA AND    FOREIGN LANGUAGE MATERIAL


Understanding spoken language using statistical and computational methods steven greenberg international computer scie

Automatic Phonetic Transcription

  • MAINSTREAM ASR SYSTEMS USE MOSTLY AUTOMATIC    ALIGNMENT DATA TO TRAIN NEW SYSTEMS

    • These materials are highly inaccurate (35-50% incorrect labeling of phonetic segments and an average of 32 ms (40% of the mean phone duration) off in terms of segment boundaries

  • IT IS BOTH TIME-CONSUMING AND EXPENSIVE TO MANUALLY LABEL    AND SEGMENT PHONETIC MATERIAL FOR SPONTANEOUS CORPORA

    • Manual labeling and segmentation typically requires 150-400 times real time to perform

  • WE HAVE DEVELOPED AN AUTOMATIC LABELING OF PHONETIC    SEGMENTS (ALPS) SYSTEM TO PROVIDE TRAINING MATERIALS FOR    ASR AND TO ENABLE RAPID DEPLOYMENT TO NEW CORPORA AND    FOREIGN LANGUAGE MATERIAL

    • Such material will be extremely useful for developing pronunciation models and new algorithms for ASR


Understanding spoken language using statistical and computational methods steven greenberg international computer scie

Automatic Phonetic Transcription

  • MAINSTREAM ASR SYSTEMS USE MOSTLY AUTOMATIC    ALIGNMENT DATA TO TRAIN NEW SYSTEMS

    • These materials are highly inaccurate (35-50% incorrect labeling of phonetic segments and an average of 32 ms (40% of the mean phone duration) off in terms of segment boundaries

  • IT IS BOTH TIME-CONSUMING AND EXPENSIVE TO MANUALLY LABEL    AND SEGMENT PHONETIC MATERIAL FOR SPONTANEOUS CORPORA

    • Manual labeling and segmentation typically requires 150-400 times real time to perform

  • WE HAVE DEVELOPED AN AUTOMATIC LABELING OF PHONETIC    SEGMENTS (ALPS) SYSTEM TO PROVIDE TRAINING MATERIALS FOR    ASR AND TO ENABLE RAPID DEPLOYMENT TO NEW CORPORA AND    FOREIGN LANGUAGE MATERIAL

    • Such material will be extremely useful for developing pronunciation models and new algorithms for ASR

  • THE ALPS SYSTEM CURRENTLY LABELS SPONTANEOUS MATERIALS    (OGI Numbers Corpus) WITH ca. 83% ACCURACY


Understanding spoken language using statistical and computational methods steven greenberg international computer scie

Automatic Phonetic Transcription

  • MAINSTREAM ASR SYSTEMS USE MOSTLY AUTOMATIC    ALIGNMENT DATA TO TRAIN NEW SYSTEMS

    • These materials are highly inaccurate (35-50% incorrect labeling of phonetic segments and an average of 32 ms (40% of the mean phone duration) off in terms of segment boundaries

  • IT IS BOTH TIME-CONSUMING AND EXPENSIVE TO MANUALLY LABEL    AND SEGMENT PHONETIC MATERIAL FOR SPONTANEOUS CORPORA

    • Manual labeling and segmentation typically requires 150-400 times real time to perform

  • WE HAVE DEVELOPED AN AUTOMATIC LABELING OF PHONETIC    SEGMENTS (ALPS) SYSTEM TO PROVIDE TRAINING MATERIALS FOR    ASR AND TO ENABLE RAPID DEPLOYMENT TO NEW CORPORA AND    FOREIGN LANGUAGE MATERIAL

    • Such material will be extremely useful for developing pronunciation models and new algorithms for ASR

  • THE ALPS SYSTEM CURRENTLY LABELS SPONTANEOUS MATERIALS    (OGI Numbers Corpus) WITH ca. 83% ACCURACY

    • The algorithms used are capable of achieving ca. 93% accuracy with only minor changes to the models


Understanding spoken language using statistical and computational methods steven greenberg international computer scie

Phonetic Feature Classification System


Understanding spoken language using statistical and computational methods steven greenberg international computer scie

Spectro-Temporal Profile (STeP)

  • STePs provide a simple, accurate means of delineating the acoustic    properties associated with phonetic features and segments

Vocalic


Understanding spoken language using statistical and computational methods steven greenberg international computer scie

Spectro-temporal Profile (STeP)

  • STePs incorporate information about the instantaneous modulation    spectrum distributed across the (tonotopic) frequency axis and can be    used for training neural networks.

Fricative


Understanding spoken language using statistical and computational methods steven greenberg international computer scie

Label Accuracy per Frame

  • Frames away from the boundary are labeled very accurately


Understanding spoken language using statistical and computational methods steven greenberg international computer scie

Sample Transcription Output

  • The automatic system performs very similarly to manual transcription in terms of both labels and segmentation

    • 11 ms average concordance in segmentation

    • 83% concordance with respect to phonetic labels


Understanding spoken language using statistical and computational methods steven greenberg international computer scie

In Conclusion ….


Understanding spoken language using statistical and computational methods steven greenberg international computer scie

Grand Summary

  • SPONTANEOUS SPEECH DIFFERS IN CERTAIN SYSTEMATIC WAYS   FROM CANONICAL DESCRIPTIONS OF LANGUAGE AND MORE FORMAL   SPEAKING STYLES    

    • Such insights can only be obtained at present with large amounts of phonetically labeled material (in this instance, four hours of telephone dialogues recorded in the United States - the Switchboard corpus)

    • Automatic methods will eventually supply badly needed data for more complete analyses and evaluation

  • THE PHONETIC PROPERTIES OF SPOKEN LANGUAGE APPEAR TO BE    ORGANIZED AT THE SYLLABIC RATHER THAN AT THE PHONE LEVEL

    • Onsets are pronounced in canonical (i. e., dictionary) fashion 85-90% of the time

    • Nuclei and codas are expressed canonically only 60% of the time

    • Nuclei tend to be realized as vowels different from the canonical form

    • Codas are often deleted entirely

    • Articulatory-acoustic features are also organized in systematic fashion with respect to syllabic position

    • Therefore, it is important to model spoken language at the syllabic level

  • THE SYLLABIC ORGANIZATION OF SPONTANEOUS SPEECH CALLS INTO QUESTION SEGEMENTALLY BASED PHONETIC ORTHOGRAPHY

    • It may be unrealistic to assume that any phonetic transcription based exclusively on segments (such as the IPA) is truly capably of capturing the important phonetic detail of spontaneous material


Understanding spoken language using statistical and computational methods steven greenberg international computer scie

That’s All, Folks

Many Thanks for Your Time and Attention


Understanding spoken language using statistical and computational methods steven greenberg international computer scie

Temporal View of Language


Understanding spoken language using statistical and computational methods steven greenberg international computer scie

Linguistic Automatic Speech Recognition

  • CHARACTERIZE SPOKEN LANGUAGE WITH GREAT PRECISION

    • Currently, manual transcription is the only means by which to collect detailed data pertaining to spoken language. Computational methods are currently being developed to perform transcription automatically in order to provide an abundance of data for statistical characterization of spontaneous discourse.

  • USE THIS KNOWLEDGE TO DEVELOP COMPUTATIONAL TECHNIQUESTAILORED TO THE PROPERTIES OF THE SPEECH DOMAIN

    • A detailed knowledge of spoken language is essential for deriving a computational framework for ASR. The phonetic properties of speech are structured in different ways depending on the location within the syllable, word and phrase. Such knowledge is currently under-utilized by mainstream ASR.

  • FOCUS ON LOWER TIERS OF SPOKEN LANGUAGE FOR THE PRESENT

    • It is fashionable to emphasize the importance of “language” models (i.e., word co-occurrence properties) in ASR. However, most of the problems lie in the acoustic-phonetic front end and therefore this domain should be attacked first.

  • USE KNOWLEDGE OF HOW HUMAN LISTENERS UNDERSTAND SPOKEN LANGUAGE TO GUIDE DEVELOPMENT OF ASR ALGORITHMS

    • Current ASR acoustic models are not based on perceptual capabilities of human listeners, but on a distorted representation of what is important in hearing. It is important to perform intelligibility experiments to ascertain the identity of the truly important components of the speech signal and use this knowledge to develop robust, acoustic-front-end models for ASR.


Understanding spoken language using statistical and computational methods steven greenberg international computer scie

Linguistic ASR Research @ ICSI

  • PERCEPTUAL BASES OF SPEECH INTELLIGIBILITY

    • Human listening experiments identifying the specific properties crucial for understanding spoken language

  • MODULATION-SPECTRUM-BASED AUTOMATIC SPEECH RECOGNITION

    • Using auditory-based algorithms (linked to the syllable) for reliable ASR in background noise and reverberation

  • SYLLABLE-BASED AUTOMATIC SPEECH RECOGNITION

    • Development of a syllable-based decoder for ASR

  • STATISTICAL PROPERTIES OF SPONTANEOUS SPEECH

    • Detailed and comprehensive statistical analyses of the Switchboard corpus pertaining to phonetic, prosodic and lexical properties, used for developing pronunciation models (among other things)

  • AUTOMATIC PHONETIC LABELING AND SEGMENTATION

    • Development of (the first) automatic phonetic transcription system using articulatory-acoustic features (e.g, voicing, manner, place etc.)

  • AUTOMATIC LABELING OF PROSODIC STRESS

    • Development of (the first) automatic system for labeling prosodic stress in English

  • AUTOMATIC SPEECH RECOGNITION DIAGNOSTIC EVALUATION

    • Detailed and comprehensive analyses of Switchboard-corpus ASR systems in order to identify factors associated with word error


Understanding spoken language using statistical and computational methods steven greenberg international computer scie

Linguistic ASR at ICSI

  • SENIOR PERSONNEL

    • Steven Greenberg - Linguistic ASR, Spoken Language Statistics, Speech Perception

    • Lokendra Shastri - Neural Network Design, Higher-level Language & Neural Processing

  • GRADUATE STUDENTS

    • Shawn Chang - ANN-based ASR, Automatic Phonetic Transcription & Segmentation

    • Michael Shire - Temporal & Multi-Stream Approaches to Automatic Speech Recognition

    • Mirjam Wester - Pronunciation Modeling in Automatic Speech Recognition

  • UNDERGRADUATE STUDENTS

    • Micah Farrer - Database Development for ASR Analysis

    • Leah Hitchcock - Statistics of Pronunciation and Prosody of Spoken Language

  • TECHNICAL STAFF

    • Joy Hollenback - Statistical Analyses, Data Collection and Maintenance

  • ASSOCIATES AT ICSI

    • Hynek Hermansky, Nelson Morgan, Liz Shriberg and Andreas Stolcke

  • ASSOCIATES AT LOCATIONS OTHER THAN ICSI

    • Takayuki Arai (Sophia University, Tokyo) - Speech Perception, Signal Processing

    • Les Atlas (University of Washington, Seattle) - Acoustic Signal Processing

    • Ken Grant (Walter Reed Army Medical Center) - Audio-visual Speech Processing

    • David Poeppel (University of Maryland) - Brain Mechanisms of Language

    • Tim Roberts (UC-San Francisco Medical Center) - Brain Imaging of Language Processes

    • Christoph Schreiner (UCSF) - Auditory Cortex and Its Relation to Speech Processing

    • Lloyd Watts (Applied Neurosystems) - Auditory Modeling from Cochlea to Cortex

  • CURRENT FUNDING

    • National Security Agency - Automatic Transcription of Phonetic and Prosodic Elements

    • National Science Foundation - Syllable-based ASR, Speech Perception, Statistics of Speech


Understanding spoken language using statistical and computational methods steven greenberg international computer scie

Linguistic ASR at ICSI (continued)

  • FORMER ICSI POST-DOCTORAL FELLOWS

    • Takayuki Arai - Sophia University, Tokyo

    • Dan Ellis - Columbia University (as of September 1, 2000)

    • Eric Fosler - Bell Laboratories, Lucent Technologies

    • Rosaria Silipo - Nuance Communications

  • FORMER ICSI GRADUATE STUDENTS

    • Jeff Bilmes - University of Washington, Seattle

    • Eric Fosler - Bell Laboratories, Lucent Technologies

    • Brian Kingsbury - IBM, Yorktown Heights

    • Katrin Kirchhoff - University of Washington, Seattle

    • Nikki Mirghafori - Nuance Communications

    • Su-Lin Wu - Nuance Communications

  • FORMER ICSI UNDERGRADUATE STUDENTS

    • Candace Cardinal - Nuance Communications

    • Rachel Coulston - University of California, San Diego

    • Collen Richey - Stanford University


Understanding spoken language using statistical and computational methods steven greenberg international computer scie

Publications - Linguistic ASR

AUTOMATIC SPEECH RECOGNITION DIAGNOSTIC EVALUATION

Greenberg, S., Chang, S. and Hollenback, J. (2000) An introduction to the diagnostic evaluation of the Switchboard-corpus automatic speech recognition systems. Proceedings of the NIST Speech Transcription Workshop, College Park

Greenberg, S. and Chang, S. (2000) Linguistic dissection of switchboard-corpus automatic recognition systems. Proceedings of the ICSI Workshop on Automatic Speech Recognition: Challenges for the New Millennium, Paris.

AUTOMATIC PHONETIC TRANSCRIPTION AND SEGMENTATION

Chang, S., Shastri, L. and Greenberg, S. (2000) Automatic phonetic transcription of spontaneous speech (American English). Proc. Int. Conf. Spoken Lang. Proc., Beijing.

Shastri, L. Chang, S. and Greenberg, S. (1999) Syllable detection and segmentation using temporal flow neural networks. Proceedings of the International Congress of Phonetic Sciences, San Francisco, pp. 1721-1724.

STATISTICAL PROPERTIES OF SPOKEN LANGUAGE AND PRONUNCIATION MODELING

Fosler-Lussier, E., Greenberg, S. and Morgan, N. (1999) Incorporating contextual phonetics into automatic speech recognition. Proceedings of the International Congress of Phonetic Sciences, San Francisco.

Greenberg, S. and Fosler-Lussier, E. (2000) The uninvited guest: Information's role in guiding the production of spontaneous speech, in the Proceedings of the Crest Workshop on Models of Speech Production: Motor Planning and Articulatory Modelling, Kloster Seeon, Germany .

Greenberg, S. (1999) Speaking in shorthand - A syllable-centric perspective for understanding pronunciation variation, Speech Communication, 29, 159-176,

Greenberg, S. (1997) On the origins of speech intelligibility in the real world. Proceedings of the ESCA Workshop on Robust Speech Recognition for Unknown Communication Channels, Pont-a-Mousson, France, pp. 23-32.

Greenberg, S., Hollenback, J. and Ellis, D. (1996) Insights into spoken language gleaned from phonetic transcription of the Switchboard corpus, in Proc. Intern. Conf. Spoken Lang. (ICSLP), Philadelphia, pp. S24-27.

AUTOMATIC LABELING OF PROSODIC STRESS IN SPONTANEOUS SPEECH

Silipo, R. and Greenberg, S. (1999) Automatic transcription of prosodic stress for spontaneous english discourse. Proceedings of the International Congress of Phonetic Sciences, San Francisco.

Silipo, R. and Greenberg, S. (2000) Prosodic stress revisited: Reassessing the role of fundamental frequency. Proceedings of the NIST Speech Transcription Workshop, College Park.


Understanding spoken language using statistical and computational methods steven greenberg international computer scie

Publications - Linguistic ASR (continued)

MODULATION-SPECTRUM-BASED AUTOMATIC SPEECH RECOGNITION

Greenberg, S. and Kingsbury, B. (1997) The modulation spectrogram: In pursuit of an invariant representation of speech, in ICASSP-97, IEEE International Conference on Acoustics, Speech and Signal Processing, Munich, pp. 1647-1650.

Kingsbury, B., Morgan, N. and Greenberg, S. (1999) The modulation-filtered spectrogram: A noise-robust speech representation, in Proceedings of the Workshop on Robust Methods for Speech Recognition in Adverse Conditions, Tampere, Finland.

Kingsbury, B., Morgan, N. and Greenberg, S. (1998) Robust speech recognition using the modulation spectrogram, Speech Communication, 25, 117-132.

SYLLABLE-BASED AUTOMATIC SPEECH RECOGNITION

Wu, S.-L., Kingsbury, B., Morgan, N. and Greenberg, S. (1998) Incorporating information from syllable-length time scales into automatic speech recognition, IEEE International Conference on Acoustics, Speech and Signal Processing, Seattle, pp. 721-724.

Wu, S.-L., Kingsbury, B., Morgan, N. and Greenberg, S. (1998) Performance improvements through combining phone- and syllable-length information in automatic speech recognition, Proceedings of the International Conference on Spoken Language Processing, Sydney, pp. 854-857.

PERCEPTUAL BASES OF SPEECH INTELLIGIBILITY GERMANE TO ASR

Arai, T. and Greenberg, S. (1998) Speech intelligibility in the presence of cross-channel spectral asynchrony, IEEE International Conference on Acoustics, Speech and Signal Processing, Seattle, pp. 933-936.

Greenberg, S. and Arai, T. (1998) Speech intelligibility is highly tolerant of cross-channel spectral asynchrony. Proceedings of the Joint Meeting of the Acoustical Society of America and the International Congress on Acoustics, Seattle, pp. 2677-2678.

Greenberg, S., Arai, T. and Silipo, R. (1998) Speech intelligibility derived from exceedingly sparse spectral information, Proceedingss of the International Conference on Spoken Language Processing, Sydney, pp. 74-77.

Greenberg, S. (1996) Understanding speech understanding - towards a unified theory of speech perception. Proceedings of the ESCA Tutorial and Advanced Research Workshop on the Auditory Basis of Speech Perception, Keele, England, p. 1-8.

Silipo, R., Greenberg, S. and Arai, T. (1999) Temporal Constraints on Speech Intelligibility as Deduced from Exceedingly Sparse Spectral Representations, Proceedings of Eurospeech, Budapest.


Understanding spoken language using statistical and computational methods steven greenberg international computer scie

Syllable Frequency - Spontaneous English

The distribution of syllable frequency in spontaneous speech differs markedly from that in dictionaries


Understanding spoken language using statistical and computational methods steven greenberg international computer scie

Word Frequency in Spontaneous English

Word frequency as a function of word rank approximates a 1/f distribution, particularly after rank-order 10

Word frequency is logarithmically related to rank order in the corpus (I.e., the 10th most common word occurs ca. 10 times more frequently than the 100th most common word, etc.

Computed from the Switchboard corpus (American English telephone dialogues)


The intricate web of research

The Intricate Web of Research


  • Login