1 / 22

You’re Not From ‘Round Here, Are You? Naïve Bayes Detection of Non-native Utterance Text

You’re Not From ‘Round Here, Are You? Naïve Bayes Detection of Non-native Utterance Text. Laura Mayfield Tomokiyo Rosie Jones Carnegie Mellon University. Overview. Motivation Speech data Accent detection as document classification Classification performance Discriminative tokens

tavia
Download Presentation

You’re Not From ‘Round Here, Are You? Naïve Bayes Detection of Non-native Utterance Text

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. You’re Not From ‘Round Here, Are You? Naïve Bayes Detection of Non-native Utterance Text Laura Mayfield Tomokiyo Rosie Jones Carnegie Mellon University

  2. Overview • Motivation • Speech data • Accent detection as document classification • Classification performance • Discriminative tokens • Conclusions

  3. Non-native speech recognition The warship U.S.S. Jarrett has pulled into port in San Diego, CA after training voyage Native recognizer (word accuracy = 26.7): Tomorrow CPU a sister at has spilled into port and sandy and afford after a training wage Non-native recognizer (word accuracy = 73.3): The worst eighty U.S.S. chart has pulled into port in San Diego California after training warrior

  4. Motivation • Practical • can we detect non-native users with enough accuracy to switch acoustic models? • Exploratory • how well does an algorithm based only on text features work? • what tokens are discriminative for non-native speakers?

  5. Speech examples Read speech Over the next two months, public officials, Native American leaders, businesses and environmental groups will come up with plans for meeting the law’s requirements. Spontaneous speech Local specialties I like to have anything very special in Boston, very native in Boston.

  6. Speech data

  7. Transcripts and hypotheses “A safety net for salmon: environmentalists, the government, and ordinary folks team up to save the Northwest’s wondrous wild salmon” A safety net for the salmons Environment= environmentalists… A safety net forced simon Um environmental activists… Classification based on transcripts: Classification based on hypotheses: • Usually gives a good idea of gold standard • Finds true differences in linguistic usage • Implicitly models acoustics • Benefits from amplified difference between native and non-native samples

  8. Related work • Acoustic feature based accent discrimination (e.g. Fung and Liu 1999) • Competing HMM based accent discrimination (e.g. Teixeira et al 1996) • Classification of documents according to style (Argamon-Engleson et al 1998), author (Mosteller and Wallace 1964)

  9. Accent detection as document classification Native speaker utterances Classifier Non-native speaker utterances

  10. Accent detection as document classification Test speaker utterances Classifier Classification decision: native or non-native?

  11. Experimental methodology • Rainbow naïve Bayes classifier • Both word and part-of-speech tokens were examined • Classification based on token unigrams and bigrams • No feature selection initially • Stopwords were not excluded from feature set • Data randomly split into 30% testing, 70% training data for evaluation; evaluation repeated 20 times and classification results averaged • Utterances from the same speaker never appeared in both training and test sets

  12. Classification of spontaneous speech (transcripts only) Native/ Japanese Native/ Chinese Japanese/ Chinese Native/ Non-native Native/ Japanese/ Chinese

  13. baseline Classification of read speech A train: same texts test: same texts

  14. Classification of read speech baseline A train: same texts test: same texts B train: disjoint texts test: disjoint texts C train: disjoint texts test: same texts D train: same texts test: disjoint texts

  15. Classification of read speech baseline A train: same texts test: same texts B train: disjoint texts test: disjoint texts C train: disjoint texts test: same texts D train: same texts test: disjoint texts

  16. Feature Selection

  17. Discriminative sequences transcriptions hypotheses

  18. Conclusions • Transcriptions of spontaneous speech can be classified with high accuracy for both 2-way and 3-way distinctions • Read speech samples, which are simple transformations of native-produced text, can be classified with high accuracy • Recognizer output is classified more accurately than transcripts

  19. Future directions • Incorporating the classification decision in acoustic model selection • Minimizing the number of samples from the test speaker needed for classification • Applying classification to parsing grammar selection, language model construction, writer identification

  20. Discriminative POS sequences

  21. Discriminative word sequences

  22. Phone-based classification Discriminative tokens Condition B

More Related