1 / 37

Uses for Automatic Speech Recognition with Diverse English Speakers

Uses for Automatic Speech Recognition with Diverse English Speakers. 2002 American Speech-Language-Hearing Association Annual Convention Atlanta, Georgia World Congress Center, Room: A314, Saturday, Nov 23 2002 4:30PM – 5:30PM

mckenzie-norton
Download Presentation

Uses for Automatic Speech Recognition with Diverse English Speakers

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Uses for Automatic Speech Recognition with Diverse English Speakers 2002 American Speech-Language-Hearing Association Annual Convention Atlanta, Georgia World Congress Center, Room: A314, Saturday, Nov 23 2002 4:30PM – 5:30PM Presenters/Authors: Kathleen Eilers Crandall, Ph.D., Paula M. Brown, Ph.D., Donna E. Gustina, and Stephen S. Campbell National Technical Institute for the Deaf Rochester Institute of Technology

  2. Seminar – Presenters Kathleen Eilers crandall, Ph.D. Department of English, National Technical Institute for the Deaf, Rochester Institute of Technology Paula M. Brown, Ph.D., CCC-SLP Department of Speech and Language, National Technical Institute for the Deaf, Rochester Institute of Technology

  3. The Glossograph • Fay wrote about an experimental mechanical device used to transcribe human speech, and said, • “… it is not unreasonable to hope that some instrument will yet be contrived …“ Fay, E.A. (1883). The glossograph. American Annals of the Deaf, 28, 67-69.

  4. Sci-Fi or Reality? "The pen was an archaic instrument, seldom used even for signatures...Apart from very short notes, it was usual to dictate everything into the speak-write…” (Nineteen eighty-four.Orwell, 1949)

  5. Two Projects • Teacher use of ASR: • English Classroom/Lab Project • Student use of ASR: • Speech Project Funded by a grant from the Parsons Foundation of California

  6. English Classroom/Lab Project

  7. English Classroom/Lab ProjectPurpose Investigate direct use of ASR by classroom teacher to learn: • Is acceptable recognition level attained? • Under what conditions? • Style of speaking • Communication mode • Language complexity

  8. Use of ASR by an intermediary Intermediary, a ‘captionist,’ re-speaks professor’s words into a computer Intermediary summarizes professor’s words into a computer (‘interpreted speech’) Intermediary may use C-print (a shorthand typing system) in combination with ASR http://cprint.rit.edu/ Related Work

  9. Related Work Use of ASR by the primary speaker • iCommunicator™ http://www.myicommunicator.com/product_info.html • Liberated Learning Environment http://www.liberatedlearning.com (St. Mary’s University, Halifax, Nova Scotia)

  10. Speech Project

  11. Speech ProjectIntent • Can ASR become better than a naïve listener? • Can ASR serve as an effective and motivating feedback system?

  12. Speech ProjectHow ASR Is Used Educationally Visual displays provide feedback regarding speech production • Natural way of learning • Expect feedback to reflect accuracy • Assume if don’t get right picture, you were wrong

  13. English Classroom/Lab Project

  14. English Classroom/Lab ProjectTeacher -- Students • Teacher -- Speaker • Native speaker of American English • User of ASL as a second language • Trained the ASR equipment • Students -- Readers • Young adult college students who are deaf or hard-of-hearing • Reading and writing skills at the lowest quartile of entering students • Enrolled in basic level English language reading and writing courses

  15. English Classroom/Lab ProjectEvaluation Procedures • ASR Software: • Dragon Naturally Speaking • IBM ViaVoice • Microsoft Office • Speaking styles: • Spontaneous conversation • Dictation-like speech • Communication modes: • Speaking • Simultaneously speaking and signing

  16. English Classroom/Lab Teacher stationControl systemSmart Board & LCD Projector Student Stations

  17. Vary by population and message predictability New vs. Known information Fluent readers vs. Language learners Reading for pleasure vs. Reading to master new information CLOZE research and prediction of missing information English Classroom/Lab ProjectAccuracy Needs

  18. English Classroom/Lab ProjectResults: ASR Software

  19. English Classroom/Lab ProjectResults: Communication Mode

  20. English Classroom/Lab ProjectResults: Language Complexity

  21. English Classroom/Lab ProjectCorrecting Text • Error correction • What to correct • When to correct • How to correct

  22. Multitasking Demands • Normal tasks for speaker/teacher • Formulating ideas relevant to topic • Attending to learning needs of students • Meeting lipreading and sign language needs • Added tasks for speaker/teacher • Speaking to produce readable ASR text • Monitoring text • Making corrections

  23. Speech Project

  24. Speech ProjectTraining Sequence • Read a paragraph • Correct and train recognition errors • Reread paragraph • Correct and train recognition errors • Create transfer paragraph or spontaneous speech • Correct and train recognition errors

  25. Recognition Accuracy

  26. Improvement Across Sessions

  27. Improvement Within Session

  28. Speech ProjectImprovement Evaluated • Improvement across sessions • Improvement within a session • Improvement with speaker training • Improvement with ASR training

  29. RecommendationsDiscussionQuestions

  30. Grammatical Correctness • Is ASR accuracy affected by the grammatical correctness of the user’s speech? • Student written responses spoken as written: Accuracy – 93.8% • Student written responses spoken after corrected: Accuracy - 94.3%

  31. Style of Speaking • Style of speaking that more closely resembles dictation approaches a usable accuracy rate. • Lowering the complexity does not improve accuracy.

  32. Conditions of Use Direct use of ASR by a language teacher -- Useful only under very controlled conditions. • Illustrating the generation of written language • Demonstrating the use of notes and outlines to produce written text • Translating selected sign language utterances into English text during discussions

  33. ASR: Classroom Use Prepared Outline Teacher’s Screen Student’s Screen

  34. Considerations • Training • Critical to reach over 90% accuracy • Training with conversation • Corrections • Familiarity with strategies • Dictate, Spell, Right click • Equipment • Microphone headsets - design, comfort, and size • Demand on computer processor • Effect of optional settings

  35. Teaching/Learning Issues: Does ASR promote the learning of reading and writing for Deaf and Hard-of-Hearing students? How do students process this information? Do students attend to multiple inputs? Can teachers attend to this many tasks effectively? Language Processing

  36. More Questions • Who is at fault? • Speaker or ASR receiver? • Acceptability of input • Various voices • Nontypical speakers • User friendliness • Want immediate use

  37. Kathleen Eilers Crandall, Ph.D. Department of English National Technical Institute for the Deaf Rochester Institute of Technology Lyndon Baines Johnson Building - 2264 Phone: (585) 475-5111 Fax: (585) 475-6500 Email: kecncp@rit.edu Web: http://www.rit.edu/~kecncp Paula M. Brown, Ph. D., CCC-SLP Department of Speech and Language National Technical Institute for the Deaf Rochester Institute of Technology Lyndon Baines Johnson Building - 3851 Phone: (585) 475-6593 V/TDD Fax: (585) 475-6500 Email: pmbnci@rit.edu Web: http://www.rit.edu/~462www/ Presenters

More Related