evaluating the effect of predicting oral reading miscues n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Evaluating the Effect of Predicting Oral Reading Miscues PowerPoint Presentation
Download Presentation
Evaluating the Effect of Predicting Oral Reading Miscues

Loading in 2 Seconds...

play fullscreen
1 / 11

Evaluating the Effect of Predicting Oral Reading Miscues - PowerPoint PPT Presentation


  • 87 Views
  • Uploaded on

Evaluating the Effect of Predicting Oral Reading Miscues. Satanjeev Banerjee, Joseph Beck, Jack Mostow Project LISTEN (www.cs.cmu.edu/~listen) Carnegie Mellon University Funding: NSF IERI. Why Predict Miscues?. Reading Tutor helps children learn to read.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Evaluating the Effect of Predicting Oral Reading Miscues' - taima


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
evaluating the effect of predicting oral reading miscues

Evaluating the Effect of Predicting Oral Reading Miscues

Satanjeev Banerjee, Joseph Beck, Jack Mostow

Project LISTEN (www.cs.cmu.edu/~listen)

Carnegie Mellon University

Funding: NSF IERI

slide2

Why Predict Miscues?

  • Reading Tutor helps children learn to read.
  • Speech recognizer listens for miscues(reading errors)
    • E.g.: listen for “hat” if sentence to be read has word “hate”
  • Accurate miscue prediction helps miscue detection.
real word substitutions
Real Word Substitutions
  • Miscues = substitutions, omissions, insertions
  • Real word substitution = misread target word as another word
    • E.g. read “hat” instead of “hate”
  • Most miscues are real word substitutions
  • ICSLP-02: predicted real word substitutions
  • Here: evaluate effect on substitution detection
how evaluate substitution detection

# substitutions detected

Substitution detection rate =

# substitutions child made

1

1

4

2

How Evaluate Substitution Detection?

substitution

substitution undetected

false alarm

substitution detected

=

# false alarms

False alarm rate =

=

# words correctly read

evaluation data
Evaluation Data
  • Sentences read by 25 children aged 6 to 10
rote method
Rote Method
  • Uses the University of Colorado miscue database.
  • For each target word
    • Sort substitutions by # children who made them.
    • Predict that the top n substitutions will reoccur, for this word.
extrapolative method
Extrapolative Method
  • Predict the probability that a word is a likely substitution for another word
    • Pr ( substitution “hat” | target “hate”)
  • Use machine learning to induce a classifier
  • Train using University of Colorado miscue database.
extrapolative method cont d
Extrapolative Method cont’d

Given a target word, predict substitution if

Pr ( substitution candidate | target word ) > threshold

combining rote and extrapolative
Combining Rote and Extrapolative
  • Aim: Get n substitutions for a given word.
  • Step 1: Use top n substitutions from rote.
  • Step 2: If rote predicts k substitutions, k < n,
    • Then add top n – k substitutions from extrapolative.
  • Intuition: rote is more accurate, so use when available. If not available, fall back on extrapolative.
results from combining algorithms
Results from Combining Algorithms

Truncation = The first 2 to n-2 phonemes of a word – models false starts. [/K AE/ and /K AE N/ for /K AE N D IY/; none for “hate”]

Theoretical max = use only those miscues the child actually made.

conclusion
Conclusion
  • Evaluated effect on substitution detection of
    • Two previously published algorithms
    • A combination of the two algorithms.
  • Combined approach improved on current configuration (truncations) by
    • Reducing false alarms by 0.52% abs (12% rel)
    • Increasing miscue detection by 1.04% (4.2% rel)
  • Take-home sound byte: Listening for specific reading mistakes can help detect them!