1 / 7

Classifying Reading Levels Using Statistical Language Models: A Study by Johnson and Hsieh

This study explores the classification of English text passages into appropriate reading levels using statistical language models. We analyze a dataset comprised of English novels across various grades, specifically targeting 5th-12th grade materials. Two models were developed: a generative language model for initial classification and a discriminative multinomial logistic regression model that incorporates additional features. Results indicate that statistical models significantly improve accuracy over traditional measurements. Future research aims to enhance performance through higher-order language models and address potential overfitting.

nerice
Download Presentation

Classifying Reading Levels Using Statistical Language Models: A Study by Johnson and Hsieh

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Classifying Reading Levels with Statistical Language Models Johnson Hsieh Sameer Shariff

  2. Problem • Given a passage of English text, can we classify the text at the appropriate reading level? • Dataset – English novels from various reading lists • 5th-7th grade - 15 books • Tom Sawyer, Black Beauty, etc. • 8th-10th grade - 16 books • A Tale of Two Cities, The Call of the Wild, etc. • 11th-12th grade - 17 books • Pride and Prejudice, The Awakening, etc.

  3. Approach • Build language models for each class • Classify new text based on model that was most likely to generate this text (generative model) • Model 1 • Classify text based purely on these language models with some interesting smoothing techniques • Model 2 • Build a discriminative multinomial logistic regression model that uses these language models as just one of many features

  4. Data Separability • A hard problem

  5. Language Model Results Accuracy = (# of books predicted correctly)/(total # of books) Weighted Accuracy = ((# predicted correctly) + 0.5 * (# off by one))/(total # of books)

  6. Multinomial Logistic Regression Results • Without language model: • With language model:

  7. Conclusions and Future Work • Statistical language models do capture information that can help differentiate between different reading levels, better than traditional measures such as Flesch-Kinkaid • Multinomial logistic regression models with additional features outperform the pure language model approach, though using the language model as a feature greatly improves performance • Future Work • Explore higher order language models • Investigate language model overfitting

More Related