1 / 22

Pre-text revisited: Do we understand how writing prompts function?

Pre-text revisited: Do we understand how writing prompts function?. Mark Chapman. Contents. The testing context Prompt effect literature Prompt categorization The dataset - essays and discourse variables Analyses of essays by prompt Significant differences in essays by prompt

tress
Download Presentation

Pre-text revisited: Do we understand how writing prompts function?

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Pre-text revisited: Do we understand how writing prompts function? Mark Chapman

  2. Contents • The testing context • Prompt effect literature • Prompt categorization • The dataset - essays and discourse variables • Analyses of essays by prompt • Significant differences in essays by prompt • Implications for writing test design

  3. Writing test • Select 1 prompt from 2 • 30 minutes to compose a response • Scored independently by 2 raters

  4. Prompt effect in writing assessment • Effect of prompt wording on score awarded • Leu, Keech, Murphy & Kinzer, 1982 • Brossell & Ash, 1984 • Effect of prompt wording on discourse variables • Greenberg, 1981 • Hirokawa & Swales, 1986 • Effect of different integrated prompts on discourse variables • Cumming et al., 2005 • O’Loughlin & Wigglesworth, 2007

  5. Problems with previous studies • Used holistic score as measure of prompt effect • Compared prompts that were very similar • Had small sample sizes • Analyzed stimuli differences not prompt • Little prompt categorization

  6. Research Questions • What are the distinguishing characteristics of independent writing prompts? • How do these characteristics affect the test-takers’ final written products?

  7. Writing prompt characteristics • Applied previous prompt taxonomies • Swales, 1982; Horowitz, 1991; Lim, 2009

  8. The dataset

  9. Analyses • Factor structure of discourse variables (DVs) • Minimize number of MANOVA analyses to reduce risk of Type I error • MANOVA analyses of DV differences in responses elicited by each prompt

  10. Factor Analysis

  11. Factor structure

  12. MANOVA – lexical sophistication

  13. MANOVA - cohesion

  14. MANOVA - accuracy

  15. MANOVA – academic vocabulary

  16. MANOVA – syntactic complexity

  17. ANOVAs – fluency; writing score; temporal cohesive markers

  18. Little evidence of significant differences in . . . • Holistic score awarded • Accuracy

  19. Significant differences in . . . • Lexical sophistication • Cohesion • Academic language use • Syntactic complexity • Fluency

  20. Prompts responsible for most differences

  21. Implications for writing assessment • Score interpretation • Construct definition • Consider which domains are appropriate • Consider permissible response modes • Consider number of rhetorical cues

  22. Thank you For more information, please contact chapman.m@cambridgemichigan.org

More Related