Generalization of RtI Procedures to Written Language Merilee McCurdy, Ph.D. University of Nebraska - Lincoln

1. Generalization of RtI Procedures to Written Language Merilee McCurdy, Ph.D. University of Nebraska - Lincoln

2. Description From Program As schools begin to implement RtI procedures, it is important to realize that RtI procedures can be used with all academic areas, including writing and mathematics. This presentation will review writing assessment issues such as norming writing skills, universal screening procedures, and progress monitoring techniques. In addition, writing interventions that can be used within RtI frameworks will be identified for each writing skills area.

3. Research in Reading To date, much of the literature and research in RtI has been done in the area of reading. Why? • Research has shown that those children who become adults with low levels of literacy are at a disadvantage in a society that has high demands for effective literacy skills in the work place. Torgeson, J.K. (2000) • There has not been a significant change in the reading performance of students from 1992 to 2005 (National Center for Educational Statistics, 2003). • A 1997 study documented that at least 20% of the population in the United States had reading difficulties (Lyon & Moats, 1997).

4. RtI and Reading • Research has focused efforts on understanding reading deficits and identifying effective reading programs or interventions. • This concentration of RtI procedures in the area of reading has been appropriate as the field seeks to understand and develop initial RtI procedures. • Unfortunately, this intense focus has not been similar in other academic areas, especially for written language.

5. The Significance of Writing • Writing has become the neglected element of school reform (National Commission on Writing, 2004) • The Commission asserts that students must “struggle with details, wrestle with facts, and reword raw information and dimly understood concepts into language they can communicate to someone else.” • In short, students must write if they are to learn. The National Commission on Writing. (2003). The neglected “R”: The need for a writing revolution.

6. The Significance of Writing • The National Commission on Writing for America’s Families, Schools, and Colleges (2004) described writing as a threshold skill for employment and promotion and indicated that people who cannot write well are less likely to be hired, retained, and/or promoted. National Commission on Writing for America’s Families, Schools, and Colleges. (2004). Writing: A ticket to work…or a ticket out. NY: The College Board • Writing skills are often needed for demonstrating learning (e.g., responding to exam items) and communicating across a variety of employment settings.

7. The Significance of Writing:Percentage of students, by writing achievement level, grades 4, 8, and 12: 1998 and 2002

10. RtI and Writing Schools appear to be having difficulty preparing students to meet society’s demands in the area of writing. The RtI process can assist with: • Identifying students who are at-risk in the area of writing • Providing or developing intensive interventions in the area of writing • Monitoring weekly progress of student progress • Using data to make decisions regarding students’ needs following a predetermined length of time

11. RtI and Writing • In many ways, RtI procedures developed for the area of reading can be easily generalized to writing • However, due to the lack of research on written language, many questions remain.

12. Questions Questions regarding RtI and writing assessment: • How much writing time is needed for a valid evaluation of writing? • Is writing performance stable across assessments? • How can story organization and story development be evaluated using CBA techniques? • Which variables should be used for initial decision making? • Which variables should be used for ongoing decision making? • Are some assessment variables useful independently or in a combination with others? • What type of writing prompts should be used?

13. Questions Questions regarding RtI and writing interventions: • How are Tier 1 services evaluated? • What are empirically based interventions for writing? • Can interventions be delivered in small groups? • How much intervention time is required to observe changes? • Which variables are more reliable and valid for progress monitoring? • What gains should be expected? • How can students be motivated to write?

14. RtI Writing Team: Lincoln Public Schools • What we have achieved so far…. • Established an RtI Writing Team (2004) • Collected normative data • Grades K-2 (2005-2006) • Grades 3-5 (2006-2007) • Initiated small pilot studies

15. RtI Writing Team: Lincoln Public Schools • Future Plans…. • 2007-08 • Pilot K-5 writing procedures • Small pilots at high school and middle schools to determine normative data collection procedures • Evaluate research based interventions • Train others to score writing probes (reliably) • 2008-2009 • Implement RTI writing district-wide at elementary schools • Begin norming at middle schools • others will score if meet criteria for training • 2009-2010 • ?????

16. Core Components for Quality RtI Implementation • School-wide buy-in and implementation plan • Team Leadership • Integration of Services • Implementation Infrastructure • Parent Involvement • Universal Screening and Assessment ** • Individual Progress Monitoring ** • Planned Service Delivery Decision Rules • Scientifically Supported Instruction ** • Intervention Delivery • SLD Verification

17. Assessment • Curriculum-Based Measurement (CBM) • Content of the assessments are based on the instructional curriculum. • Measures are presented in a standardized format. • Material for assessment is controlled for difficulty by grade levels. • Measures are generally brief. Shapiro (2004) • Development of Norms • Universal Screening • Progress Monitoring

18. General Assessment Procedures for Writing • The student is given a “story starter” in the form of a starting sentence or partial sentence • Provides the student with an idea to write about • The student is asked to think about their story for one minute and to write for three minutes • At the end of the writing period, the examiner collects the student’s writing

19. Differences between Reading and Writing CBM Reading • Takes one minute • Administered individually • Must be scored concurrently with performance Writing • Takes four minutes • Can be administered in groups • Permanent products allow for later scoring

20. Other data to consider • Classroom writing products • Classroom writing assessments/rubrics • Statewide writing assessments • Reading performance • Motivation!

21. Development of Norms • LPS Normative Procedures • N=250 per grade; randomly selected by district • One story starter administered at fall, winter, spring (different starter each period) • Same starter used with each grade (1st/2nd) • Kindergarten • Letter writing (fall and winter) • Story starter in spring • Used a “scoring team” to increase reliability

22. NORMATIVE VARIABLES BY GRADE TLW Total Letters Written TWW Total Words Written CWS Correct Writing Sequences CMIWS Correct minus Incorrect Writing Sequences %CWS Percent Correct Writing Sequences

23. Universal Screening The practice of assessing all students to identify those who are not making academic or behavioral progress at expected rates. National Association of State Directors of Special Education, Inc. (2005) A classroom-wide, school-wide, or district-wide assessment used to identify students who are at risk for academic failure or behavioral difficulties and could potentially benefit from specific instruction or intervention. A critical prerequisite to providing early school-based prevention and intervention services for students at risk for or with academic, behavioral, or emotional difficulties. Glover & Albers (2007)

24. Universal Screening “Assessments are in place that provide teachers with information about the progress of all students to determine which students need closer monitoring, additional intervention, and/or adjusted learning opportunities” • Norms are necessary to (1) identify students who need additional supports, (2) develop growth expectations, and (3) set entrance and exit criteria. • Develop norming procedures based on school resources

25. Universal Screening • Scoring: It doesn’t take that long! • Requires between 22 and 31 seconds to score TWW • Requires between 25 and 37 seconds to score WSC • Requires between 57 and 151 seconds to score CWS • If all three scores are used, it takes between 1 ½ minutes and 2 ½ minutes to score one writing sample. Gansle, Noell, VanDerHeyden, Naquin, & Slider, 2002 Malecki & Jewell, 2003

26. Difficulties regarding RtI Writing and Universal Screening • Several components of writing to consider: • Spelling • Grammatical usage • Mechanics of writing • Numerous variables to consider!! • It has not yet been determined which variables: • are most sensitive to change over time • are most valid and reliable for a given grade level

27. Production-dependent Correctly Written Words Total Words Written Correct Writing Sequences Mature Words Letter Sequences Correct Total Words Spelled Correctly Production independent % Correct Writing Sequences % Correctly Spelled Words Mean Length of Correct Writing Sequences Writing Variables A measure of both fluency and accuracy = Correct Minus Incorrect Writing Sequences (CMIWS), an accurate-production measure

28. How to set growth expectations? • Low and variable levels of growth on many variables. • From AIMSweb norms: Data represent growth across entire normative sample

29. Progress Monitoring The practice of assessing students to determine if academic or behavioral interventions are producing desired effects. National Association of State Directors of Special Education, Inc. (2005) Provides critical information about student progress that is used to ensure the use of effective educational practices and to verify that students are progressing at an adequate rate. “Assessments are in place that provide teachers with information about which students are benefiting from intervention and which need additional supports.”

30. Progress Monitoring Procedures 1. Based upon the norms you have decided to use and each student’s screening results, set a goal for each student. • This goal should reflect an average gain per week as determined by the normative data. 2. Once the student’s intervention has begun, monitor the student’s progress at least once per week. For writing, may want to include more than 1 writing sample. 3. Graph the student’s scores on a chart.

31. Progress Monitoring Procedures 4. Review the chart weekly to determine whether progress is being made. 5. After the student has been in an intervention for a specified amount of time: • Look at the level and the rate of progress • Determine whether the goal was attained • Plan for next steps • These are general RtI procedures and should be the same for regardless of academic area!!

32. The Complexity of Writing:Sample Student Data

33. Further Complexity of Writing

34. Advantages of Writing Variables • Although numerous variables add to the complexity of assessment, they aid in identifying specific difficulties. • Awareness of specific difficulties further assists in developing/identifying interventions for the next phase of intervention (if necessary)

35. Scientifically Supported Instruction “Core instruction and interventions have been validated through scientific research” Must demonstrate that districts have a process for identifying interventions with scientific evidence of effectiveness.

36. Do Not Forget Tier 1 Tier 1 must be evaluated to examine effectiveness of curriculum for most students. This can be accomplished through universal screening. “When teachers use a scientifically supported curriculum and appropriate instructional practices, they are preventing many problems from occurring.” Nebraska Department of Education Technical Assistance Document (2006)

37. Interventions in RtI • Basic Message: Intervention delivery for writing is very similar to reading procedures. However… MORE RESEARCH IS NEEDED ON INTERVENTIONS THAT CAN BE USED IN THE SCHOOLS!!

38. What makes a good Writer? • Writing extends beyond mastering grammar, punctuation and the ability to identify parts of speech. • Writing is best understood as a complex intellectual activity. • It requires students to “stretch their minds, sharpen their analytical capabilities, and make accurate and valid distinctions.” (National Commission on Writing) • Writing is a way for students to demonstrate what they know AND a way to help them understand what theyknow.

39. Areas for Intervention • Three primary writing stages: • Prewriting • Planning • Story Development • Idea Development • Writing • Story Organization • Grammar • Spelling • Fluency • Post Writing • Editing/Revising

40. Challenges to improving students’ writing • Time for writing in and out of the classroom • Assessment or measurement of progress and results • Integrating technology into the teaching and learning of writing • Support for teaching and other classroom issues The National Commission on Writing. (2003). The Neglected “R”: The need for a writing revolution.

41. Characteristics of Strong Writing Instruction • Writing Strategies - involves teaching students strategies for planning, revising, and editing their compositions • Summarization - involves explicitly and systematically teaching students how to summarize texts. • Collaborative Writing - uses instructional arrangements in which adolescents work together to plan, draft, revise, and edit their compositions. • Specific Product Goals - assigns students specific, reachable goals for the writing they are to complete. • Word Processing - uses computers and word processors as instructional supports for writing assignments. • Sentence Combining - involves teaching students to construct more complex, sophisticated sentences. • Prewriting - engages students in activities designed to help them generate or organize ideas for their composition. • Inquiry Activities - engages students in analyzing immediate, concrete data to help them develop ideas and content for a particular writing task. • Process Writing Approach - interweaves a number of writing instructional activities in a workshop environment that stresses extended writing opportunities, writing for authentic audiences, personalized instruction, and cycles of writing. Effect Size above .50

42. Tier 2 and 3 Instruction: Identifying Target Skills • Generate hypotheses based on accuracy or fluency concern: • Questions to ask when identifying target skill: • Does the student have letter identification and letter formation skills? • Can the student produce legible text? • Did the student use the planning time? • Does the student produce an adequate amount of text? • Does the student produce a grammatically correct story? • Does the student understand story organization? • Could motivation be impacting performance?

43. Classwide Fluency Strategies • Classwide and individual performance feedback strategies are very successful at increasing student writing production. • Students must practice writing to improve writing skills!

44. Building Effective Interventions • Writing Program composed of: • Direct Instruction • Goal Setting • Writing Practice • Feedback • Individual Feedback • Charting • Rewards McCurdy, M., Skinner, C. H., Watson, T. S., & Shriver, M. D. (accepted). Examining the effects of group contingencies on the writing performance of middle school students with learning disabilities in writing. School Psychology Quarterly.

45. Evaluating Published Programs • Examine for: • Technical Adequacy • Research Support • Grade appropriateness/match to need • Relationship to 6 Traits of Writing • Price

47. Intervention Resources • Strategic Instruction Model (SIM) • Center for Research on Learning • http://www.ku-crl.org/sim/strategies.shtml • National Center on Accelerating Student Learning • http://kc.vanderbilt.edu/casl/index.html • Mnemonics for Planning and Editing • Cognitive Strategy Instruction; Dr. Bob Reid – UNL • http://www.unl.edu/csi

48. Intervention Delivery: Resources! • Interventions can be delivered in small groups • Behavior management procedures will be necessary • Motivation and engagement is a concern Consider and evaluate “comprehensive” reading programs • These programs may also impact writing performance • May serve as the first supplementary intervention • Must monitor for improvement or lack of improvement

49. LPS Goals of Pilot Year (2007-2008) • Identify any modifications to district RtI procedures • Identify appropriate assessment variables for progress monitoring and decision making • Incorporate district assessment data into decision making structure • Establish structured decision making criteria for measuring adequate progress or referral to special education • Locally validate multiple interventions for use with writing concerns • Prepare for the future • Norming at upper levels (3, 5, or 10 minute writing times?) • Procedures at upper levels • Interventions for writing skills in the upper grades

50. References Daly, E., Glover, T., McCurdy, M. (2006). Response to intervention: Technical assistance document. Lincoln, Nebraska: Nebraska Department of Education. Gansle, K. A., Noell, G. H., VanDerHeyden, A. M., Naquin, G. M., & Slider, N. J. (2002). Moving beyond total words written: The reliability, criterion validity, and time cost of alternative measures for curriculum-based measurement in writing. School Psychology Review, 31, 477- 497. Glover, T. A., & Albers, C. A. (2007). Considerations for evaluating universal screening assessments.Journal of School Psychology, 45, 117-135. Graham, S., & Perin, D. (2007). Writing next: Effective strategies to improve writing of adolescents in middle and high schools - A report to Carnegie Corporation of New York. Washington, DC: Alliance for Excellent Education. Lyon, G.R. & Moats, L.C. (1997). Critical conceptual and methodological considerations in reading intervention research. Journal of Learning Disabilities, 6, 578-588.