Development of a Math Screening Assessment on a Districtwide Basis

1. Development of a Math Screening Assessment on a Districtwide Basis Mike Jacobsen-Assessment and Curriculum Director Andy McGrath-Principal Glacier Middle School White River School District 360-829-3820 mjacobse@whiteriver.wednet.edu Washington Educational Research Association Annual Conference December 5-7 2007

2. By The End of This Presentation You Will: • Understand how the district implements a K-10 CBM reading assessment system • Understand how the WRSD developed a math screener • District-wide focus • Establish a committee • Pilot process • Full implementation • Fall, winter & spring data 06-07 • Next steps

3. Basic Definitions • CBM=Curriculum Based Measurement • Developed Initially at University of Minnesota Institute for Research on Learning Disabilities • Measures students progress in basic skills using existing curriculum • Psychometrically sound • ORF=Oral Reading Fluency • What is measured is students’ ability to read out loud, accurately and fluidly • DIBELS=Dynamic Indicators of Basic Early Literacy Skills • Researchers from the University of Oregon coined the phrase

4. What is CBM? • Standard, simple, short duration fluency measures of reading, spelling, written expression and mathematics computation • WRSD Reading CBM is very similar to DIBELS with one exception • WRSD Math Screener is different than DIBELS in math • In reading CBM is oral reading fluency • Measures “vital signs” of student achievement • Academic thermometer

5. Big Ideas About CBM • Extensive data supporting validity of use as a measure of basic skills • Principle use is in formative evaluation • Sensitive to changes in performance due to instruction • Easy to use within classrooms • Brief • Repeatable

7. ORF and Other Reading Tests • 1999- 3rd Grade Qualitative Reading Inventory to 3rd Grade ORF=.89 • 1999-3r Grade ITBS to 3rd Grade ORF=.64 • 1999-2th Grade Gates-MacGinitie to 2nd Grade ORF=.84 • 1999-3rd Grade Gates-MacGinitie to 3r Grade ORF=.77 • 1999-4th Grade Gates-MacGinitie to 4th Grade ORF=.64 • 1999-5th Grade Gates-MacGinitie to 5th Grade ORF=.86

8. ORF and WASL Relationships • 1998- 4th Grade WASL to 5th Grade ORF=.70 • 1999-4th Grade WASL to 4th Grade ORF=.51 • 2003-6th Grade ORF to 7th Grade WASL=.68 • 2000-4th Grade ORF to 4th Grade WASL=.66 • 2002-4th Grade ORF to 4th Grade WASL=.65

9. ORF and WASL Relationships

10. CBM ORF/WASL

12. Why Assess Computational Fluency? • “Many of the difficulties children have in arithmetic result from not understanding number ideas supposedly learning at an earlier time” • Engelhart, Ashlock & Wiebe, 1984 • “In most cases the precision and fluency in the execution of the skills are the requisite vehicles to convey the conceptual understanding.” • H. Wu, 1999

13. White River School District Assessment Process • Implemented during the 98-99 school year for K-6 Reading • 6th-8th grade added 2002 • 9th/10th grade added 2005 • Implemented during the 2006-2007 school year for 1-10 Math screener • Kindergarten students, initial sound fluency, letter names and segmenting phonemes • Grades 1-10 orally read passages from appropriate grade level material • Conducted three times per year during September, January and May

20. Background of Development of the Math Screener: District Learning Improvement Planning • Established Fall of 2005 • Approximately 30 members, teachers, building administrators, central office administrators, parents and school board members • Each building had a stipend position for a teacher who served as DLIP coordinator • Met monthly during the 05/06 school year • The first meeting was on structure and goals, research on effective schools and role of the district

21. Background: District Learning Improvement Planning • Established Fall of 2005 • Approximately 30 members, teachers, building administrators, central office administrators, parents and school board members • Each building had a stipend position for a teacher who served as DLIP coordinator • Met monthly during the 05/06 school year • The first meeting was on structure and goals, research on effective schools and role of the district

22. Background: District Learning Improvement Planning • The second meeting focused on district-wide information using the data carousel format • WASL trend data-desegregated • ITBS • CBM • Demographics • Safe and Civil Surveys • Nine Characteristics • Healthy Youth Survey • Sports and Arts program participation • Curriculum alignment • Professional development

26. Background: District Learning Improvement Planning • Used data from the carousel process to identify three major focus areas: • Professional development • Curriculum alignment-math • Math • Each focus area had co-chairs • Every member of the district learning improvement team was on one of the focus area committees • Outcome oriented

27. Math Committee • District Math TOSA Kathie Ross and Andy McGrath Co-chaired the Math Committee • Goal: To produce a math assessment that will reliably predict a student’s success on the WASL (not diagnostic) • To produce an assessment that can be given in 20-30 minutes and can be graded in a timely manner without added cost

28. Math Committee • To Achieve This Goal: • We added teachers to the committee from each level primary, intermediate, middle and high school • Committee Makeup • 3 Administrators • 7 Teachers • 1 Central Office • 2 Parents

29. Math Committee • ASSESSMENT DEVELOPMENT • Committee met for about 2 months discussing the makeup of the assessment: • Assessment Structure: • 20 Total Questions • 12 Computation • 8 Applied Problems • Single Number Answer • Reading fluency assessment already established in district • Reviewed Fuchs and Fuchs-Monitoring Basic Skills Progress-2nd Ed. • Reviewed Ken Howell’s et all- Multilevel Academic Skills Inventory-Revised • Next step split subcommittee into three groups: • Elementary • Middle • High School

30. Math Committee-Assessment Cont. • The Groups using the GLEs as a guide developed a draft assessment for each grade level • Assessments were brought back to full committee to be discussed and edited • Developed assessments for grades 2 – 10 • Assessment give three times a year in conjunction with reading assessment

31. Sample Page 4th Grade Computation

32. Sample Page 4th Grade Applied Problems

33. Pilot Process • IMPLEMENTATION • An assessment for each grade level completed by April 2005 • Piloted last May with volunteer classrooms at least two per grade level • Pilot results to Assessment Office analyze math assessment and reading fluency to see if this would be a good predictor of WASL success • If the assessment proved to be an accurate predictor of WASL success then implement district wide Fall 06

34. Pilot Process • Manila envelope provided to each pilot teacher • Directions for Administration • Instructions for Scoring • Student Response Sheets • Test Key • Copies of student response forms provided to each teacher • 624 students grades 1-8 participated • Statistically strong relationships with WASL math and spring oral reading fluency demonstrated

35. Pilot Results