Evaluation Results 2002-2004

1. Missouri Reading Initiative Evaluation Results 2002-2004

2. MRI’s Evaluation Activities: • Surveys *Teacher Beliefs and Practices (pre/post) Annual Participant Questionnaire • *Data Collection Test Scores Standardized Tests Classroom Assessments (DRA) MAP Demographics Special Education Information • MAP Analyses *For schools beginning in 2002

3. MAP ANALYSES: Map analyses compare schools that have finished the MRI program with randomly chosen samples of non-MRI elementary schools Results indicate MRI schools generally outperform non-MRI schools

4. Notes for MAP Analyses • Note: With all the following MAP Analyses charts the numbers are not as important as the comparative performance between MRI and non-MRI schools. This is because: • There is a lot of variation in the data from year to year and school to school • The calculation of the baseline changes as more data becomes available: • For the 2002 schools 1999 was the baseline • For the 2003 schools an average of 1999/2000 was the baseline • For the 2004 schools an average of 1999/2001 was the baseline

5. Notes for Chart 1 • This chart compares MRI and non-MRI schools performance on the 2002 MAP Communication Arts Index • Baseline year is 1999; Outcome year is 2002 • Each sample has 15 schools = number of schools that finished MRI Spring 2002 • Total random sample = 150 (large enough number to satisfy statistical significance, high confidence levels)

7. Notes for Chart 2 • This chart compares MRI and non-MRI schools performance on the 2002 MAP Reading Index • Baseline year is 1999; Outcome year is 2002 • Each sample has 15 schools = number of schools that finished MRI Spring 2002 • Total random sample = 150 (large enough number to satisfy statistical significance, high confidence levels)

9. Notes for Chart 3 • This chart compares MRI and non-MRI schools performance on the 2003 MAP Communication Arts Index • Baseline year is an average of 1999/2000 (smoothes out variations); Outcome year is 2003 • Each sample has 20 schools = number of schools that finished MRI Spring 2003 • Total random sample 200 (large enough number to satisfy statistical significance, high confidence levels)

11. Notes for Chart 4 • This chart compares MRI and non-MRI schools performance on the 2003 MAP Reading Index • Baseline year is an average of 1999/2000 (smoothes out variations); Outcome year is 2003 • Each sample has 20 schools = number of schools that finished MRI Spring 2003 • Total sample 200 (large enough number to satisfy statistical significance, high confidence levels)

13. Notes for Chart 5 • This chart compares MRI and non-MRI schools performance on the 2004 MAP Communication Arts Index • Baseline year is an average of 1999/2001 (smoothes out variations); Outcome year is 2004 • Each sample has 27 schools = number of schools that finished MRI Spring 2004 • Total random sample 270 (large enough number to satisfy statistical significance, high confidence levels)

15. Notes for Chart 6 • This chart compares MRI and non-MRI schools performance on the 2004 MAP Reading Index • Baseline year is an average of 1999/2001 (smoothes out variations); Outcome year is 2004 • Each sample has 27 schools = number of schools that finished MRI Spring 2004 • Total sample 270 (large enough number to satisfy statistical significance, high confidence levels)

17. Adequate Yearly Progress • As mandated by federal law, Missouri schools must make yearly progress goals in MAP scores • For 3rd Grade Communication Arts those goals were defined as 19.4% of students achieving levels of Proficient or better in 2003, and 20.4% for 2004. The following Table provides a comparison between MRI schools and state-wide results.

18. Percentage of Schools Meeting AYP Levels2003=19.4% 2004=20.4%Proficient and Advanced

19. Participant Survey • Participants rate the usefulness of component utilization,practice change, "buy in", attitudes toward the program and trainer, etc. • Results drive program change; e.g., Orientation

20. Notes for Participant Survey Slide • This slide introduces the survey and its uses. The table in the next slide demonstrates how the survey is often used. In this case: • 2002 survey respondents identified the problem of being “overwhelmed” • Program responded by redesigning orientation and other details • Program satisfaction improved from 02 to 04

22. Special Education We track the effects of MRI on Special Education in two ways: • Beginning with schools that started MRI in the Fall of 2002, all students with IEPs are identified and the type of IEP is described (Reading, Math, Speech, etc.) • Annual reports are made from schools about their IEP evaluation process: referrals, evaluations, and IEPs.

23. Notes for Special Education Chart • This Chart is for schools that were in their 3rd year 2003-2004 • Many schools do not have this data, or it is not easily accessed • 9 of 18 in 2002 • 5 of 23 in 2003 • 10 of 34 in 2004 • The data points to a decrease in referrals, evaluations, and assignment of IEP over the time schools participate in MRI. • Data collection began from the onset of schools in 2002-2003 with a complete report in 2005.

25. Developmental Reading Assessment (DRA) The following Table describes the changes in the percentages of cohorts of students who scored At or Above Grade Level according to the DRA at 15 MRI schools who have completed two years in the program. Key: =% At or Above GL S031=1st Graders in Spring 2003 S042=2nd Graders in Spring 2004 F02 =2nd Graders in Fall 2002 S043=3rd Graders in Spring 2004  = Percentage Change

26. Change in DRA Grade Level