1 / 26

Warren County School District READ 180 Partnership Mid-Year Gains Analysis Export of Data in February 2011

Warren County School District READ 180 Partnership Mid-Year Gains Analysis Export of Data in February 2011. Executive Summary. An analysis was performed from data for 6 schools that used READ 180 during the 2010-2011 school year. The data was exported on 2/3/11 and contained:

gaius
Download Presentation

Warren County School District READ 180 Partnership Mid-Year Gains Analysis Export of Data in February 2011

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Warren County School DistrictREAD 180 PartnershipMid-Year Gains AnalysisExport of Data in February 2011

  2. Executive Summary • An analysis was performed from data for 6 schools that used READ 180 during the 2010-2011 school year. • The data was exported on 2/3/11 and contained: • 95% of 302 total students in the export took a post-test. • 278 student records with READ 180 use had SRI pre-test and post-test nearly 18.2 weeks apart on average. • The district averaged 23 total READ 180 sessions of 13.6 minutes in length. • The analysis assumes that a year’s gain in grades 3-5 = 140 L, at grades 6-8 = 70 L and at grades 9-12 = 50 L. This is based on gains for students reading at the 25th percentile. • FOCUS FOR MEETING: • Improve implementation fidelity practices within the district to maximize student reading gains by end of year.

  3. Executive Summary • Evidence of Success: • 54 of 278 students (19%) had 2.0 or more years of reading growth by mid-year. • 101 of 278 students (36%) had at least a year’s reading growth by mid-year SRI. • Youngsville HS had approximately 1 year of reading growth by mid-year. • 5 of 6 schools had reading gains during the school year. • Challenges: • 23 days of READ 180 program usage in 18 weeks of school (90 days) • Students with potentially non-valid post-tests. Large declines from pre-test to post-test are typically the result of student behavior problems during test administration.

  4. READ 180 Superstar Students 48 students with evidence of strong participation in the program achieved 1 to 3 years’ reading growth—so many that they would not fit on one page!16 student-level results are displayed below. Each row represents 1 student’s data from the indicated class. SuperStars have at least 20 sessions by mid-year, pre-test SRI over 100 Lexiles and 1 to 3 year’s growth.

  5. READ 180 Session Use Analysis

  6. Summary Reading Gains for READ 180 Students Overall, 5 of 6 schools showed positive gains. * 70 Lexiles is approximately equal to a year’s growth at grades 6-8 and 50 at grades 9-12.

  7. READ 180 Summary Gains and Software Usage Data Youngsville HS achieved growth levels of over 1 year. Annual growth expectations are 140 L at grades 3-5, 70 L at grades 6-8 and 50 L at grades 9-12. Average Sessions Per Week does not account for vacation, inclement weather, assemblies, etc within the school calendar.

  8. Classroom Summary Gains and Software Usage Data 39% of classes have achieved growth levels of approximately 1 year. Small class sizes may reflect students enrolled in multiple classes on Scholastic Achievement Manager, including non-READ 180 classes.

  9. Classroom Summary Gains and Software Usage Data Small class sizes may reflect students enrolled in multiple classes on Scholastic Achievement Manager, including non-READ 180 classes.

  10. Summary Reading Gains of READ 180 by Grades Year of Growth MS Grades 6-8 Year of Growth HS Grades 9-12 All grades showed positive gains. * 70 Lexiles is approximately equal to a year’s growth at grades 6-8 and 50 at grades 9-12.

  11. Data Analysis Process and Valid Data Metrics Subsets of 278 Valid Records: READ 180 Data Records: Export Data Cleaning Process Analysis Data *BR (Beginning Reader) is less than 100 Lexiles. Proficient is based on the Scholastic SRI performance standards by grade. Examine Post-Test Results indicates students with a decline in SRI of 60 or more Lexiles. Highlighting of cells based upon cut-offs outlined in column definitions.

  12. Critical Thinking: Examine SRI Test Scores • SRI is a computer adaptive test • Targeting improves accuracy of SRI • Without being targeted, SRI will assume the student is AT GRADE LEVEL for their grade in SAM. • Untargeted students will receive questions beyond their ability to comprehend. • Failure with questions leads to frustration and potentially a loss in motivation. • Motivation during SRI affects performance • Examine SRI after each administration to verify that the score is in the expected range for the student based upon other reading assessments and knowledge of the student. • Pre-test SRI below 100 Lexiles (Beginning Reader) may represent a lack of motivation. Low pre-test SRI with high comprehension ability will skew successive SRI. BR students with increases over 500 Lexiles between 2 SRI administrations may have invalid pre-test SRI. • With each post-test, if the student has improved comprehension their Lexile should increase. A decline in of 60 Lexiles or more may indicate a lack of motivation and invalid post-test SRI. For more information on the importance of SRI function, consult the SRI Technical Guide. ISBN 978-0-439-74216-0

  13. Impact of Student LexileDecline Upon Mean Gains Mean Lexile Gain of All Students at Indicated School KEY: Mean Lexile Gain of Students without Lexile Decline of 60 or More Mean Lexile Decline of Students with Lexile Declines in Excess of 60 13 Detailed explanation of above graph in Appendix: Yellow bars are previously reported school averages, green bars show potential average.

  14. SRI Best Practices • Before Initial SRI (Pre-Test) • Determine students’ approximate reading levels. • Target reading ability in Scholastic Achievement Manager to decrease • Standard Error of Measure (for new students). • Select an appropriate intervention program for students reading at the “Below Basic” or “Far Below Basic” level. • After Each Post-Test • Run an SRI “Growth Report” from SAM to identify: • Students whose SRI has dropped more than 60 Lexiles (maximum targeted SRI Std. Error Measure) or have abnormally low pre-test SRI. • Students who have not taken a post-test SRI. • Use “Student Progress Report” / “Student Yearly Progress Report” to see student’s long-term SRI trend. • Discuss results with all students to determine cause of SRI declines. • Determine an appropriate course of action, such as permitting students who believe they could have done better to take another SRI administration. For more information on the importance of SRI targeting, refer to the SRI Best Practices Guide, Item # 273300.

  15. Analysis Conclusions Summary Findings: • Youngsville HS demonstrated approximately 1 year’s growth at the mid-year SRI administration. • Students with a higher number of READ 180 software sessions had higher Lexile gains. • Mean pre-test SRI scores ranged from 124 Lexiles to 975 Lexiles. • Mean weeks between SRI pre- & post-tests ranged from 12.1 to 20.0 weeks. • The data indicate an average READ 180 software session usage of 23 days and 1.6 days per week, with an average range of 9.8 to 18.6 minutes per session. • There is opportunity to improve this year’s implementation through effectiveness monitoring and by increasing time on software.

  16. Recommendations • Follow-up with students whose mid-year SRI score indicates a drop from previously demonstrated reading ability at 2010-2011 pre-test. • Take steps to replicate the success of schools that showed greatest gains. • Reinforce importance of on-model implementation to building-level leadership by following regular reporting protocol and review of implementation status. • Encourage teachers to enroll in literacy-focused professional development courses. • Invest in an ongoing coaching and support program to increase implementation fidelity and enhance teacher instructional practices. • Evaluate expansion of READ 180/System 44 into additional areas of high need.

  17. Appendix

  18. Methods • Software data was collected from SAM database via export for analysis. • The data set provided information on both student reading achievement gains and implementation metrics from all schools where the program is in use. • The analysis assumes that software usage is a proxy for classroom implementation. On-model implementation of READ 180 should result in usage metrics of 4-5 times per week and 16-20 minutes of time per session. • The analysis was conducted at the school level. The analysis examined mean Lexile gains in comparison to implementation fidelity metrics such as sessions per week and minutes per session. Version 1: 2010-2011 Warren Cty, PA mid-year Gains Analysis (Basic).

  19. Summary Usage Metric Explanations • # Students with Valid Data: The number of students with a pre-test and post-test score at least 6 weeks apart and with 6+ sessions of program usage. • Mean Pre-Test Lexile: Mean student pre-test score as measured in Lexiles. • Mean Post-Test Lexile: Mean student post-test score as measured in Lexiles. • Mean Lexile Gain: Mean student Lexile gain from pre-test to post-test. • Individual Student Level Gains: Based upon pre-test SRI Lexile score percentile, student’s expected yearly growth estimate is based upon grade level peer performance from Fall to Spring SRI. • % of Students with 1+ Years Gain: The percent of students who have achieved 1+ years of reading gains between pre-test and post-test.

  20. Summary Usage Metric Explanations • Weeks of Time Between Pre-Test and Post-Test: The number of weeks between pre-test and post-test. • Mean Days of READ 180 Software Usage: The mean total of software sessions, per student, between pre-test and post-test. • Average Sessions per Week: The average number of software sessions per week, as calculated by the READ 180 software. In an on-model implementation, this metric should range from 3-5 sessions. • Average Session Length in Minutes: The average number of minutes per software session, as calculated by the READ 180 software. In an on-model implementation, this metric should range from 16-20 minutes. • Growth Rate Indicator: Green coded schools are demonstrating over 1 year’s reading gains. Yellow coded schools are modestly successful but may require additional professional development. Red coded schools may need to be examined to determine what factors impacted success.

  21. Data MetricsExplanations READ 180 Data Records: • Record With READ 180 Data: Total number of records exported from Scholastic Achievement Manager with software usage. • Records With SRI Pre-Test: Total number of records exported which have an initial SRI test this school year. • Records With SRI Post-Test: Total number of records exported which have an SRI test, other than the pre-test, this school year. • Records With No Post-Test : Number of records which had no post-test at the time of the analysis. Highlight when this is over 20% of pre-test records. • Pre-Test & Post-Test Less Than 6 Weeks Apart: Records Total number of records which do not have two SRI tests which are at least six weeks apart. Highlight when this is over 20% of post-test records. • Less Than 6 Software Sessions: Records which have less than 6 software sessions. Highlight when this is over 20% of post-test records. • # Students with Valid Data: The number of students with a pre-test and post-test score at least 6 weeks apart and with 6+ sessions of program usage.

  22. Data MetricsExplanations Breakdown of Valid Records: • Students with 20+ Sessions: Students with valid records and 20+ sessions to date for the current school year. Highlight when this is less than 2/3 of valid records. • BR Students at Pre-Test: Students identified as Beginning Readers (below 100 Lexiles) at the pre-test. Highlight when this is over 10% of pre-test records. • Students Proficient at Pre-Test: Total number of students scoring at or above proficient level for their grade (nextpage). Highlight when this is over 10% of pre-test records. • Examine Post-Test Scores: Students with a decline in excess of 60 Lexiles from pre-test to post-test. Such an decline in performance could reflect a student motivation problem. Highlight when this is over 10% of pre-test records.

  23. Grade Levels and SRI Lexile Levels

  24. Lexile Decline Analysis Explanation Mean Lexile Gain of All Students at Indicated School KEY: Mean Lexile Gain of Students without Lexile Decline of 60 or More Mean Lexile Decline of Students with Lexile Declines in Excess of 60 Gains without students who may need retested Gains reported previously Drop in scores of students with large declines # students represented by each bar 24

  25. SAM Class Names • Classes in Scholastic Achievement Manager • Students can be enrolled in multiple classes within SAM. • An export can identify 1 class per record – even if the student is enrolled in many classes. • Classes in SAM are sorted in alphabetical order prior to export assignment of class enrollment. • Class that begin with numbers or characters (ie @) will naturally come before letters. • SAM Class Name Best Practices • Class names for actual instructional classes should indicate teacher, period & software used with the students in that class. • Large “catch-all” or import classes should be named in such a way that they will naturally fall away in data exports: “zz – 8th Grade” or “zz – Test Group” • Remove outdated, temporary & irrelevant class designations from the SAM database. 25 For more information on Managing Classes and Groups, refer to the READ 180 Software Manual EE1.0.1 pages 53-72, ISBN # 0-439-73475-4.

  26. Yearly Growth on SRI For each student, expected Lexile gains depend upon their individual Fall pre-test SRI percentile. Average gains at the school and gradelevel have been calculated using research-driven Lexile analyses. Average yearly growth for elementary, middle and high school levels based upon student populations: 140 Lexiles at grades 3-5, 70 Lexiles at grades 6-8 and 50 Lexiles at grades 9-12. To the right are approximate grade level expectations based upon 25th Percentile students in need of Tier 2 reading intervention. Schools with a high percentage of students in one grade level may demonstrate Lexile growth more aligned to that grade than the school level analysis average indicated above. Grades 1, 2, 11 & 12 are approximated from smaller sample sizes than other grades

More Related