Value-added Accountability for Achievement in Minneapolis Schools and Classrooms

1. Value-added Accountability for Achievement in Minneapolis Schools and Classrooms Minneapolis Public Schools December, 2000 www.mpls.k12.mn.us/REA/

2. Minneapolis Pubic SchoolsMeasuring Up Against High Standards • MPS developed standards-based reading and math tests - Northwest Achievement Levels Tests (NALT). • All students participate • Sensitive to growth • Predictive of high stakes outcomes • Developed indicators in concert with stakeholders who are internal and external to the district. • Indicators of school performance are based on multiple indicators. • 33 for elementary and middle schools • 24 for high schools

3. Standards Based means: • We use a mutually agreed upon rubric for each indicator of school performance. • Rubric scoring on a “1” to “5” scale. • “3” Current standard • “5” Long term goal • Data are reported by site, by indicator. • Community reports list percent of indicators at standard (3 or better) “Measuring Up.”

4. Selecting a unit of Analysis • Minneapolis Public Schools in concert with multiple stakeholders agreed upon a matrix of indicators. • School Personnel • Community input (e.g. NAACP) • Chamber of Commerce • Minneapolis Foundation • Families

5. Criteria for Indicators • Meyer (1996) • Indicators must measure things that matter or are valued by society (MCA, MBST). • Indicators must be able to avoid being “corrupted.” • inclusion v. exclusion criteria • Indicators must be able to measure the unique contribution schools “add” for each measured outcome.

6. Stakeholders agreed on using… • Multiple indicators of school performance. • See chart of indicators from Appendix. • These indicators include: • Current level of performance • cross-cohort • Gain and value-added measures

7. Elementary & Middle School QPA Indicators

8. Fatal Flaws of a level or means approach. • Mean test performance is contaminated by factors other than school performance. • Mean performance is out of date, and does not address cumulative, compounding effects of previous instruction. • Mean performance is affected by mobility. • Mean performance interferes with localizing performance.

9. Distribution of school average 1999 NALT mathematics scores by free or reduced price lunch percentages

10. Distribution of school average mathematics gain scores on the 1999 NALT by free or reduced price lunch percentages.

11. Cross-Cohort AnalysisDistrict Level Standard

12. The problem with cross-cohort analysis at the school level Expected Growth= 26 points/year in 3rd grade reading Look at the “bounce” in cross-cohort analysis

13. Northwest Achievement Levels Test Validity Coefficients with Minnesota State Tests

14. Linkage with State Standards

15. True gain model: Growth Curves

16. True Gain model: Percent of Students Making One Year Gains District Standard

19. Value-added Model • Primary question is this, “How can an equitable accountability system be put in place when there is a large degree of variability in the way student characteristics are distributed among our schools?” • English language learners • Special education programs • Race/ethnicity • Poverty • MPS relies on an empirical- and literature-based model to make predictions, not set expectations! • Schools are acknowledged/rewarded for “Beating the Odds.”

20. Value-added Teacher Effects • Post-test reading score= • Pretest reading score • + Free or reduced price lunch • + Racial/ethnic code • + Neighborhood poverty concentration • + Lives with both parents code • + Limited English Proficiency status • + Special Education status • + Teacher effects

21. Teachers who “beat the odds” in second grade reading reported: • more use of small group instruction • more development of word attack skills • more individual student oral reading • more guidance during initial practice • more explicit and direct phonics instruction • more use of systematic motivation strategies