The Impact of MMP School Involvement on Student Achievement

1. The Impact of MMP School Involvement on Student Achievement Cindy M. Walker & Kevin McLeod University of Wisconsin - Milwaukee Based upon work supported by the National Science Foundation Grant No. 0314898.

2. Overview • What is the Milwaukee Mathematics Partnership (MMP)? • What are the goals of the MMP? • Where are we at now? • How do we measure school-level involvement in the MMP? • Evaluation Design • Results • Key Insights

3. What is the MMP? • A community-wide collaborative PK-16 effort among school, university, union, government, business, and community organizations that seeks to substantially improve mathematics achievement for the 100,000 K-12 Milwaukee Public Schools students. • The MMP involves mathematics faculty and mathematics educators in collaboration with PK-12 educators in strengthening district curricula, student assessment measures, and re-designing pre-service and in-service teacher preparation focused on the needs of an urban district. • Core Partners include: (1) UWM; (2) MPS; and (3) MATC

4. What are the goals of the MMP? • Goal #1: Comprehensive Mathematics Framework • A collective vision of deep learning and quality teaching of challenging mathematics across the Milwaukee Partnership. • Strategies include PK-12 student Learning Targets and Model Performance Assessments, alignment of high school coursework with college expectations, and increased enrollment and success in challenging mathematics courses, including Advanced Placement and International Baccalaureate mathematics courses

6. What are the goals of the MMP? • Goal #2: Distributed Leadership • Institute a distributed mathematics leadership model that engages all partners and is centered on school-based professional learning communities. • Strategies include Math Teacher Leaders, school-based Learning Team, Principal Mathematics Leadership endeavor, and district mathematics leadership.

7. What are the goals of the MMP? • Goal #3: Teacher Learning Continuum • Build and sustain the capacity of teachers, from initial preparation through induction and professional growth, to understand mathematics deeply and use that knowledge to improve student learning. • Strategies include the IHE Mathematics Network; Design Teams focused on core mathematical preparation for all PK-8 teachers, an elementary mathematics minor, and a secondary mathematics capstone course; teachers on special assignment as Teachers-In-Residence at the university; teacher recruitment; content-focused induction; and school-based professional learning communities.

8. What are the goals of the MMP? • Goal #4: Student Learning Continuum • Ensure all students, PK-16, have access to, are prepared and supported for, and succeed in challenging mathematics. • Strategies include School Educational Plans, mathematics alignment for the Tutoring and Family Literacy Initiative, and Transitioning to College Mathematics effort

9. Where are we at now? • The MMP is a mature project, now in its seventh year • Substantive funding has shifted from NSF to the State of Wisconsin • Specifically, the state has provided funding for many schools in MPS to have a fully released Math Teacher Leader (MTL) • Therefore it is important to continually reinforce the claim that school-level involvement in the MMP is a critical factor for promoting student achievement gains

10. How do we measure school-level involvement in the MMP? • Lack of experimental design, therefore, we have continually strived to measure variability in participation in MMP activities. • The most current work examined the level of school-level involvement in key MMP-sponsored activities over a four year period • These activities include: • Attendance at Math Teacher Leader meetings • Participation in courses offered at UWM

11. MTL meetings • Held monthly during the school year • Full-day in-service meetings • Attended by approximately 140 MTLs each month over the course of three meetings (i.e. ≈ 50 per meeting) • Three main strands covered at each meeting: • Mathematics Content (including Mathematical Knowledge for Teaching) • Leadership • Assessment • Different content strands covered each year

12. UWM in-service courses offered • Several courses offered each year for university credit including:

13. Evaluation Hypotheses • MMP school-level involvement expected to predict student achievement growth, defined as change in percent of students proficient at a school on state mandated test (WKCE), from 2005-2008 • MMP school-level involvement expected to predict percent of students at school that are classified as proficient in mathematics in the fall of 2008

14. Evaluation Design • Each school was given an MMP-involvement score that ranged from 0 (no involvement) to 43.65 (highest level of involvement). • These scores were created by summing two scores, one that quantified attendance at MTL meetings and one that quantified staff participation in UWM courses

15. Evaluation Design • MTL attendance was expressed as the total percentage of meetings attended by at least one representative from the school. • For example, a school that had sent at least one person to every MTL meeting over the four years would receive a score of 4.0 because each year they would have received a 1.0, reflecting 100% participation each year • A school that had sent at least one person to 2 out of 9 meetings the first year, 9 out of 9 meetings the second year, 7 out of 9 meetings the third year, and 5 out of 9 meetings the fourth year would receive a score of 2.56 (e.g. 2/9 + 9/9 + 7/9 + 5/9 = 23/9 ≈ 2.56)

16. Evaluation Design • Course enrollment was the sum of: (1) unique teachers in a school that enrolled in at least one course, (2) the average number of courses taken by those teachers, and (3) the average credits earned by those teachers. • For example, a school that had five teachers, two of which enrolled in two 1-credit courses and three of which enrolled in three 3-credit course would receive a score of 9 (e.g. ) • A school that had one teacher that enrolled in three 3-credit courses would receive a score of 7 (e.g. 1 + 3 + 3 = 7.0)

17. Evaluation Design • All non-zero MMP-involvement scores were converted to z-scores to classify schools in the following manner: • Schools with z > +1 were classified as having HIGHinvolvement with the MMP • Schools with -1 < z < +1 were classified as having MODERATEinvolvement with the MMP • Schools with z < -1.0 were classified as having LOW involvement with the MMP • Schools with an original MMP-involvement score of zero were classified as having NOinvolvement with the MMP

18. Evaluation Design • Student proficiency was measured by the state mandated standardized assessment, the Wisconsin Knowledge and Concepts Examination (WKCE). • Student achievement growth reflected the change in the percent of students classified as proficient in 2008, as compared to 2005. • For example, a school with 20% of students proficient in 2005 and 35% of students proficient in 2008 received a score of 15%

19. Evaluation Design • After the metrics were compiled, a one-factor ANOVA was conducted using MMP involvement as the grouping factor and either student proficiency in 2008 or student achievement growth as the dependent variable. • The analysis was conducted using a Welch correction, when considering student achievement growth, as the assumption of homogeneity of variance was found to be violated for the four different groups.

20. Evaluation Results: Growth StandardDeviation F3, 174 = 7.45, p < 0.001

21. Evaluation Results: Growth Mean Growth F3, 59.3 = 5.48, p = 0.002

22. Evaluation Results: 2008 Proficiency Percent Proficient F3, 194 = 25.14, p < 0.001

23. Key Insights • It is imperative that MSP projects: • Clearly articulate their core strategies and activities; • Document and measure the impact of those strategies, and; • Develop evidence that those strategies lead to desired outcomes • We clearly articulated two primary professional development strategies • Math Teacher Leader meetings, and • Mathematics content courses for in-service teachers

24. Key Insights • While schools were not required to participate, all were encouraged to take advantage of these professional development activities with the promise that participation would lead to better student outcomes. • Documentation of participation over time and relating those results to student outcomes helped to provide compelling evidence to the district, and to the state, that the MMP has made a positive impact. • This message is being carried forward as critical evidence for sustaining MMP efforts.