1 / 25

Evaluating the Vermont Mathematics Initiative (VMI) in a Value Added Context

Evaluating the Vermont Mathematics Initiative (VMI) in a Value Added Context. H. ‘Bud’ Meyers, Ph.D. College of Education and Social Services University of Vermont Douglas Harris, Ph.D. The Evaluation Center The Vermont Institutes, Montpelier, VT. Purpose.

abel-herman
Download Presentation

Evaluating the Vermont Mathematics Initiative (VMI) in a Value Added Context

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Evaluating the Vermont Mathematics Initiative (VMI) in a Value Added Context H. ‘Bud’ Meyers, Ph.D. College of Education and Social Services University of Vermont Douglas Harris, Ph.D. The Evaluation Center The Vermont Institutes, Montpelier, VT

  2. Purpose • Describe the context, purpose, structure, staffing and content of the instructional program for teachers • Describe the logic model that provides the framework for evaluation of the program • Examine the methodology for assessing the value added to student learning by teacher preparation in mathematics • Present findings from cross-sectional analyses of eight cohorts of students who have been taught by VMI teachers and four cohorts of control students. • Present findings from a qualitative evaluation of the process of implementation of the components of the program from the perspectives of participants and administrators • Present recommendations arising from the findings of the evaluation

  3. Context, purpose, structure, staffing • A field based masters degree in Education with a primary emphasis on mathematics content • Partially funded by local districts, partially MSP • Focus on Grades k-8 • Includes focus on teacher leadership • Includes action research • Three years • Mathematicians and educators

  4. Content-12 Courses Course 1:Mathematics as a Second Language (3 credits) Course 2:Functions and Algebra for Elementary Teachers (3 credits) Course 3:Trigonometry for Elementary Teachers (2 credits) and Algebra and Geometry II (1 credit) Course 4:Measurement, Geometry, and Probability for Elementary Teachers (3 credits) Course 5: Number Theory for Elementary Teachers (3 credits)

  5. Courses…contd. • Course 6:Statistics, Action Research, and Inquiry into Effective Practice, I (3 credits) • Course 7: Statistics, Action Research, and Inquiry into Effective Practice, II (3 credits) • Course 8: Algebra and Geometry for Elementary Teachers, III (3 credits) • Course 9: Statistics, Action Research, and Inquiry into Effective Practice, III (3 credits) • Course 10: Calculus for Elementary Teachers, I (3 credits)

  6. Courses…contd. • Course 11: Calculus for Elementary Teachers, II (3 credits) • Course 12: Capstone VMI Experience (3 credits)

  7. Methods • Quantitative- Cross-sectional and longitudinal • Two intervention groups • Group 1=16 schools, multiple VMI teachers (HiEnd) • Group 2=22 schools, single teacher per school (RegIntervention) • Control group=22 schools, matched to Groups 1 and 2 (combined) on demographics • Longitudinal comparisons for 2 cohorts, 1999 and 2000, grades 4 to 8. (See Figure 2)

  8. Methods • Qualitative • Interviews • Participants • Graduates • Administrators • Observations • Review of course materials and participant portfolios • Course evaluations • Interviews of School Staff and Leadership

  9. Sample Characteristics Figure 1

  10. Longitudinal comparisons Figure 2 x = Comparisons

  11. Question 1: What do comparisons with comparable schools show, over time, when students are matched from grade 4 through grade 8 and grade 10 during the two cohort groups (1999-2005, and 2000-2006)? Figure 3 Significant differences in 2005. t=3.22, p<.001, df 1, 1300. Figure 4 Significant differences in 2004. t = 5.27, p<.001, df 1, 1488 Significant differences in 2006. t = 3.39, p<.001, df 1, 1107

  12. Discussion • As indicated in Figures 3 and 4, each of the VMI cohorts outperforms the Control schools in similar patterns of difference. Results for comparisons at the first data point for each cohort (1999 and 2000) are not significantly different while they emerge as significant in 2005 and 2006.

  13. Question 2: How do levels of implementation of the VMI curriculum in classrooms relate to levels of achievement of students who experience them? (1999 Cohort) Figure 5 Significant differences in 2005. adjusted for differences in 1999, HiEnd v Reg and Control, F = 7.96, df 2, 1235, p <.001

  14. Question 2: How do levels of implementation of the VMI curriculum in classrooms relate to levels of achievement of students who experience them? (2000 Cohort) Figure 6 Significant differences in 2006 adjusted for differences in 2000, HiEnd v Reg and Control, F = 9.91, df 2, 1093, p <.001 Note: The average standard deviation for groups is about 40 points across all groups. Mean differences between groups is thus approximately .25 S.D. between HiEnd and Control groups.

  15. Discussion: VMI HiEnd Schools significantly out-performed the matched schools and appear to carry most of the difference between the VMI and matched schools in both cohorts from 1999 through 2006. Analyses of covariance adjusted the score differences at the first two data points in each cohort. The final data point for each cohort represents group differences at grade 10.

  16. Question 3. Are school level effects related to income level of students and levels of content implementation of the VMI curriculum? (1999 Cohort) Do gaps close? Figure 7

  17. Question 3. Are school level effects related to income level of students and levels of content implementation of the VMI curriculum? (2000 Cohort) Do gaps close? Figure 8

  18. Discussion:

  19. Quantitative Summary • Longitudinal comparisons of VMI and Control schools indicate a persistent pattern of advantage for VMI schools • Longitudinal comparisons indicate that the VMI schools, with more VMI teachers, had a significantly greater long term gain on percentile ranks. Stronger teachers may not have as great an effect in the shorter term when isolated. • Longitudinal comparisons suggest that SES differences are reduced over time when high quality mathematics instruction is offered to all students

  20. Qualitative Findings (2005 and 2006) • Impact on Participants and Their Teaching • Finding 1: Mathematics Content • An overarching theme is the impact of the VMI experience on the teacher’s own understanding of mathematics content. This is consistent among participants in all three years of the VMI experience and remains the highest priority of graduates as they reflect on the VMI experience • Finding 2: Increased Confidence Related to Mathematics • Increased understanding of mathematics content impacts the confidence of participants as related to mathematics, to teaching mathematics, and to their enthusiasm for and enjoyment of mathematics. • Finding 3: Impact on Instructional and Assessment Practice • Participants graduates (many of who are math teacher leaders), and principals report that the instruction in VMI, increased content knowledge, and increased confidence have had major impact on instructional and assessment practice in the classrooms of participants and of teachers impacted by VMI-trained math teacher leaders • Finding 4: The Impact of Action Research on Classroom Practice • The impact of action research is mixed, with some participants, graduates, and principals reporting considerable impact, others less so. • Finding 5: Principal Support • The active support of principals makes a profound difference in VMI participants’ work in the classroom and in leadership positions.

  21. More Qualitative Findings • Finding 6: Personal Impact on Participants • Beyond the impact of VMI on teachers in relation to math content, instruction, and leadership, the program also has profound personal impact on many participants. • Impact on Students • Finding 7: Transfer of VMI Content to the Classroom • Teachers report direct transfer of mathematics content used in VMI to the math experiences of their students. • Finding 8: Impact on Student Problem Solving • Teachers and teacher leaders report that the problem solving emphasis in VMI has significant impact on their understanding of how to engage students in problem solving in the classroom. • Finding 9: Impact of Action Research on Students • Some teachers believe that the interventions begun in their action research projects will continue to impact their students over time. • Impact on Teacher Leadership in Schools and Districts • Finding 10: Impact on Teacher Leaders • Teachers working as teacher leaders credit VMI for providing the knowledge, confidence, and support for them to take on leadership roles. This is true of teachers working in leadership at the school, district, and state levels. • Finding 11: Impact on curriculum, instruction, and assessment • Teacher leaders report that the knowledge base and contacts developed during VMI have significant impact on the development and implementation of mathematics curriculum, instruction, and assessment. • Finding 12: Need for Ongoing Professional Community • There is a common desire among VMI graduates to maintain the type of professional learning community afforded them through VMI.

  22. Recommendations

  23. Educational or Scientific Importance of the Study The National Research Council, (2004) US Department of Education (2004) and researchers at RAND (2004) have cited the need for better evaluations of teacher professional development programs. The combination of mixed method approaches with multiple strategies such as longitudinal studies of students acting as their own controls over time is the present attempt to respond to current concerns. In addition, intervention students in combination with matched groups at multiple intervals is a further attempt to provide additional confidence in results.

  24. Want a copy of the paper or PowerPoint?? • Bud.Meyers@uvm.edu

More Related