1 / 16

Prepared by Steven H. Shaha, PhD, DBA

PD 360 Impact Assessment: Initial Findings regarding the Impact of PD 360 on Student Proficiency Rates. Prepared by Steven H. Shaha, PhD, DBA. Overarching Research Question:. Does teacher engagement i n PD 360 and Observation 360, tools within the Educator E ffectiveness S ystem ,

lecea
Download Presentation

Prepared by Steven H. Shaha, PhD, DBA

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. PD 360 Impact Assessment:Initial Findings regarding the Impact of PD 360 on Student Proficiency Rates Prepared by Steven H. Shaha, PhD, DBA

  2. Overarching Research Question: Does teacher engagement in PD 360 and Observation 360, tools within the Educator Effectiveness System, significantly affect student success?

  3. Methods • Design: Quasi-experimental, retrospective, pre-post, normalized treatment-control / participation vs. non-participation (2009-10, 2010-11) • Goal: Multi-State, large n with comparable student populations (matched, controlled) • Student Change:*Metric was percent students classified as Proficient or Advanced in respective States.

  4. Sample Participation • Systematic sample of 187 schools in 8 States • N determined by a priori Power analysis • Schools eligible for inclusion in the sample as participating Schools met the following criteria: • More than 10 teachers total • 80% or more of teachers viewed materials • Minimum average of 90.0 minutes of viewing per teacher for the school • Districts included were only those for which eligible schools were included • Normalizing for difference in socio-economic and demographic factors between participating Schools and their Districts cumulatively as the statistical comparison group Data • Participation data were extracted from the Internet-based professional development application as surveilled • Student performance data were captured from publically available, Internet-accessed sources (school as unit of measure, percent Proficient or Advanced as metric)

  5. Improvements in Proficiency Rates Correlated with PD 360 Sample Demographics • Eight States • 187 Schools

  6. Metrics for Differentiating Advantages for Higher Engagement Organizations: Leadership, Implementation and Accountability • Focus Objectives Set Up • Observations Performed • Percent Registered Users • Percent of Users in Communities • Minutes Viewed • Forums Viewed • Programs Viewed • Segments Viewed • Links Viewed • Follow-up Questions Answered • Reflection Questions Answered • Focus Objectives Set Up • Forums Posted • Downloaded Files • Uploaded Files • Participation in Communities Educator Participation Educator Engagement These are the 15 metrics for which higher engagement schools were significantly higher than their lower engagement counterparts

  7. PD 360 Impact AssessmentExecutive Summary: Initial Findings • Statistically significant and resounding advantages were identified favoring schools with PD 360. • The higher the quantified utilization, the greater the statistical advantage. • High utilization led to performance advantages ranging from twiceto 15 timesthe gains in proficiency rates versus district averages (p<.01 and p<.001, respectively). • Low utilization led to gains in proficiency rates from 10% to twice the district averages (p<.01 and p<.001, respectively).

  8. Improvements in Proficiency Rates Correlated with PD 360 High Utilizers Outperformed all others High Utilization results in more the TWICE the gains in Math proficiency rates than does Low Utilization (p<.001) But even low utilizers outperformed no PD 360 High Utilization results in nearly TWICE the gains in Reading proficiency rates than does Low Utilization (p<.001)

  9. Improvements in Proficiency Rates Correlated with PD 360 High Utilizers outperformed no PD 360 High Utilization results in more the FIFTEEN times the gains in Math proficiency rates than the Districts average (p<.001) High Utilization results in nearly TWICE the gains in Reading proficiency rates than the Districts average (p<.001) 15.8 times greater gains in Math vs. District 90.6% greater gains in Reading vs. District

  10. Improvements in Proficiency Rates Correlated with PD 360 But even Low Utilizers outperformed no PD 360 Low Utilization results in more the SEVEN times the gains in Math proficiency rates than the Districts average (p<.001) Low Utilization results in nearly 10% greater gains in Reading proficiency rates than the Districts average (p<.01) 9.3% greater gains in Reading vs. District 7.4 times greater gains in Math vs. District

  11. Improvements in Proficiency Rates Correlated with PD 360 More Minutes per User resulted in Higher Performance High Minutes per User result in more the TWELVE times the gains in Math proficiency rates than Districts avg. (p<.001) High Minutes per User result in nearly TWICE the gains in Reading proficiency rates than the Districts average (p<.001) 71.5% greater gains in Reading vs. District 12.0 times greater gains in Math vs. District

  12. Improvements in Proficiency Rates Correlated with PD 360 But even Low Minutes per User outperformed no PD 360 Low Minutes per User result in more the TEN times the gains in Math proficiency rates than the Districts average (p<.001) Low Minutes per User result in more than 30% greater gains in Reading proficiency rates than the Districts average (p<.001) 32.8% greater gains in Reading vs. District 10.4 times greater gains in Math vs. District

  13. Improvements in Proficiency Rates Correlated with PD 360 High Percentage of Users that Viewed resulted in more the THIRTEEN times the gains in Math proficiency rates than the Districts average(p<.001) Higher percentage of Viewers resulted in Higher Performance High Pct of Users that Viewed resulted in TWICE the gains in Reading proficiency rates than the Districts average(p<.001) 95.8% greater gains in Reading vs. District 13.5 times greater gains in Math vs. District

  14. Improvements in Proficiency Rates Correlated with PD 360 But even Low percentage of Viewers outperformed no PD 360 Low Percent of Users that Viewed resulted in more the NINE times the gains in Math proficiency rates than the Districts average(p<.001) Low Percent of Users that Viewed resulted in more than 20% greater gains in Reading proficiency rates than the Districts average(p<.001) 23.6% greater gains in Reading vs. District 9.4 times greater gains in Math vs. District

  15. Improvements in Proficiency Rates Correlated with PD 360 More Viewers resulted in Higher Performance High Number of Users that Viewed resulted in more the THIRTEEN times the gains in Math proficiency rates than the Districts average(p<.001) High Number of Users that Viewed resulted in nearly TWICE the gains in Reading proficiency rates than the Districts average(p<.001) 92.5% greater gains in Reading vs. District 13.4 times greater gains in Math vs. District

  16. Improvements in Proficiency Rates Correlated with PD 360 But even fewer viewers outperformed no PD 360 Low Number of Users that Viewed resulted in nearly TEN times the gains in Math proficiency rates than the Districts average(p<.001) Low Number of Users that Viewed resulted in more than 30% greater gains in Reading proficiency rates than the Districts average(p<.001) 31.6% greater gains in Reading vs. District 9.7 times greater gains in Math vs. District

More Related