Pilot Implementation Data: What We’ve Learned So Far

1. Pilot Implementation Data: What We’ve Learned So Far Gretchen Weber Director of Educator Effectiveness Initiatives January 24, 2012

2. Teacher Evaluation Hopes and Worries: September 2011 • 72% of teachers, 83.9% of principals, and 100% of administrators were moderately to very excited about the new system. • The top aspects of excitement: • Professional growth/development • Discussions with administrators • Clearly defined, evidence-based standards • 23.9% of teachers, 47.6% of principals, and 50% of administrators were moderately to very worried about the new system. • The top worries: • Time • Fidelity • Buy-in

3. Principal Evaluation Hopes and Worries: September 2011 • 75% of principals and 85.7% of administrators were moderately to very excited about the new system. • Top aspects of excitement: • Professional growth/development • Clearly defined, evidence-based standards • 26.9% of principals and 38.5% of administrators were moderately to very worried about the new system. • Top worry: • Time

4. TPEP Case Studies: Methodology • Self-Report Data • Professional Development Resources • Surveys • Teachers • Principals • Administrators • Interviews • Teachers • Principals • Pilot leads • Union/association representative

5. TPEP Case Studies: Constructs • The new evaluation framework and goals • Performance measures • Professional development • Communication • Time and resources for implementation • District and school culture changes

6. TPEP Case Studies: Cross-Case Analysis • Each of the site-specific case study reports was reviewed. • Major themes across sites were recorded. • Survey responses from teachers, principals, and administrators across sites were produced to indicate areas of strength and weakness. • The report represents the total sample.

7. TPEP Case Studies: Overall Findings on Teacher Evaluation • The vast majority of teacher respondents in the pilot reported understanding the new system—its goals, measures, and how to achieve a high rating. • They reported that the system is useful and helpful to professional growth. • They see the new teacher evaluation system as better than the previous system in helping them improve their practice. • Simultaneously teacher respondents raised concerns about the new system, particularly the time involved in evaluation activities but also communication with non-pilot teachers and the extent to which the system can account for differences in teachers’ content areas or student populations.

8. TPEP Case Studies: Overall Findings on Principal Evaluation • The development of principal evaluation systems is at various stages in the pilots. While interviewees in a couple districts described a well-specified principal evaluation system, many interviewees in other districts suggest that the development of the system is just beginning. • The vast majority of principals report understanding the goals and measures of the new principal evaluation system. • A strong majority report that it is better than the previous system in terms of helping to improve their practice. • Similarly to the new teacher evaluation system, principals raised concerns about the amount of time involved and communication with non-pilot principals.

9. TPEP Case Studies: General Findings • The new evaluation system takes more time and money to understand and implement than the previous one. • Over 760 total session hours have been spent formally training educators on the new system thus far (through September 30, 2011). • 64.4% of teachers reported spending a much greater amount of time on evaluation activities than previously.  • 65.8% of principals spent more than 10 hours on evaluation activities per teacherin the new evaluation system thus far. • In total, 57.9% of administrators reported spending 21 or more hours on evaluation activities for principals. • Only 47.8% of administrators believe there are adequate monetary resources to support the implementation of the new evaluation systems.

10. TPEP Case Studies: General Findings • The investment may be worth it. The practice of educators, already of high quality in Washington, appears to be improving even more. • The measures being used in the new teacher evaluation system provide feedback for improving instructional practice. More than 88% of teachers and principals agree. • The system supports professional growth. 75.4% of teachers consult the instructional framework to inform their professional practice. 45.8% of principals who report having a leadership framework consult the framework to inform their practice. • Over 84% of teachers and principals agree that the measures in the teacher evaluation system are fair, incorporate the most important aspects, and provide a comprehensive picture of teacher performance.  • 84.7% of principals indicate that their system provides useful feedback to improve their practice as a leader.

11. TPEP Case Studies: General Findings • Over the long-term the investment should continue to pay off. A shift in the culture across schools and districts appears to be happening and the work of the pilots is just the start. • 74.6% of teachers agree that the teacher evaluation system is more objective than before. • Almost 90% of teachers and principals indicate that that the new systems specifies how to improve, identifies areas of strength, and encourages better teaching.  • 70.8% of principals and 90.9% of administrators agree that the new principal evaluation system is better than the old one to improve leadership practice.  • 78.3% of principals and 100.0% of administrators report more productive conversations due to new principal evaluation system.

12. TPEP Case Studies: Next Steps • Spring data collection • Self-report data (time and resources) • Surveys • Document review • Final case studies and cross-case analysis

13. Gretchen Weber P: 630-649-6511 E-Mail: gweber@air.org 1120 East Diehl Road, Suite 200 Naperville, IL 60563-1486 General Information: 800-356-2735 Website: www.air.org