Washington State Teacher and Principal Evaluation Project

1. Washington State Teacher and Principal Evaluation Project Preparing and Applying Formative Multiple Measures of Performance Gathering Evidence

2. Entry Task • As you enter, jot down on a sticky note: • One disadvantage of having an observation only evaluation system • One advantage of the current evaluation system

3. Welcome! • Agenda • Connecting • Learning • Implementing • Reflecting • Wrap-Up • Introductions • Logistics • Agenda

4. Modules • Introduction to Educator Evaluation in Washington • Using Instructional and Leadership Frameworks in Educator Evaluation • Preparing and Applying Formative Multiple Measures of Performance: An Introduction to Self-Assessment, Goal Setting, and Criterion Scoring • Including Student Growth in Educator Evaluation • Conducting High-Quality Observations and Maximizing Rater Agreement • Providing High-Quality Feedback for Continuous Professional Growth and Development • Combining Multiple Measures Into a Summative Rating

5. Session Norms • Pausing • Paraphrasing • Posing Questions • Putting Ideas on the Table • Providing Data • Paying Attention to Self and Others • Presuming Positive Intentions • What Else?

6. Connecting Builds community, prepares the team for learning, and links to prior knowledge, other modules, and current work

7. Overview of Intended Participant Outcomes for Part D Participants will know and be able to: • Determine specific measures to support implementation of evaluation in their districts • Understand the evaluation cycle across a school year and how multiple measures fit in • Articulate ways to integrate multiple forms of evidence into educator evaluation to move beyond an observation-only evaluation system • Determine how to create criterion scores after gathering of evidence aligned with the Washington State Criteria

8. Guidance Icon Key G! RCW 28A.405.100 g G! RCW 28A.405.100 A capital “G!” indicates that the guidance represents Washington state law. A lower-case “g” indicates that the guidance represents research-based best practice but is not mandated by law.

9. Connecting the Pieces of the Puzzle • Part A described how using multiple measures strengthens a teacher evaluation system. • Part B discussed self-assessment. • Part C discussed goal setting. • The goal of criterion scoring is to begin the process of putting the pieces of a comprehensive teacher evaluation system together.

10. The Year-Long Evaluation Cycle Criterion Rating g Summative Rating G! StandardsG! Evidenceg Frameworks + Student Growth Rubrics 8 Criteria • Observation • Student Growth • Evidence District- determined process Distinguished Proficient Basic Unsatisfactory State- determined process Distinguished Proficient Basic Unsatisfactory • Step 1 • Criteria aligned to instructional /leadership and student growth rubrics • Professional goals (g) • Instructional/leadership goals (g) • Student growth goals (G!) • Step 2 & 3Select and collect evidence • 2 observations (G!) • Student growth (G!) • Other evidence (g) Step 5 Summative Score (G!) Step 4 Determine 8 Criteria Scores (g)

11. Evidence Cover Page

12. Evidence Collection Form

13. Digging for Evidence: Evidence for John Anderson • Let’s return to his self-assessment on Criterion 5 • Student strengths: Last year’s data demonstrate that students are well-behaved. • Student weaknesses: Students need to learn how to regulate their behavior; students need to learn how to work constructively with their peers. • Teacher strengths: Scored high on Criterion 5 last year; Students felt comfortable asking for help from the teacher last year • Teacher weaknesses: Did not participate in service-learning projects last year; decreased engagement in classroom as a result.

14. Digging for Evidence: Artifacts of John Anderson • Included are six artifacts that can be used as evidence for Criterion 5. • Evidence cover page has two things missing • Alignment to instructional framework dimensions in criterion 5 • Evidence statement • With your partner, take a set of artifacts, align it to the framework in Criterion 5, and develop evidence statements

15. Digging for Evidence Debrief • With your district team: • Do you think you need all six pieces of evidence to measure Criterion 5? Why or why not? • How would you score each piece of evidence? • How would you put all of this evidence together to create a score for Criterion 5?

16. Critical Decisions About Methods • Key messages for multiple measures • Need a variety of evidence • Not about collecting as much asyou can; this is about quality, not quantity • Evidence should be based on naturally occurring documents/materials • Prioritize evidence collected based on goals and criteria.

17. Learning Understand best practices in gathering evidence

18. OSPI’s: Guiding Principles for Criterion Scoring • The primary goal of any system of teacher evaluation is to promote teacher and student learning. • Accurate teacher evaluation requires trained observers using a research-based instructional framework. Trained observers make accurate assessments of practice based on evidence. • The value of accurate assessments of practice is to shape the conversations that lead to improved practice. • Embedded in each instructional framework is a system for growth in teaching practice. • Reliability and validity of the instructional framework relies on implementation of the full framework rather than individual components/indicators. • It is imperative to remain in the formative mindset until the final summative rating is determined.

19. The Criterion Scoring Process • Step 1. Collect, align, and draw conclusions about evidence based on framework alignment to criteria • Step 2. Determine score for each indicator based on preponderance of evidence • Step 3. Enter score in Criterion Scoring Matrix • Step 4. Determine criterion score Criterion Scoring Process

20. Learning Activity: Step through the process Step 1: Collect, align, and draw conclusions about evidence based on framework alignment to criteria. • Organize the evidence that you have – both scores and more formative information • Consideration: • How did you score the evidence? Is it on the same scale as the criteria? If not how do you convert it? • If evidence aligns to multiple indicators, is the score the same on each indicator? • For example, if you administer a student survey that relates to multiple indicators, will each indicator get the same score, or a different score based on the dimension in the survey?

21. Step 1: Sorting through the evidence Criterion 5: Learning Environment (Danielson)

22. Learning Activity: Step through the process Step 2: Determine score for each indicator based on preponderance of evidence • Based on all of the evidence that you have, determine a preponderance score for each indicator. • Consideration: • Does some evidence have more weight than other evidence? • Does the evidence provide enough information to be as accurate and close to truth as possible?

23. Step 2: Creating indicator score Criterion 5: Learning Environment (Danielson)

24. Learning Activity: Step through the process Step 3: Enter score in Criterion Scoring Matrix • Based on the indicator scores you determined in step 2, place the scores in the Criterion Scoring Matrix Criterion 5: Learning Environment (Danielson)

25. Learning Activity: Step through the process Step 4. Determine Criterion Score • Use the indicator scores in the Criterion Scoring Matrix to determine criterion score. • Analyze the combination of scores by each indicator within a criteria to determine the score. • Considerations: • Based on the preponderance scores for the instructional framework indicators, is there a clear criterion score (e.g., are the majority of indicators scored at one level)? • If there is not a clear criterion score based on the preponderance scores, use the guiding questions in the scoring document (next slide).

26. Learning Activity: Step through the process Step 4. Determine Criterion Score Guiding Questions • What else do I need to see or consider to make a final decision – what is available to me? • What is the distribution of evidence over time? • Has there been demonstrated and consistent improvement? If there was growth, was the growth sustained? • What would be the tipping point? If I consistently saw “X,” I would feel confident that the performance is Basic. If I consistently saw “Y,” I would feel confident that performance is Proficient. • What is the essence of the indicator? The criterion? Go back and find the key words in the framework/rubric. What does the evidence tell you about the evaluatee’s performance and growth with regard to this essential aspect of the indicator/criterion? • Is this evaluatee more Basic than s/he is Proficient, or more Proficient than s/he is Basic in this indicator? What is the evidence based in the framework/rubric to support your decision?

27. Learning Activity: Step through the process • Step 4. Determine Criterion Score – clear score. Criterion 5: Learning Environment (Danielson)

28. Learning Activity: Step through the process • Step 4. Determine Criterion Score – unclear score • If, for example, the indicator scores were as follows, what would you do? This is where the formative evidence is very helpful. Criterion 5: Learning Environment (Danielson)

29. Learning Activity 2: Austin’s Butterfly • We are going to watch a short video, Austin’s Butterfly • http://vimeo.com/38247060 • After the video, discuss: • How did you feel after watching the video? • How does this video relate to the guiding principles of OSPI? • How does this relate to educator evaluations and the goal of evaluations?

30. Learning Activity 2: Try it yourself. • Use the Criterion Five evidence from the beginning activity to go through the 4-step criterion scoring process • Step 1. Collect, align, and draw conclusions about evidence based on framework alignment to criteria. • Step 2. Determine score for each indicator based on preponderance of evidence • Step 3. Enter score in Criterion Scoring Matrix • Step 4. Determine criterion score

31. Learning Debrief • With your district team: • What did you like about the criterion scoring process? • What did you find difficult about this process? • Share one outstanding question you have about the criterion scoring process.

32. Implementing Articulate the ways collecting evidence through artifacts can operationalize the revised evaluation system to improve teaching and learning

33. Increasing Clarity and Feasibility • Districts should think strategically about how to increase clarity around the teacher evaluation process and ensure the process is feasible for teachers and evaluators. • Some examples follow on the next few slides.

34. Increasing Clarity and Feasibility – Step 1 G! RCW 28A.405.100 • Create a table with the relevant evaluation measures listed for all Washington State Criteria – so everyone knows what measures should contribute to each rating • This also shows teachers where to focus their evidence collection and decreases the amount of evidence evaluators need to review • Use Handout 13A, B, or C depending on instructional framework

35. Increasing Clarity and Feasibility – Step 2 g G! RCW 28A.405.100 • Describe the entire evaluation process in detail, describing roles of teachers and evaluators. • Use Handout 14, which provides multiple questions to consider.

36. Identifying Tools and Processes for Gathering and Organizing Evidence g G! RCW 28A.405.100 • Create a plan for using Evidence Review in teacher evaluation. • Use the “Plan for Implementing Evidence Review” Handout 15 of your packet to structure your conversation with your school team.

37. Implementing Activities Debrief • Each team shares one decision that was made today to increase the clarity and feasibility of the teacher evaluation process.

38. Reflecting

39. Whip Around and Plus/Delta Debrief • Whip Around: One significant “ah-ha moment” today • Take a few minutes and create at least two sticky notes for the Plus/Delta Chart on your way out. • Plus: What was a real “plus” of today’s session? What went well and should be repeated? • Delta: Where is there room for improvement and change?

40. What’s Next? • Homework Options • District: Start a district teacher evaluation guidebook that includes all of the nuts and bolts of the teacher evaluation process. Use the information you recorded on the “Plan for Implementing Evidence Review” handout as a starting place. • School or Teams: Identify times when teachers are already asked to collect artifacts – whether that means submitting lesson plans or turning in PLC meeting minutes. • Individual: Consider authentic evidence of practice that you create. Choose a couple and identify which criteria they provide evidence of.

41. Thank you! INSERT PRESENTER’S NAME AND E-MAIL ADDRESS