Washington State Teacher and Principal Evaluation Project

1. Washington State Teacher and Principal Evaluation Project Conducting High-Quality Observations

2. Entry Task: Confidence Conversation • As you enter, please have a brief discussion with your district team to decide your level of confidence in whether the following statements are true for your district: • Our evaluators know how to minimize bias when collecting observation data. • Our evaluators know how to capture observation evidence using a running record approach to note-taking. • Our teachers trust our observers to capture accurate evidence that will provide them with useful feedback for professional growth. Write your district name on three sticky notes and place them on the confidence scales posted on [INSERT LOCATION] for each statement.

3. Welcome! • Agenda • Connecting • Learning I • Learning II • Learning III • Learning IV • Reflecting • Wrap-Up • Introductions • Logistics • Agenda

4. Modules • Introduction to Educator Evaluation in Washington • Using Instructional and Leadership Frameworks in Educator Evaluation • Preparing and Applying Formative Multiple Measures of Performance: An Introduction to Self-Assessment, Goal Setting, and Criterion Scoring • Including Student Growth in Educator Evaluation • Conducting High-Quality Observations and Maximizing Rater Agreement • Providing High-Quality Feedback for Continuous Professional Growth and Development • Combining Multiple Measures Into a Summative Rating

5. The Evaluation System Components

6. Session Norms • Pausing • Paraphrasing • Posing Questions • Putting Ideas on the Table • Providing Data • Paying Attention to Self and Others • Presuming Positive Intentions • What Else?

7. Connecting Builds community, prepares the team for learning, and links to prior knowledge, other modules, and current work

8. Module Overview: 2 Parts • Conducting High-Quality Observations • Maximizing Rater Agreement Reminder! • This module provides an orientation to the basic concepts. • This module does not go into great depth about evidence relating to any of the specific instructional or leadership frameworks and instead leaves it up to the districts to seek additional training.

9. Overview of Intended Participant Outcomes Participants will know and be able to: • Describe the role of observation as a methodology for gathering evidence of educator performance and its role in achieving rater agreement. • Utilize the revised WACs to identify the number and length of observations for different categories of staff. • Identify common observation biases in their own and others’ practice. • Utilize appropriate strategies for minimizing bias in the observation. • Utilize best practices in collecting observation data.

10. Observation as One Piece of the Puzzle Remember: Observation is one measure within a comprehensive evaluation system that includes multiple measures, including the following: • Self-Assessment • Goal-Setting • Artifact Review Observation is another piece, helping us move closer to enacting TPEP’s Core Principles and the new system

11. TPEP Core Principles “We Can’t Fire Our Way to Finland” • The critical importance of teacher and leadership quality • The professional nature of teaching and leading a school • The complex relationship between the system for teacher and principal evaluation and district systems and negotiations • The belief in professional learning as an underpinning of the new evaluation system • The understanding that the career continuum must be addressed in the new evaluation system • The system must determine the balance of “inputs or acts” and “outputs or results”

12. Rater Agreement Depends on High- Quality Evidence Collection Practices Criterion Rating Summative Rating Evidence Standards • Observation • Student growth data • Artifacts • Other evidence relevant to the frameworks Criterion 1 District- determined process Distinguished Proficient Basic Unsatisfactory State- determined process Distinguished Proficient Basic Unsatisfactory Criterion 2 Criterion 3 Frameworks + Student Growth Rubrics Criterion 4 Criterion 5 Criterion 6 Student Growth Measures (From 3 specific criteria) Student Growth Impact Ratings: Low, Average, High Criterion 7 Criterion 8

13. The extent to which the scores between the raters exhibit consistency and accuracy against predetermined standards. The predetermined standards are the instructional and leadership frameworks and rubrics that define the basis for summative criterion-level scores. Rater Agreement

14. What the Research Says… • Evaluators need multiple opportunities and settings to observe and assess educator practice. • Multiple observations paired with timely feedback are a key part of a strong evaluation system. • Using multiple observers and multiple observations improves the reliability of scores more than having longer observation periods. • Using a mix of shorter and longer observations can decrease costs while maintaining increased reliability of scoring. • Using a mix of internal and external observers (school-based and not) can help determine the validity of practice scores. (Sources: Ho & Kane, 2013; Sartain et al., 2011)

15. Voices on Observation • Handout 1: Studies link classroom observations to student achievement, by Stephen Sawchuck • Handout 2: Feedback for Better Teaching, by The Measures of Effective Teaching (MET) Project. (pp. 1-2 and 6-7; skip pp. 3-5)

16. Voices on Observation Wrap-Up • What are the key messages and ideas in the two articles with regard to observations? • Based on what we just read, what are some best practices for observation?

17. Learning Understand the new requirements for educator observation in Washington, as well as best practices and strategies for conducting high-quality observations

18. Learning Introduction: Meet the New Rules • 1. In the 2012-2013 school year, Mr. Davis was in his eighth year of teaching at your school. He has continuing contract status and was rated “satisfactory” in his last four evaluations. True or False: Mr. Davis is eligible for focused evaluation in 2013-2014. • Answer: True RCW 28A.405.100 (12)(b)(iv)(c)(i)

19. Learning Introduction: Meet the New Rules • 2. Fill in the blank: Mr. Garvey has been a principal in North Thurston for the past five years. He is starting his first year as a principal in your district in September. Mr. Garvey should have a ___________ evaluation [focused or comprehensive] this year. • Answer: Comprehensive RCW 28A.405.100 (12)(b)

20. Learning Introduction: Meet the New Rules • 3. Fill in the blank: Mr. Burke has taught for ten years in your district and has continuing contract status. He is currently on a comprehensive evaluation cycle. Mr. Burke must be observed ____ times this year for no less than _____ minutes total. • Answer: 2 and 60 WAC392-191A-070

21. Learning Introduction: Meet the New Rules • 4. Fill in the blank: Mrs. Wright is a 20-year veteran teacher but just started teaching at your school this year. As a new employee, Mrs. Wright must be observed once for ___minutes during the first ____calendar days of the employee period. • Answer: 30 and 90 WAC392-191A-070 (2)

22. Learning Introduction: Meet the New Rules • 5. Mr. Jones is starting his third year in teaching and is a provisional teacher. This year, he must be observed: • A. At least two times for a total of 120 minutes • B. At least three times for a total of 90 minutes • C. At least four times for a total of 150 minutes • Answer: B WAC392-191A-070 (3)

23. Learning Introduction: Meet the New Rules • 6. Ms. Thomas has been on a focused evaluation for the past two years. In both years, she was rated as proficient. This year: • A. She must have a comprehensive evaluation. • B. She and her supervisor can decide whether to continue with a focused evaluation or switch to a comprehensive evaluation. • C. She must have a focused evaluation. • Answer: B WAC392-191A-070(12)(c)(iv)

24. Learning Introduction: Meet the New Rules • 7. Mr. Tucker had a comprehensive evaluation last year and was rated as “proficient”; however, he and his evaluator identified Criteria 4 as an area that needed improvement. • True or False: Mr. Tucker is required to use Criterion 4 for his focused evaluation next year. • Answer: False WAC392-191A-120(2)

25. Learning Introduction: Meet the New Rules • 8. Mr. Tanyu has continuing contract status and is on a focused evaluation cycle. He selected Criteria 7: “Communicating and collaborating with parents and school community.” You notice that Criteria 7 does not need to be assessed through observation; other methods, such as artifact review, may be more appropriate. Mr. Tanyu: • A. Does not need to be observed this year. • B. Must be observed this year. • C. Districts can decide whether or not to require observation in this situation. • Answer: False WAC 392-191A-110

26. Learning Introduction: Meet the New Rules • 9. Part I: Principal Smith completed an observation of Ms. Williams on Tuesday and promptly wrote up the results of the observation later that day. Principal Smith must give a copy of the report to Ms. Williams within: • A. four days. • B. no more than one week. • C. three days. • 9. Part II: Principal Smith and Ms. Williams must hold a minimum of ____ confidential conferences during each year. • Answers: C and 2 WAC 392-191A-130

27. Learning Introduction: Meet the New Rules • 10. True or False? The purpose of observation is to: • (a) collect accurate, relevant evidence to evaluate against instructional and/or leadership frameworks, and • (b) provide educators with quality feedback. • Answer: True!

28. Learning Content I: High-Quality Observation • Frequent: research agrees that a single observation of teacher or leader practice is insufficient for capturing evidence to generate accurate and reliable ratings. • Comprehensive: captures evidence appropriate for assessing the educator against the required criteria. • Accurate: minimizes bias and error to ensure that collection and review of the data produces an authentic assessment of the educator’s performance. • Useful & Timely Feedback: generates detailed, focused feedback for improvement.

29. Learning Content I Activity: High-Quality Observation • What does it mean to be [INSERT DESCRIPTOR] in your observations? • How does this principle of high-quality observations better support the improvement of teaching and learning?

30. Learning Content I Activity: Debrief • Share out your two key ideas for your Guiding Principle • Frequent • Comprehensive • Accurate • Feedback

31. Learning Content II: Strategies for Reducing Bias in Observation A skilled observer: • 1. Understands each component and indicator on the district rubric thoroughly and deeply. • 2. Gathers and sorts sufficient evidence of practice as it happens in the classroom or school. • 3. Recognizes and puts aside preferences and biases. • 4. Interprets the evidence appropriately to give an accurate rating using the evaluation instrument. (McClellan, Atkinson, & Danielson, 2012)

32. Common Sources of Bias in Observation • Bias enters the process in two places: • Observation bias: while recording data during an observation, every observer has a “lens” altered by: • Specializations: content or grade level • Personal instructional preferences • Past observations or impressions of the educator • Individual bias or experiences. • Rater error and bias: observers rate performance too high or too low based on considerations other than the instructional or leadership framework.

33. Reducing Bias in Observation • Record evidence, not judgment or opinion: • Evidence describes a concrete behavior, practice, interaction, or material that can be used to support a rating decision and provide clear feedback. • Judgment or opinion comments on or interprets what is being observed but does not provide concrete evidence. • Observation notes are written as evidence statements: • Evidence statement: A factual statement, free from opinion or bias, that describes teacher practice.

34. Reducing Bias in Observation • EXAMPLE: Opinion Statement: “Yes! Good questioning, engages the students.” Evidence Statement: T: “How do you know, or how did you get that answer?” S: (Multiple hands shoot up) T: Calls on Alissa S: “I went back and used Pythagorean’s formula.” S writes out process on board and explains each step.

35. Strategies for Recording Evidence • Record: • Direct quotes • Questions asked • Observed actions or movements • Literal descriptors • Avoid: • Jotting down extraneous information • Jotting down initial thoughts on ratings

36. Learning Activity II: Evidence Versus Opinion • Examine Handout 4 • Highlight factual statements • Underline judgment statements or statements not based on evidence

37. Learning Content III: Best Practices for Collecting Observation Data • Holding the Pre-observation Conference • Review the proposed lesson plan • Listen to the teacher’s perspective and learn about the classroom context • Understand the teacher’s professional goals and expectations • Identify any specific areas the teacher would like feedback on

38. Best Practices for CollectingObservation Data • Before Observing: • Focus: Review pre-conference notes and the educator’s professional goals • Clean Your Slate: Purposefully identify and set aside previous impressions from earlier observations

39. Best Practices for CollectingObservation Data • While Observing: • Be “a fly on the wall”: Avoid interacting with the participants to minimize your influence on what you are observing. • Record only information you actually observe—do not “fill in the blanks” or assume other practices are occurring without actually observing them. • Record the depth and frequency of key “look-fors” (e.g., types of questioning, student behaviors, re-direction to task).

40. Best Practices for CollectingObservation Data • Balance taking detailed notes with observing the action in front of you. • Use abbreviations to increase efficiency (e.g., “T” for teacher, “S” for student, or components by their first two letters—“LD” for lesson delivery). • Paraphrase around short quotes to convey the spirit and tone of the interaction.

41. Example Observation Notes Best Practices for CollectingObservation Data

42. Learning Activity III: Practicing Collecting Observation Data • Video Background • “Ms. K” is teaching a sixth-grade science lesson. • 25 students, ranging from 11 to 13 years old, are in her class. • Two students are English language learners and several students have IEPs for accommodations in reading. NBPTS Video #161: Probing Student Understanding (http://198.101.195.20/video.php?p=15346487)

43. Learning Activity III: Practicing Collecting Observation Data • Compare your notes for the following: • Step 1: Do the notes capture the same data? • What are you missing? • Step 2: Do the notes record evidence, such as: • quotes? • questions? • actions or movements? • Step 3: Do the notes include judgments or opinions? • Step 4: Did you identify any personal biases while you were taking notes?

44. Learning Content IV: Overview of eVAL • How many of you know what eVAL is? • Is anyone currently using eVAL? • Is anyone considering using eVAL?

45. eVAL • A web-based tool designed to manage the evaluation process and documentation • A free resource! • Personalized for each district for their instructional framework, resources, and documents • Voluntary for all districts, who can use as many or as few of eVAL’s features as they’d like (or none at all) • Extremely secure, with limited access physically and virtually to its servers

46. eVAL • eVAL helps evaluators and educators in five specific ways: • Practice: Enter formative evaluation data regarding videos or classroom observations and engage in a dialogue about shared (or different) perceptions of quality instruction. • Conference: Plan for pre-conferences, observations, and post-conferences—including sending questions in advance of meetings and scheduling observations.

47. eVAL • Observe: Principals can… • Enter notes during observations (or upload them from other software). • Align evidence from notes to instructional and leadership frameworks and provide feedback to teachers. • Score and annotate the rubric and manage all aspects of evidence gathered in the observation. • Create post-conference observation reports.

48. eVAL • Summary & Reports: Principals can… • Submit final summative scores for teachers to their district office. • Reflect on all the evidence gathered for the year. • Score final summative rubrics. • Print final summative reports, including schoolwide evaluation summaries, individual evaluation trends, and discrepancies between teacher self-assessments and evaluator scores.

49. eVAL • For more information, check out the eVAL website: • http://tpep-wa.org/resources/eval/ • Watch eVAL video walkthroughs: • http://tpep-wa.org/resources/eval/eval-video-walkthroughs/

50. Reflecting