Multiple Measures in Teacher Evaluation

1. Multiple Measures in Teacher Evaluation Measures of Teacher Effectiveness Emphasizing Student Learning Volusia Teachers Organization September 27, 2010

2. AFT’s new approach to teacher development and evaluation In order to increase student achievement and improve schools, the focus of teacher evaluation must always be on the continuous improvement of teachers throughout their careers.

3. The role of teacher evaluation in systemic reform Evaluation is only one critical component needed to increase student learning and ensure a great teacher in every classroom. Issues that districts have to grapple with when looking at Teacher evaluation systems: • Are there clear and coherent standards? • Is the curriculum rich and non-scripted? • Are there high-quality student assessments? • Are there support systems including ongoing, job embedded professional development in place for teachers to develop and improve? • Are there adequate resources available and is the school environment conducive to teaching and learning?

4. A Standards-Based Framework for Comprehensive Teacher Development and Evaluation Professional Teaching Standards Assessing Teacher Practice Systems of Support Implementation Professional Context

5. Standards for Assessing Teaching Practice Based onevidence of both good teaching practice and student learning. When assessing teacher practice, it is necessary to: • Use valid and reliable measures; • Include performance standards that are appropriate for the developmental level of the teacher—novice, mid-career and veteran; and • Incorporate a variety of evaluation techniques to capture the breadth of good teaching and professional practice, including classroom observations, review of lesson plans, self-assessments, teaching artifacts and portfolio assessments.

6. Evidence Adapted from Goe, L. 2010 NCCTQ • Evidence is what you collect in order to build a case for a particular recommendation • If you’re evaluating teachers, you collect strong evidence (whether you find it in a classroom observation or through an analysis of the teachers’ contribution to student learning growth) that allows you to create a profile of a teacher’s effectiveness • Evidence is strongest when the measures and procedures used are standardized and can demonstrate growth.

7. How to use multiple measures to define teacher effectiveness Evaluating teachers fairly and accurately requires the use of multiple indicators of effectiveness AND multiple ways to measure those indicators. These indicators can be combined in various ways based on their relationship to student learning or organizational values

8. Why multiple measures? Increases validity (appropriateness) Increases reliability (consistency) Decreases subjectivity Increases comfort level of both evaluator and teacher Strengthens decision-making process They create a more accurate measure of teacher performance

9. Any evaluation system that includes student learning, must include multiple measures if ALL teachers—regardless of grade or subject—are to receive constructive feedback and accurate ratings. Multiple measures are the only way to ensure comprehensive evaluations for ALL teachers. Why multiple measures? Currently, only a fraction of teachers teach in subjects or grades that have large-scale standardized tests.

10. Multiple measures of teacher performance and student outcomes Depending on how we define an “effective teacher,” we could look at multiple measures when evaluating teachers: Evidence of instructional quality Evidence of student learning Evidence of professional responsibility

11. Evidence of student learning Student written work, performances, group work or presentations scored using standard rubrics Scores on district wide designed assessments Teacher developed assessments Portfolios Student learning objectives Grades IEP goals and objectives Student “capstone” projects (e.g., graduation, end-of-course, research or thesis paper, portfolios of art work, service learning projects)

12. Evidence of student learning cont’d. Student written reflections on work/performance Notes/logs from teacher-student conferences Feedback from computer simulated tasks Homework completion rates Running records Formal observations of student growth (e.g., presentations, group discussions, student interactions) Student verbal responses (“Think Alouds”) Completed student work contracts Assessment rubrics

13. Student learning as measured by student learning objectives (“SLOs”) Student learning objectives are annual targets for growth that a teacher sets at the beginning of the year and strives to attain by the end of the year (or at the end of a semester if appropriate). They are informed by a student needs assessment and aligned to the school’s standards and curricula. In addition, SLOs must be: based on outcomes (not activities) rigorous measurable. Can be developed for any teacher in any subject area or grade level. Baseline data can be collected to inform student growth targets. Assessments can be standardized or teacher-developed.

14. Austin Independent School District Student Learning Objectives: Each teacher sets two SLOs for the semester/year One SLO must address all students—others may be targeted Use broad array of assessments (go beyond state assessment) Assess student needs more directly Align classroom, campus, and district expectations Aligned to state standards/campus improvement plans Based on multiple sources of student data Assessed with pre and post assessment Targets of student growth Peer collaboration

15. Evidence of professional responsibility Administrator, supervisor reports Logs and documentation of professional activities Handouts on classroom procedures, rules and consequences posted in the room Letters to families, emails to families, family member contact logs, conference notes Reflection notes or log, notes from professional conversation with colleagues including mentors, notes from peer observations

16. Ways to combine evidence in a teacher evaluation system Conjunctive—All measures weighted equally Compensatory—Higher performance on one measure can compensate for lower performance on another or different measures have different weights Complimentary—Must achieve the standard on just one of the multiple measures Blended—Any combination of the conjunctive, compensatory and complimentary approaches

17. Examples of teacher evaluation systems that incorporate student learning The District of Columbia’s IMPACT Georgia’s CLASS Keys—Classroom Analysis of State Standards

18. DCPS IMPACT“Teaching and Learning Framework” What teachers should know and be able to do

19. DCPS IMPACT“Group 1 Teachers” Only student test scores Only 18% of DC’s teachers

20. DCPS IMPACT“Group 2 Teachers” Multiple measures of student learning

21. DCPS IMPACT Uses a “compensatory” approach

22. Georgia’s CLASS Keys—Classroom Analysis of State Standards What teachers should know and be able to do

23. Georgia’s CLASS Keys—Classroom Analysis of State Standards

24. Student achievement standard for teachers of state tested subjects and grade 24

25. Student achievement standard for teachers of state tested subjects and grade Examples of Data Sources

26. Teachers of non-tested grades and subjects 26

27. Georgia’s CLASS Keys uses a blended approach The annual evaluation process The evaluation of performance occurs at the ELEMENT level Element scores are aggregated to form STRAND rating To be “Satisfactory,” ALL STRANDS must be “Emerging” or higher

28. Georgia teacher duties and responsibilities 28

29. Georgia’s CLASS Keys uses a compensatory and conjunctive approach An Example For “Satisfactory” ALL Strands must be “Emerging” or higher.

30. The dilemma • Tension between federal and state pressures and teacher beliefs about the measures and evidence of teacher effectiveness • Teachers want a greater focus on their work in the classroom with students (instructional quality) • Assumes shared responsibility for student learning • Federal and state pressures place a greater emphasis on student achievement (as measured by student test scores • Assumes that teachers are primarily responsible for student learning

31. Evidence-based improvement Create a culture of using evidence to drive improvements in teaching and learning Good teacher evaluation should provide evidence about teachers’ practice that can be used to help teachers improve their instruction In a teacher development and evaluation system, keep the focus on improving instructional quality, not raising test scores

32. AFT’s Teacher Evaluation CommunityRegister at: Go.aft.org\teacherevaluation

33. Contact information Justin Stone Assistant Director AFT Innovation Fund jstone@aft.org