Development of a Standards-Based Classroom Observation Instrument

1. Development of a Standards-Based Classroom Observation Instrument January 10, 2007: AAPT Camille Wainwright, Pacific University wainwric@pacificu.edu • In collaboration with: • Patricia D. Morrell, University of Portland • Lawrence Flick, Oregon State University • Catherine Perkins, Oregon State University • Young-Shin Park, Oregon State University • Adele Schepige, Western Oregon University

2. Background • Oregon Collaborative for Excellence in the Preparation of Teachers (OCEPT) – NSF CETP Project began in 1997 • Focus was on the implementation of reform-based teaching practices in college science and mathematics courses as well as effect upon OCEPT- influenced student teachers and beginning teachers

3. Purpose of OCEPT • To strengthen the mathematics and science preparation of future teachers • To increase the diversity of educators teaching science and mathematics • To build an infrastructure in Oregon to sustain the progress achieved.

4. We tend to teach as we have been taught - Premise College Faculty Preservice Teachers Inservice Teachers Classroom Students

5. Research Questions: • Was there evidence that student teachers’ instructional strategies were influenced by courses from OCEPT Fellows? • Was there evidence that Faculty Fellows’ teaching was impacted by their participation in OCEPT activities? • Did the observed teaching strategies of both groups reflect reform-based practices?

6. Tools: Development of O-TOP Instrument • O-TOP (Oregon Classroom Observation Protocol) • Based on: Reform documents (NSES, NCTM, AAAS Benchmarks, RTOP, CETP tools, Horizon Research Corp., Inc., and other instruments • Incorporates observations of both teacher and student behaviors • 10 items (a manageable number) • Qualitatively measures the amount of frequency with which reform-based teaching techniques are seen in the lesson (NOT a total score or rating) • Reliability and validity studies completed

7. Initial Reliability Study:PerCent Agreement • Item Same Score Within One Point • 1 100% 100% • 2 29% 57% • 3 57% 100% • 4 57% 100% • 5 43% 71% • 6 57% 100% • 7 71% 100% • 8 86% 100% • 9 71% 100% • 10 57% 100%

8. Following Discussion: • Following discussion of the Focus and behaviors described by Items 2 and 5, inter-rater reliability (within one point) was achieved at a rate of 100% on subsequent observations.

9. Item #1

10. Item #2

11. Item #5

12. OTOP ‘Range’ version • A second version of the OTOP was designed which uses a continuous range for evaluation rather than a numerical scale. Not Characterizes Observed Lesson

13. Conclusions – Student Teacher and Beginning Teacher Observations • Reform-based teaching strategies were being used (to some extent) by all participants: student teachers and beginning teachers including 3-year case studies. • Qualitative evidence of connections between reform-based strategies experienced in undergraduate mathematics and science courses and new teachers’ methodologies. • No discernible pattern in the frequency of the items among the participants over the three year span.

14. Conclusions – Faculty Observations and Interviews • Reform-oriented strategies are evident • Examining faculty particularly successful with strategies can provide models for others • There were areas of weakness and strengths • Science strengths: • Incorporating collaboration into lecture formats • Encouraging metacognition, challenging of ideas, generation of alternative solutions or interpretations

15. Conclusions, continued (Faculty) • Mathematics strengths • Use of multiple representations • Interdisciplinary connections • Mathematics for Elementary Teachers strengths • Interdisciplinary connections • Encouraging divergent thinking • Promoting conceptual understanding

16. Implications: • O-TOP can be used to provide descriptive feedback to faculty • Can be used to identify areas in which additional support to higher education faculty is necessary • Instrument provides a common language for higher education faculty to use

17. Implications, continued: • Faculty may develop increased awareness of instructor-student interactions • Peer observations may increase faculty interest in new teaching approaches • Collaboration between science, mathematics, and education faculty could result

18. Additional(Unanticipated) Applications of O-TOP: • School of Education University Supervisors have used the instrument to provide feedback to their student teachers following observations in the field. • Many college, university and community college faculty have adopted the O-TOP as the protocol for implementing peer reviews within their departments. • New teachers have indicated that the O-TOP provides a user-friendly checklist of best practices to consider during lesson and unit planning.

19. Additional (Unanticipated) Applications of O-TOP : • Experienced K-12 teachers have utilized the observation protocol as a self-assessment component of their ongoing professional development. • K-12 faculty have asked principals to use the O-TOP during the annual evaluation process (especially principals who are unfamiliar with standards-based teaching in mathematics and science). • Even faculty outside of mathematics and science education have commented on its ability to describe effective teaching within their content areas

20. Camille L. Wainwright, PhD Pacific University Forest Grove, OR 503-352-2963 wainwric@pacificu.edu