1 / 57

Peer Reviewer Training Workshop

Peer Reviewer Training Workshop. Shea Wang & Trudi Ohki CAFÉ Sept. 22, 2014. Workshop Overview. Upon completion of this workshop, you will be able to: Conduct a pre-observation meeting Review teaching materials

colin-neal
Download Presentation

Peer Reviewer Training Workshop

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Peer Reviewer Training Workshop Shea Wang & Trudi Ohki CAFÉ Sept. 22, 2014

  2. Workshop Overview Upon completion of this workshop, you will be able to: • Conduct a pre-observation meeting • Review teaching materials • Use the narrative, checklist, rating, and data collection forms for classroom observation • Conduct a post-observation debrief • Write a final post-observation report

  3. Activities Overview • Practise using the pre-observation meeting form • Practise using the review of teaching materials form • Practise classroom observation skills and techniques • Practise using the narrative, checklist, rating, and data collection forms for classroom observation • Practise post-observation debrief • Practice post-observation report writing

  4. Faculty Evaluation Policies: Instructional Faculty – C5065 Non-Instructional Faculty – C5066

  5. Evaluation at MacEwan Policy C5065 guided by the philosophy that we: • promote a culture of teaching and learning • foster professional development and scholarly activity • promote fairness and transparency • timely and accurate feedback • comply with collective agreements • undertake as a process involving multiple stakeholders, and a variety of assessment approaches

  6. Evaluation at MacEwan Faculty strive for excellence in teaching, which is assessed by: teaching (including curriculum development) research, scholarly, or artistic engagement service (collegiate citizenship) collegiality (professionalism, respect for students & peers, and ethical behaviour) professional development

  7. C5065 – Instructional Faculty Requirements based on appointment type

  8. C5065 – Probationary Requirements

  9. Introduction What is Peer Review? • Scholarly publications – review process to assess quality of the work • Evaluation of classroom/online teaching and teaching materials • ASAC position review committees • Any evaluation process where the evaluators are peers

  10. Peer Review Literature Peer review is appropriate for… • Course content, expertise in field of study • Course objectives and materials • Appropriate teaching methodology • Course organization • Appropriate tests, assignments, evaluation methods Seldin, 1999

  11. Peer Review at MacEwan • Teaching effectiveness • Classroom visits • Teaching materials • ASAC position review committees

  12. Qualifications of Peer Reviewers • Criteria developed by FEC • Completion of an appropriate peer review training course… and / or demonstration of peer review experience • Minimum 3 years full-time teaching experience • Pedagogical expertise • Possess a continuing appointment • Recommendation by Dean / Associate Dean / Director / senior administrator

  13. Attributes of Peer Reviewers • Utilizes effective instructional practices (demonstrated through dossiers, annual reports, awards, etc) • Demonstrates collegiality • Provides constructive, objective feedback focusing on teaching behaviours and improvement • Models ethical behaviour • Appreciates different styles / modes of teaching

  14. Exceptions… • For all peer reviewers, should there be a deficit in meeting the recommended qualifications, the Dean, Associate Dean, Director, or senior administrator will assess and provide final approval.

  15. Choosing Peer Reviewers • Chairs will support and advise faculty members in the selection of two peer reviewers to conduct classroom observations and review of relevant teaching materials. • It is not necessary that peer reviewers be content-area experts. • Peer reviewers can expect to be asked to perform peer reviews outside of their home department(s).

  16. Defining Effective Teaching Conversations about teaching (30min) • Pair and Share Break into small groups and discuss the following sentences: I most enjoyed teaching when __________ I knew I had a problem when __________ • Have a member of your group report on your discussion to the main group *Chickering and Gamson, 1987

  17. Measuring Good Teaching Myths • Dr. Fox… • Expressiveness… • High Ratings Reflect Lenient Grading • Workload/course difficulty… • Rank and experience… • Gender… https://facultycommons.macewan.ca/headlines/professors-from-sfu-share-mean-student-evaluations

  18. Measuring Good Teaching

  19. Measuring in context • Take 15 min. and complete the following sentences in your group: • I knew my teacher was a good teacher when/because… • I knew my teacher was a poor teacher when/because…

  20. Literature on “Good” Teaching Eight dimensions of college instruction : • Course organization & planning • Faculty communication • Faculty/student interaction • Assignments, exams, & grading • Instructional methods & materials • Course outcomes • Student effort & involvement • Course difficulty, workload, & pace (Marsh, 1984 & 1987)

  21. Literature on “Good” Teaching (2) • Encourages student interaction • Encourages student cooperation • Active learning • Prompt feedback to students • Emphasizes time on task • High expectations • Respect (Chickering and Gamson, 1987)

  22. Literature on “Good” Teaching (3) Good teachers… • Are good learners; • Display enthusiasm; • Know how to modify their teaching strategies; • Encourage learning for understanding; • Demonstrate an ability to transform and extend knowledge, rather than just transmit it; • Set clear goals; • Provide high-quality feedback • Show respect for their students (Ramsden, 2003)

  23. Overall… “Good” Teaching • Parker Palmer – “capricious chemistry” • Good teaching is ultimately irreducible to a simple technical formula • Characteristics are beyond common descriptors or attributes of a checklist • Solution: multifaceted approach • Student feedback, peer review, self-reflection, and administrative evaluation

  24. Peer Review Activity In your small group, reflect on your past experiences with giving and receiving peer feedback. Address the questions: • What did you learn from the experience? • In what ways it was a positive/negative experience? • Did it influence your instruction? • How did receiving and giving feedback change the way you thought about teaching and learning?

  25. Peer Observation The two forms of observation: • Summative observation - This observation is an evaluation. It can be done for administrative or supervisory reports for the purpose of job retention, promotion, salary increases, etc. • Formative observation - The purpose of a formative peer observation is to help instructors become better teachers and more knowledgeable professionals.

  26. Peer Review Process at MacEwan Evaluation Policy outlines two main areas of teaching assessed by peer reviewers: • Teaching materials - lecture notes, course outlines, tests and exams, assignments, grading rubrics, etc.; and • Instructional delivery - observation of teaching in formal lectures, as well as other delivery modes such as labs, studio, clinical, and online/distance formats

  27. Peer Review Process at MacEwan Five main elements: • Pre-observation meeting • Review of teaching materials • Classroom observation • Post-observation debrief • Post-observation report

  28. Forms Some of the forms available… • Narrative form • Checklist form • Rating form • Data Collection (see appendices of workbook)

  29. Pre-Observation Meeting • “Emotionally, it is important to address distrust of evaluation, violation of personal style and space, insecurities about time, and fear of bias” (Chism, 2007) • Essential that trust be established - Acknowledge the emotional aspect. • Clear objectives of observation should be outlined - Communicate expectations • Stressed that the process is collaborative - Engage in active listening skills (Review pre-observation forms in appendices)

  30. Course Materials Some examples of materials include: • Syllabus • Course guides • Course packets and assigned texts • PowerPoint Presentations , overheads, BlackBoard, etc • Handouts • Videos and other multimedia supplements • Project assignment directions and handouts

  31. Procedure for Evaluation of Course Materials • Importantly, we are not focusing on student achievement measures when reviewing teaching materials. Rather, we are looking at content, design, clarity, etc • Materials enable reviewers to see an instructor's philosophy enacted, and can alert reviewers to the areas in which a given instructor excels • Materials can promote discussion about the overall goals (Review course materials checklist in appendices)

  32. Classroom Observation • Observers should use the same observation form – they should take separate minutes/notes • At least two observers should attend each class observation session • At least two visits is preferable • No unannounced visits • Discuss with the instructor the format of the review prior to visit: Where will you sit? Will you be introduced?

  33. Classroom Observation During the Classroom Visit… • Be unobtrusive (basically, remain quiet) • Do not ask questions • Do not volunteer answers • Do not correct the instructor • Be attentive • Note specific examples of observed behaviours (positive & negative)

  34. Classroom Observation Challenges • One class may be not representative • Atypical performance (good or bad) • Observer may not have a good understanding of the context • Personal biases on what constitutes “good teaching”

  35. Some Solutions • Attend 2+ classes • Video tape class? • Focus-group students after class? • Use the data collection form, cite examples • Discuss visit with co-reviewer, debrief together • Contextualize the course – reflect back on: • Pre-observation meeting • Examination of course materials

  36. Peer Observation Exercise Pre-observation… • In your small group, decide on the form you will be using for review (narrative, checklist, rating) • Review form and highlight areas that each reviewer will be responsible for focusing on and taking notes about Observation… • Choose one of the two scenarios to review • Reflect on the class from the following viewpoints: • Viewpoint 1: students • Viewpoint 2: instructor

  37. Peer Observation Exercise • Compare your list of observer behaviors with others in your group • Be reflective and systematic through the criteria outlined in your observation form • Embed your observations & conclusions in examples of observed behaviours • Did the instructor’s behavior appear to be what you might expect in a formal observation?

  38. Peer Observation Exercise Role Play • In your small group, designate an individual to be the instructor, the reviewer, and a reporter • Break into large groups based on role as “instructors” & “reviewers” & “reporters” • Compare the notes your groups took during the observation and discuss strategies for approaching the debrief (and how the characters might react) • Break back into your small group and role play the post-observation debrief meeting

  39. Post Observation Debrief • Meet with the instructors immediately following the classroom visit • Ask the instructor to describe how they felt the class went in relation to the goals, objectives and expectations outlined in the pre-observation meeting • At this time, the peer reviewers should bring up any immediate concerns that were highlighted by the observation session. **This is an important aspect of the review as it can potentially change the observers’ perceptions about what they saw and heard.

  40. Post Observation Debrief Procedure • Ask the person who taught how s/he felt the lesson went • Reconfirm the observation focus points for the final report • Offer positive comments on what you think went well • Identify any urgent issues • Ask questions for clarification, as needed • End on a courteous note • Provide an expected timeline for the final report and final meeting

  41. Post-Observation Exercise • Prepare yourself to care – watch that you are not overly forgiving or overly critical (put yourself in their shoes) • Use descriptive language rather than evaluative • Be responsive to the goals outlined in the pre-observation meeting • Check to make sure the instructor has heard your message as you intended • Be forward-looking and positive in your discussion, i.e. what can be improved over what went wrong

  42. Peer Review Final Report Policy C5065 states… • Section 4.4.4, “Peer reviewers shall prepare brief written assessments, grounded in evidence, and submit a copy of reports to the faculty members and Chairs.”

  43. Who Sees the Final Report? • Faculty member – used to improve teaching effectiveness, make adjustments as necessary for application for continuing status • Chair and Dean – used for administrative review • ASACs – used for personnel decisions

  44. Characteristics of a Good Report • Reference the goals established during the pre-observation meeting • Provide a written record of what was observed and what was discussed • Consider the audience • Use examples from the observation • Language is clear, concise, avoids jargon and abbreviations • Tone is supportive and phrasing is positive • Action-oriented and provides specific and practical suggestions for improvement

  45. Characteristics (cont’d) • One, joint report per instructor - Structure of report should be agreed upon by all reviewers • Return report in a timely manner • Review with faculty member within 3 days of classroom visit • Consider faculty member response before submitting for administrative review • Confidential (Review report template in appendices)

  46. Peer Review for Online Courses • Online Peer Review Form (4 categories) • Student behaviour • Faculty-student interaction (multiple interactive communication strategies) • Technology support • Learning environment

  47. Challenges - Peer Review for Online Courses A connection between specific materials and instruction and student action that manifest learning objectives. Technology and accessibility – Are instructors responsible for these? Yes, no, maybe, more nowadays?

  48. What Is A Good Online Course? • Content? Easy use of technology? Interaction? • Learner-centered outcomes-based instruction: • Clear objectives as stated in: • Syllabus: must be clear. Super important. • Grading Rubrics: explains how they will be graded Cont’d.

  49. What Is A Good Online Course? • Content? Easy use of technology? Interaction? • Learner-centered outcomes-based instruction • Clear objectives as stated in: • Models or Examples of quality work from successfully submitted and graded student work • Explanations of the relevance of course materials to learning. • WHY are we doing this? (think of students perspective). Cont’d.

  50. What Is A Good Online Course? • Content? Easy use of technology? Interaction? • Learner-centered outcomes-based instruction • Assessment tools should be used: • Throughout the course to establish: • Learning styles • Skill/knowledge level • Challenge preconceptions • Clarify current learning Cont’d.

More Related