Evaluation: A Challenging Component of Teaching

1. Evaluation: A Challenging Component of Teaching Darshana Shah, PhD. PIES 4-19-2011

2. Next 45 minutes………. • Learning Objectives • Briefly review the characteristics of evaluation • Discuss types of evaluation instruments • Understand the framework for selecting and developing assessment methods

3. What is Evaluations?

4. Evaluation?????? Who Evaluates Whom?

5. Different Settings

6. Purpose of EvaluationWell integration of Teaching, Learning & Assessment

7. Evaluation/ Assessments • Assessments = Observations /measurements • Individual appraisal • Evaluation = interpretations / conclusions • Appraisal of all activities, curriculum, process outcomes, student learning and teacher effectiveness

8. Types of Evaluations • Formative evaluation- provide feedback so the learner can modify their learning approach • Summative evaluation- done to decide if a student has met the minimum course requirements (pass or fail)-

9. Purpose of Evaluation • To certify individual competence • To assure successful completion of goals/objectives • To provide feedback • To students • To faculty, course and clerkship directors • As a statement of values (what is most critical to learn) • For Program Evaluation- evaluation of an aggregate, not an individual (ex. average ability of students to perform a focused history and physical)

10. Consequences of evaluation • Steering effect- exams “drive the learning”- students study/learn for the exam • Impetus for change (feedback from students, Executive Curriculum, LCME)

11. Characteristics of Evaluation • Goal oriented • Value oriented • Comprehensive • Diagnostic valid and reliable

12. Blooms TaxonomyCognitive: mental skills (Knowledge)Affective: growth in feelings (Attitude)Psychomotor: manual or physical skills (Skills)

13. DOES Longitudinal Clinical Observation OSCE SHOWS HOW SP Practical Oral Application Essay Note review KNOWS HOW MEQ MCQ KNOWS Written Observed FORMATIVE SUMMATIVE Erica Friedman’s presentation

14. Take 5 mins……. • Recall; the goals and objectives for your course or clerkship in the context of assessment • Identify the best methods of assessing your goals and objectives

15. Purpose of EvaluationWell integration of Teaching, Learning & Assessment

16. How to Match Assessment to Goals and Teaching Methods • Define the type of learning (lecture, small group, computer module/self study, etc) • Define the domain to be assessed (knowledge, skill, behavior) and the level of performance expected (knows, knows how, shows how or does) • Determine the type of feedback required

17. KNOWS

18. KNOWs HOW

21. How we currently assess ourselves • Student evaluations (quantitative and qualitative)- most often summative • Performance of students on our exam and specific sections of USMLE

22. Things to consider!!!!! • Focus and feedback groups (formative) • Self evaluations- yearly grid completed by course directors and core faculty • Consider peer evaluation of teaching and teaching materials

23. How to be successful • Students should be clear about the course/clerkship goals and the specifics about the types of assessments used and the criteria for passing • Make sure the choice of assessments is consistent with the values of your course and the school • Final judgments about students’ progress should be based on multiple assessments using a variety of methods over a period of time (instead of one time point)

24. Critical factors for choosing an evaluation tool • Type of evaluation and feedback desired: formative/summative • Focus of evaluation: Knowledge, skills, behaviors (attitudes) • Level of evaluation: Know, Knows how, Shows how, Does • Pros/Cons: Validity, Reliability, Cost (time, $ resources)

25. A Brief Overview of TechniquesFor Classroom Setting • Assessing Course related knowledge and skills: • Background Knowledge Probe • Minute Paper • Empty outline • Memory Matrix • Focus listing