Assessment and Evaluation of Medical Students

1. Assessment and Evaluation of Medical Students Dr. Heidi Combs Dr. F. Curt Bennett

2. This is not the reaction we want the student to have when their grade is posted for the clerkship but sometimes this is how they feel!

3. How does this happen? • It occurs when there is a disconnect between what the student perceives and what the preceptor perceives.

4. At the end of our session you will be able to: • Describe the difference between assessment and feedback • List the steps in assessing a student • Appreciate different strategies in assessing where your learner is • Understand how to put the assessment together in the evaluation of a student

5. What is assessment?? Is that another word for feedback??

6. Assessment vs feedback • Assessment is the opportunity to consider the stage and progress of the learner. It flows naturally once expectations are set • Feedback is the opportunity to turn an assessment into a vehicle for growth for the learner • While giving feedback can be difficult to do, it is critical for learner development

7. Getting started

8. Step One: Know the goals • Understand the goals of the clerkship. These magic things live on the websites of all of the clerkships! It seems simple enough but often direct preceptors are not fully familiarized with them.

9. Step Two: Set expectations • Expectations flow from the goals of the clerkship • Set expectations at the beginning of the rotation and refresh them often • The expectations are a moving target since the student will be gaining knowledge and skills throughout the rotation • Realize different students will progress at different rates

10. Step Three: Share expectations • Students know the goals of the clerkship but they do not know your expectations • If the expectations are not known by the learner, the feedback based on the assessment will feel unsubstantiated

11. Step Four: Assess your learner • Assessment flows from expectations • This includes not only how they are progressing but where they are starting!! Knowing where they are allows us to move them to the next level. • Assess multiple events • Everyone has good and bad moments – recognize patterns • Write down notes of what you thought at the time

12. Keep in mind not all learners are the same!

13. Matching learner states to teaching styles Grow. G. Teaching learners to be self directed. Adult Educ Q 1991;41:125-129

14. Matching learner states to teaching styles • As a learner progresses there is often a shift from being dependent (needs substantial input and direction) to being interested (learner needs some guidance) to self- directed (learned takes personal responsibility for his/her learning) • Too much of a mismatch can be defeating but a degree of mismatch can be positive.

15. Areas for assessment • Attendance • Attitude toward the clerkship • Knowledge base • Thoroughness and accuracy of patient assessments • Mastery of skills i.e. physical exam • Interpersonal manner and communication with patients, families, staff, peers, residents and attendings • Professionalism such as acting with integrity, respect and compassion Psychiatric Residents As Teachers: A Practical Guide Second Revision Committee on Graduate Medical Education 2001-2002

16. Challenges with assessments • We tend not to distinguish between items- if a learner performs well in one area we tend to assess them well in other areas (halo effect) • Personality traits (e.g. extroversion, introversion) may have a positive or negative impact on our assessment • If we assess long after the actual training time we tend to mark toward the mean Turnbull J., Van Barneveld C. Assessment of clinical performance: in-training assessment. In: Normal GR, van derVleutem CPM, Newble DI editors. International handbook of research in medical education. Dordrecht, Netherlands:Kluwer Academic Publishers, 2002;793-810

17. Just a few notes on teaching

18. Stages in acquiring skills Awareness Learning Practice Peyton JWR. The learning cycle. In Peyton JMR, editor. Teaching and learning in medical practice. Rickmansworth, UK: Manticore Europe Limited, 1998:13-19

19. Teach The Range Of What We All Know And Do

20. Practical teaching options: Ways to impart knowledge • Demonstration- perform without commentary • Deconstruction-demonstrate while describing steps • Comprehension- trainer demonstrates while learner describes steps • Performance –learner demonstrates while describing the steps Walker M, Peyton JWR. Teaching in theatre. In: Peyton JMR, editor. Teaching and learning in medical practice. Rickmansworth, UK: Manticore Europe Limited, 1998:171-180

21. Hierarchy of knowledge: Questions to assess each level • Facts: What is the name of ….? • Comprehension: What are the causes? • Synthesis: What do these findings mean? • Analysis: What is the diagnosis, plan and likely outcome? • Evaluation: How well did you manage this patient? What have you learned? Peyton JWR, Allery L Setting Objectives. In: Peyton JMR, editor. Teaching and learning in medical practice. Rickmansworth, UK: Manticore Europe Limited, 1998:57-67

22. Steps to teaching • Create an appropriate environment • Assess the level and needs of the learner • Match your teaching style and content with the needs and level of the learner • Impart knowledge • Assess progress, retention and results • Give feedback • And then… start over

23. Create An Enabling Environment • Students Will Learn And Function Best When… • Students have compelling direction • SET EXPECTATIONS • Give tasks and explain the context and reasoning behind the task – this will support autonomy in the future • Students have an enabling structure • Allow autonomy and allow students to know they have autonomy – this conveys trust • Allow ownership – ownership enables the hardest work • Students have a supportive context • Be available and provide support – being present without being overbearing • Recognize and reinforce  everyone needs to succeed a little each day • Students have expert coaching • We are the coaches Kackman, J. Leading Teams. 2002. Harvard Business School Publishing Co. Boston. Dutton, J. Energize Your Workplace. 2003. John Wiley & Sons, Inc. Hoboken, NJ.

24. Just a word about feedback

25. What you say “You are a great student and you are doing a really nice job. Keep up the good work!”

26. What you mean “You are exactly where I expect you to be, you are going to receive a pass for the rotation”

27. What they hear “I am on fire, I am rocking this rotation. I am going to get honors. WooHoo look at me!”

28. Again how does this happen? • It occurs when there is a disconnect between what the student perceives and what the preceptor perceives.

29. Evaluation and grading

30. Graduating Class of 2012

31. Synthesizing Final Grade • Gather individual clinical performance evaluations from all faculty/residents • Attend to both category scoring and suggested grade • Consider amount of contact and grading history • Determine approximate category scores • Weigh in other factors, e.g., final examination, write-ups, topic presentations, ethics assignment, mini-CEX • Assign grade

32. How to Write a Summary Evaluation (1 or 2 sentences each) • Knowledge/reading • Clinical skills (H&P, presentations, differential dx, management) • Interpersonal/communication skills (patients, families, staff)  • Work ethic, enthusiasm, professionalism Don’t “a pleasure to work with” “will be a great pediatrician/physician” (don’t predict the future; describe the present)

33. Required Feedback Comments: Provide descriptive feedback for student on strengths and areas needing improvement. (Not for use in the Dean’s MSPE unless there is a pattern across clerkships.) (Question 20 of 24) Student should continue to be assertive in seeking out patient care opportunities and also in volunteering his knowledge and opinions on ward rounds. Required Summary of Performance: Provide summary of overall performance based on clerkship’s objectives and the student’s achievement in all areas of evaluation. (Comments for use in the Dean’s MSPE.) (Question 21 of 24) Student did an excellent job on his pediatric clerkship. His inpatient team found him to be totally reliable and could depend on him to know his patients, do excellent H&Ps, and present patients on rounds. He demonstrated great interpersonal skills and established wonderful rapport with his patients and their families. He read about his patients' problems and constantly sought to learn. Student also impressed during his time in the outpatient clinic and was described as professional, mature and hard-working.

34. Final words • Know you goals, formulate expectations, share them with students • Assessment flows from clear communicated expecations • Keep in mind all learners are not the same • Really utilize the anchors on the evaluation forms\ • Remember to include key areas in your summative comments

35. By doing the above there will be smooth sailing ahead!!!