Clinical Grading: Assessing Student Performance

1. Lecture slides available on RNnext.com Clinical Grading: Assessing Student Performance Raney Linck, DNP, RN

2. Objectives • Discuss how to implement the three keys to fairness in clinical evaluation. • Describe five specific types of grading errors. • Analyze a clinical evaluation tool for clarity and consistency. • Summarize methods to develop clinical judgment skills in student learners.

3. Clinical Grading: CHALLENGES • Every site, every student is different • Maintaining consistency among faculty • Some faculty may be new to teaching • Lack of time to meet as a team • Observation: element of randomness • Time!! What else?

4. What happens BEFORE the grading?

5. Fairness in Clinical Evaluation • Faculty identifies own values, attitudes, beliefs, and biases that may influence the evaluation process • Clinical evaluation is based on predetermined outcomes or competencies • Faculty develops a supportive clinical learning environment (Oermann & Gaberson, 2017)

6. 1. Identify One’s Own Values • Judgment and interpretation, by definition, have a subjective dimension. • Be aware of own values, attitudes, beliefs, and biases. • Students also bring their own to the experience… • Communicate your priorities.

7. 2. Base on Predetermined Outcomes • Written framework with clear descriptions. • Do students and all faculty (especially new) have consistent understanding of terms and goals? • Is evaluation form truly only reviewed at the end? • Communicate your priorities

8. PEAK PERFORMANCE CURVE (for complex tasks) based on Yerkes-Dodson Law (Johnston et al., 2012) Image: https://www.slideshare.net/WendyKettle/peak-performance-workshop-final-1

9. 3. Develop Supportive Environment • Faculty is responsible for creating a climate where students feel safe, supported and facilitated in learning. • Student comfort to ask questions, seek guidance • Managing the clinical setting, staff, assignments • How do you cultivate relationship with site & staff? • Remember: clinical is a public experience

10. How Do You Create a Supportive Climate? • Explicitly give permission to interrupt me • Get real-time feedback: how can I improve your clinical experience? • Develop my rapport with staff on unit – remember you set the tone • WOW! Cards • “We learn more from the mistakes we correct than from our easy successes.”

11. GRADING: Specific Challenges • Halo effect • Logical error • Rater drift • Reliability decay • Rating errors: • leniency • severity • central tendency (Brookhart & Niko, 2015)

12. Specific Challenges • Halo effect: making judgments primarily based on a general impression of a student • Personality is not performance. • Logical error: giving similar ratings on 2 of more criteria that are not necessarily related (Brookhart & Niko, 2015)

13. What is your STRATEGY? Specifically: How will you remember and organize your observations of multiple students?

14. IDEA: Make anecdotal notes each clinical day • Faculty: make notes each clinical day • Helps you remember specifics on: • successes & challenges • feedback you gave & if it lead to change • TABLE of patients & experiences each student had • Timeliness! Record as soon as possible… • Share in a formative way with student & be willing to incorporate student’s perspective/data (Griffin & Novotony, 2012; Oermann & Gaberson, 2017)

15. IDEA: Use a checklist for an essential skill I provide a checklist for a 5-min. head-to-toe assessment beforehand, and inform students they will be assessed on the first clinical day.

16. Specific challenges Consistency is strong… at first after CREATING a new eval tool. at first after TRAINING on an eval tool. • Rater drift: Over time, meaning is reinterpreted and changed from original intent. • Reliability decay: Over time, faculty members become less consistent.

17. IDEA: Commit to meet / connect as faculty team • Beginning • Middle • End

18. Two faculty observeIV med administration: 1. Focus isprimarily observing technique • Limited questions about the actual drug • Not particularly interested in interaction with or education of patient. 2. Focus isstudent’s drug knowledge and educating patient • “Coaches” student through each step of technique

19. FINDINGTHEBALANCE clear and specific over-whelming detail

20. Example from mental health clinical: SAFETY • Maintains professional boundaries, identifies issues in complicated boundary maintenance with some types of clients • Is alert to behavioral changes and suicidal thinking & monitors milieu for safety • Demonstrates knowledge of medications (mechanism of action, dosages, side effects, etc.) • Follows code of ethics • Maintains confidentiality • Demonstrates appropriate communication for specific disorders • Maintains communication with faculty, hospital & agency staff regarding clients’ safety

21. Specific Challenges • RATING ERRORS • Leniency: tending to rate all students toward HIGH end of scale • Severity: tending to rate all students toward LOW end of scale • Central tendency: hesitant to give high or low grades, so end up only using midpoint of the scale

22. Make rating scale clear & meaningful What do rating numbers / terms mean? Rating example #1: Simple, direct required grade to pass

23. Rating example #2: Detailed, explanatory Definitions and guidance laid out as table (see next slide) required grade to pass Key: define independence in a student role context

25. Your eval tool is organized intomeaningful categories or themes, such as… • Your program’s competencies • QSEN competencies (6) Patient Centered Care, EBP, Teamwork, Safety, QI, Informatics (KSA = Knowledge, Skills, and Attitudes) • CCNE Essentials (9) CONSIDER: Are you using that language in your postclinical discussions, your daily verbal feedback, your assignment grading?

26. One example pulling it all together… In this example, an evaluation is recorded each clinical day. from NurseThink for Nurse Educators: Success Manual (Bristol & Sherrill, 2019)

27. (Bristol & Sherrill, 2019)

28. (Bristol & Sherrill, 2019)

29. IDEA: Midway self-eval as postclinical discussion • Before clinical, have them complete the clinical eval by self-rating themselves with a number beside each individual item. • In postclinical, discuss self-evals: what’s going well and what do they want to improve? • Have each identify 3 items to focus on (turn in with name on it)

30. Examine your own eval tool… • Are the outcome items clear and specific? • Is there too much detail? Or not enough? • Is the rating scale clear & meaningful? • Are you regularly talking about these concepts during clinical & in your feedback?

31. NOTES:

32. Energizing learning in class… • Tim Bristol: “We need to make the classroom more like clinical.” • Karin Sherrill: During class, will put on a hospital gown over clothes, so students know she has become their patient.

33. Energizing learning in clinical… • Ask: Expected or unexpected finding? • Post-clinical role play: Change in patient’s condition • Student charge Create experiences to give you the assessment data you need to truly measure student learning.

34. 1. Expected or unexpected? • Patient has a slightly elevated creatinine level. • 21-year-old with no PMH who remains hypotensive after severe car accident. • 68-year-old with end stage renal disease • Patient has a pain 3/10 at surgical site. • Patient is postop day #1 • Patient is postop day #5 and had been pain-free for three days.

35. The client has a pneumothorax. Nurse records 650 mL output (bloody) into chest tube drainage system in the first 4 hours after insertion. Expected or unexpected?

36. Client is preparing to discharge after a 5-day admission for exacerbation of end stage COPD. Current O2 sat is 90% on 2L NC. Most recent ABGs show a CO2 level of 48 (slightly elevated). Expected or unexpected?

37. 2. Post-Clinical Role Play: Change in Condition • Have one student give report on their patient to the group. • Inform group you may throw out questions to anyone so they’re fully engaged. • Choose add’l student to do role play (“enter the room”) with one who gave report. • They can ask for any specific data they choose… • Assign one student to write down data they collect on a white board. • Helps everyone to engage and track

38. Act as the patient havinga change of condition… • Non-responsive • Respiratory distress • Severe pain • Significant change in BP • Confusion Variation: Role play patient needing education or clarification on: meds, treatment, or condition.

39. Explicitly reassure them: We’re role playing a tough situation in a safe environment. Involve the group in helping. Brainstorming ideas Trade out the pair “on the front line” Sometimes I will guide them in pattern matching. What could cause this condition? What would you assess to confirm or rule out?

40. Simpler variation on role play… Instead of whole role play, ask a question like: • Your client’s BP is now 90/54… • Your client’s HR just dropped 20 BPM… • Your client’s abdomen is distended & firm… • Your client is on IV heparin therapy and you notice blood on the bedsheets… • Then ask: “Nurse, what will you do now?” from NurseThink for Nurse Educators: Lab Coat Notes for Teaching Clinical (Sherrill, 2018)

41. 3. Student Charge • Create an opportunity for leadership • Help and observe other students • Spend time with charge nurse • Lead post-clinical discussion • Encourage creativity

42. Best post-clinical of my career…

43. References Bristol, T., & Sherrill, K.J. (2019). NurseThink for Nurse Educators Success Manual. Waconia, MN: NurseTim. Brookhart, S. M., & Nitko, A. J. (2018). Educational assessment of students (8th ed.). Upper Saddle River, NJ: Pearson. Johnston, C. A., Moreno, J. P., Regas, K., Tyler, C., & Foreyt, J. P. (2012). The application of the Yerkes-Dodson Law in a childhood weight management program: Examining weight dissatisfaction. Journal of Pediatric Psychology, 37(6), 674–679. doi:10.1093/jpepsy/jss040 Griffin, M.T., & Novotny, J.M. (2010). A nuts-and-bolts approach to teaching nursing (4th ed.). New York: Springer. Oermann, M.H., & Gaberson, K.B. (2017). Evaluation and testing in nursing education (5th ed.). New York: Springer. Sherrill, K.J. (2018). NurseThink for nurse educators: Lab coat notes for teaching clinical. Waconia, MN: NurseTim.

44. Optional resource If you need to create or revise a MENTAL HEALTH clinical rubric, you may find this resource useful in writing your outcomes… • Bondy, K.N., Jenkins, K., Seymour, L., Lancaster, R., & Ishee, J. (1997). The development and testing of a competency-focused psychiatric nursing clinical evaluation instrument. Archives of Psychiatric Nursing, 11(2), 66-73.