1 / 31

Developing High Quality Clinical Skills Assessments

Developing High Quality Clinical Skills Assessments. University of North Carolina – Chapel Hill School of Medicine November 10, 2011 Ann Jobe, MD, MSN Clinical Skills Evaluation Collaboration Philadelphia, Pennsylvania.

rannunziata
Download Presentation

Developing High Quality Clinical Skills Assessments

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Developing High Quality Clinical Skills Assessments University of North Carolina – Chapel Hill School of Medicine November 10, 2011 Ann Jobe, MD, MSN Clinical Skills Evaluation Collaboration Philadelphia, Pennsylvania

  2. The most important consideration in developing a clinical skills assessment is to keep in focus the purpose of the assessment

  3. Formative or Summative? Formative: to provide feedback for improvement of performance, usually midway through a course of study. Summative: to distinguish between those who are competent, and those who aren’t, usually at the end of a course of study.

  4. Test Design • Begin at the beginning… • What are you trying to assess? • What is the level of the examinee? • How much time will the examinee have? Can they complete the task in the time allotted?

  5. Test Design • What are you trying to teach and assess? • Objectives for assessment – history taking skills, PE skills, communication skills, data interpretation skills? • What is the level of the examinee? • Focus and length of checklists will be different for 1st year medical students and individuals seeking board certification

  6. USMLE Step 2 Clinical Skills Purpose is to assess ability to: • Gather information from patients • Correctly perform physical examination maneuvers • Synthesize and communicate findings to patients and colleagues Examinees are moving to supervised patient care (PGY-1)

  7. Development Of An Examination Blueprint • A test blueprint defines the requirements for each examination, regardless of where or when it is administered. • Commonly seen cases.

  8. Blueprint (continued) • Identify the criteria used to define an exam blueprint • Examination length • Case content • Examinee tasks (e.g. history, physical, communication) • Setting: inpatient or outpatient • Patient gender, age

  9. Case Selection • Documentation of commonly seen cases • if specialty focused assessment, cases will be those common to that specialty • Most common presenting complaints in clinical setting • Health Department population statistics – especially for region, state or local area

  10. Case Pool Size • Dependent on blueprint criteria • Security concerns (more is better!)

  11. Case Content Cardiovascular Respiratory Gastrointestinal Musculoskeletal Constitutional Neurological Psychiatric Genitourinary Women’s health Other Case Acuity Acute Subacute/Chronic Form Patient age Age less than 18 Age 18 – 44 Age 45 – 64 Age 65 + Patient Gender Male Female

  12. Blueprint Criteria Define Case Needs • This information is provided to faculty to guide their case development • Case content: Gastrointestinal • Case acuity: Acute • Patient age: 18-44 • Patient gender: Female • Complete a “Medically Relevant Case Details” sheet

  13. Case Development Committee • Physicians (medical school faculty): content experts • SP trainers: training experts • SPs: portrayal experts • Process evolved from experience

  14. Usually break into several groups Each group focuses on developing several cases to fit blueprint needs

  15. Case Development Process • Chief Complaint • “Medically Relevant” case details • Patient Personality Profile • Case checklist • Examinee instructions • Patient note development

  16. Chief Complaint/ Differential Diagnoses • Brainstorming process • List all of the possibilities • Then narrow the choices as development proceeds

  17. Medically Relevant Case Details • Patient /Case Name • Race • Any • African American • Caucasian • Other _________________ • BMI • Any • Weight proportionate to height • Overweight: > 24 • Other

  18. Medically Relevant Case Details • Gender • Male Female Either • Age Range • 10 year age range – example 30 years old = 25-35 • Acuity • Acute • Subacute/Chronic • Category (Blueprint) • Exclusionary Medical conditions/scars • Differential Diagnosis

  19. Medically Relevant Case Details • Communication Tasks • Fostering the Relationship • Gathering Information • Providing Information • Making Decisions • Supporting Emotions • Overview of Case Stimuli for Communication Tasks • Thread of Encounter • Primary MD

  20. Personality Profile Giving life and individuality to the patients who are being seen – more like “real patients” Not verbatim statements from a script – more general ideas of each patient’s perspective

  21. Personality Profile Who am I and how would I describe myself? Why am I here? What made me come in today? What do I think is going on and why? What are my expectations for the visit and how will I react if my expectations are not met?

  22. Personality Profile What questions do I have? What concerns/anxiety/fears do I have? How has this illness impacted my life? What do I bring with me? What do I look like? How do I behave/what’s my communication style (mood/attitude related to my illness)?

  23. Personality Profile What is my level of trust of the medical profession? What type of communication style do I appreciate in a health care provider? How will I respond to different styles of communication? How much information do I want the doctor to provide?

  24. Personality Profile What words won’t I understand (jargon)? What are my own words I use to describe what is going on? How motivated am I to accept medical advice? How do I feel about the use of complementary/alternative medicine?

  25. Case Checklist • Recording instrument • Checklist length • One concept per item • Evidence-based items • Use lay language Example: Onset (“I’ve been coughing for about a month.”)

  26. Refining the Checklist • Brainstorm a large number of items • Role play • Checklist revised • A – essential • B – important • C – relevant • F - delete

  27. Examinee Instructions • Name • Age • Setting • Presentation problem • Vital signs • Examinee Tasks • Refers to the skills that the examination assesses

  28. Role Play with SP • A physician from another group “sees” the patient • Following the encounter, the physician provides his/her differential diagnoses (see if it matches what the group selected): face validity • Revise the checklist based on observing the physician

  29. Role Play with SP • This allows an SP to provide feedback on any difficulties that arise in portrayal of the case – especially if an SP needs to repeat the case several times in an assessment session

  30. Development of Patient Note • Key essentials and case-based scoring guidelines

  31. Questions?

More Related