Bridge over Troubled Water Linking UME and GME

1. Bridge over Troubled WaterLinking UME and GME Monica L. Lypson, MD Assistant Dean, Graduate Medical EducationUniversity of Michigan Jeff Fabri, MD Rita M. Patel, MD

2. Education Across the Continuum Standardization of AssessmentStudent Portfolios to Baseline Assessment Monica L. Lypson, MD Assistant Dean, Graduate Medical EducationUniversity of Michigan Associate Chief of Staff, VA Ann Arbor Health Care System

3. Developed by the Graduate Medical Education Committee (GMEC) at the University of Michigan It is our initial step in training our residents It was established to determine residents’ baseline proficiency in particular aspects of the ACGME’s six general competencies Institutional OSCE – Post-Graduate Orientation Assessment (POA) Lypson ML. et.al. Academic Medicine. 79(6):564-70, 2004 Jun.

4. Knowledge and skills needed during the first six to eighteen weeks of residency/internship Emphasizes clinical situations that are often encountered without formal supervision 9 Assessment & Educational Stations POA CONTENT

5. Results of the POA determine the basis for individualized “learning agendas” Remediation is provided after the completion of each station Residents received educational materials that provide the “answers” to the information assessed during the POA Standardized Patient feedback is provided to the program director within 24 hours if the resident performs exceptionally well or poorly during the POA The POA as Formative Assessment

6. Administered over 4 days of Paid hospital orientation Provided to approximately 150 PGY-1 residents in over 15 specialties Cost Approximately $250 per resident for the assessment - $500 if you add in salary There are some discipline specific scenarios – e.g. Pediatric cases and examples Scores are provided to the resident and program director within 7-10 days of the POA Implementation

7. Dentistry Emergency Medicine Family Medicine General Surgery Preliminary Residents Internal Medicine Preliminary Residents Internal Medicine – Pediatrics Neurosurgery Obstetrics & Gynecology Otolaryngology Orthopedics Pathology Pediatrics Plastic Surgery Psychiatry Urology PGY-2s (Residents that did not complete their Internships at UMHS) Physical Medicine & Rehabilitation Dermatology Neurology Subspecialty Programs

8. Critical Values / Multiple Choice Computer Based assessment Review and diagnose 17 short patient scenarios Multiple Choice Format similar to United States medical Licensing Exam (USMLE) Step 3 Competencies: Patient Care Medical Knowledge Socio-Cultural Communication Standardized patient assessment Assesses the understanding of disease and treatment recommendations in the context of the patient’s health beliefs and socio-economic setting Competencies: Interpersonal and Communication Skills Professionalism Patient Care Practice-Based Learning and Improvement Station Content

9. Evidence-Based Medicine (EBM) Computer-based Assessment Generate a clinical question Residents reviewed abstracts & identify the appropriate treatment Competencies: Practice-based Learning and Improvement Medical Knowledge Images (X-Rays) Computer-based Assessment Review & diagnose 18 images Many of the common films reviewed in the middle of the night while “on call” Competencies: Patient Care Medical Knowledge Station Content

10. Informed Consent Standardized patient Assessment Obtain informed consent from a patient for a procedure Hospital & JCAHO standards and policies Competencies: Interpersonal and Communication Skills Professionalism Patient Care Patient Safety Pen & Paper Assessment Order Writing Station Legibility Signature, Date, Time Respiratory Distress Treatment of the acute Asthmatic Competencies: Systems-Based Practice Professionalism Patient Care Medical Knowledge Station Content

11. Aseptic Technique Checklist Evaluation by Expert Nursing Staff and Standardized Patients Create & maintain a sterile field while performing a “mock” I&D Universal Protocol & “Time-out” procedures covered Assessment/Remediation materials include a review of JCAHO requirements Improved nursing and house officer interactions Competencies: Patient Care Medical Knowledge Systems-Based Practice Station Content

12. System Compliance / Fire Safety Station Surgical Fire Safety (20/20) Housestaff involvement with a Patient fire Questions covering use of safety equipment & review of JCAHO requirements Video & Computer Based Assessment Competencies: Patient Care Systems-Based Practice Pain Assessment Pain Assessment Tools Educates on the appropriate medications to use for pain Explains hospital and JCAHO Pain assessment mandates PowerPoint & Computer Based Assessment Competencies: Patient Care Medical Knowledge Station Content

13. Resident Satisfaction with the POA-Survey Response Rate 93%2002-2004

14. The POA was based on the following… The University of Michigan Comprehensive Clinical Assessment (CCA)

15. A high-stakes examination for University of Michigan Medical Students Fourth Year students must pass the CCA in order to Graduate 10-15 stations over ~4-5 hours ~150 M4 students per year Summative Evaluation Standards have been set for Pass/Fail Remediation for failures Comprehensive Clinical Assessment 1991-Present Rochester AB. et.al.. Academic Radiology. 5(3):169-72, 1998

16. POA- GME Formative Informed Consent & Policy Pain Assessment Socio-Cultural Communication Evidence Based Medicine Images CCA-UGME Summative History & Physical Diagnosis Geriatrics Socio-Cultural Communication Evidence Based Medicine Images Summary of Station Comparison:CCA & POA

17. Training the Faculty in the Teaching Skills of competency assessment Medical School Objectives Project (MSOP) Accreditation Council of Graduate Medical Education Reimbursement of faculty time and effort Payment for both assessments – Medical School vs. Hospital Is the medical school or the hospital responsible for ‘educational cost’? Trainee awareness of “competency gap” between the expectations of medical School and residency Institutional Implications for GME & UGME

19. POA Demographics 2002-2004

20. Joint Hire faculty member with the Department of Medical Education Centralized OSCE – Post-Graduate Orientation Assessment (POA) This is based on the system already in place for Undergraduate Medical Education at UMHS. The University of Michigan Health System Plan for Competency Based Resident Education

21. One of ~5 departments of medical education at LCME accredited medical schools The department had focused on CME and UGME in the past with assessment and research expertise The office of Graduate Medical Education and the Department of Medical Education Joint Hire Job Description 50% appointment GME office, 50% appointment Dept. of Med. Ed. Aid Residency programs with curriculum development, core competency assessment Use this work as a vehicle of scholarly pursuit The department has a long history of UGME – now will develop its expertise in GME The Department of Medical EducationChairman: Larry Gruppen, PhD