1 / 39

Residents In Trouble

Residents In Trouble. M. Rebecca Hoffman, MD, MSPH December 8, 2008 Academy for Scholarship in Education. Overview. Scope of the Problem Literature Review Study Design Study Results (in progress). Scope of the Problem. The goal of Family Medicine Residency Residents’ backgrounds

Download Presentation

Residents In Trouble

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Residents In Trouble M. Rebecca Hoffman, MD, MSPH December 8, 2008 Academy for Scholarship in Education

  2. Overview • Scope of the Problem • Literature Review • Study Design • Study Results (in progress)

  3. Scope of the Problem • The goal of Family Medicine Residency • Residents’ backgrounds • No guidance

  4. What does the RRC say? • Must have advising system • Must give regular feedback BUT… no real guidance regarding what/how

  5. Consequences of RIT • For the individual RIT • Psychological stress • Financial stress • Eventual job security • For the program • Adequate coverage of duties • Morale • Faculty time/energy • Dismissal

  6. Consequences… • Other consequences • Recruiting • Impact on patient care

  7. Literature Review • Not much on the prevalence of RIT • Reamy et al (1): 25 year in-depth study of one family medicine residency program • 9.1% • Yao and Wright (2): Survey of Internal Medicine PDs– • 6.9% • Williams et al (3): SIU Gen Surgery Residency • 22%

  8. Kinds of Problems • Knowledge Deficits • Attitudinal Problems • Interpersonal Conflict • Psychiatric Illness • Substance Abuse • Family Stress • Relationship Disruption

  9. Other Classification • Academic Performance • Classroom, test • Clinical Performance • Applied knowledge/skills • Professional Behavior • Workplace behavior, relationships with other professionals, etc • Others • Difficult to classify

  10. So we can name it… • But what do we do with the information? • Literature reviews: • “Need to do better” • Core content review, quizzes • Counseling • Treatment of illnesses • Repetition (rotation, year) • Probation • Dismissal • Others

  11. Summary of the problem • We know it’s there, but we aren’t sure exactly how common it is • We aren’t good at predicting who will struggle • Once we identify them, we don’t know what to do with them

  12. RIT Family Medicine Study Primary Aims 1) Determine the prevalence of RIT in the SIU Family Medicine Residency Programs 2) Describe and categorize the types of troubles encountered in the FCM residency programs

  13. RIT Aims... 3) Identify and describe the kinds of processes the programs employ to address deficiencies with RIT 4) Describe outcomes of the residents who underwent remediation 5) Identify early indicators to predict RIT

  14. RIT Study Design • Chart review/resident record -based • Prevalence over significant timeframe • 10 years of entering classes • Entering class of 1993-entering of 2003 • Why? • De-identified data from all 4 programs • To further assure confidentiality • No one program singled out

  15. RIT Study • Initially modeled after Reamy et al study (1) after discussion with Dr. Reamy about the process in his residency program • Assistance from Reed Williams, PhD and Nicole Roberts, PhD

  16. Methods • Rosters of all entering residents for the 10 years from all programs • Random ID numbers assigned • Records from each program reviewed in their entirety • Mostly paper • Some electronic

  17. Methods… • Data collection instrument created based on personal experience, literature, and discussion with Reed Williams and Nicole Roberts • Feedback from academic faculty • Suggestions incorporated as needed

  18. Collected for all residents: • US Medical School Grad (Y/N) • USMLE or NBOME or COMLEX scores • Step 1, Step 2, Step 3 (when available) • Gender (M/F) • ITE Scores • PGY1, PGY2, PGY3 (when available) • Did the resident have substantial difficulties? (Y/N)

  19. For those who struggled • Time frame- when first identified • Categorization of the main problem • Academic, Clinical, Professionalism, Other • Main performance area involved • See list • Breadth of performance areas • Academic, Clinical, Professional, Other • All performance areas involved • See list

  20. For those who struggled • What remediation attempted? • See list • What was the final resolution? • Graduated with concerns • Graduated without concerns • Voluntarily left • Dismissed • Finished on probation • Not documented

  21. Severity Rating • Compared to other residents with performance problems, this resident’s problems were: • Likert scale, 1-5 • 1 among the least serious • 5 among the most serious • Narrative descriptions of overall case, time course, etc

  22. Example of Survey Instrument • Survey Monkey for data entry

  23. Data Collection • 2 raters • Independently reviewed same 5 charts at first program to establish inter-rater reliability, then 3 charts at each other program

  24. Data Collection… • Then independent record review • Any case the reviewer identified as a RIT was then reviewed by the other reviewer independently, ratings discussed after each case • “Borderline cases” also reviewed by both

  25. Data Analysis • Data entered into Survey Monkey for ease of collection • Analysis: descriptive and basic statistics • When all data complete, analysis using SPSS

  26. Results • 96 Residents reviewed (as of 12/4/2008) • US Med School Grads: 69% (66) • Male: 65% (62) • Female: 35% (34) • Prevalence of documented RIT: 38.5% (37)

  27. RIT (N=37) • US graduates: 59.5% (22) • Foreign graduates: 40.5% (15) • Male: 70.3% (26) • Female: 29.7% (11) • Male, US: 40.5% (15) • Male, Non-US: 29.7% (11) • Female, US: 18.9% (7) • Female, Non-US: 10.8% (4)

  28. RIT

  29. Most Important Single Problem Area? • Knowledge (10) • Putting everything together (3) • Data interpretation/diagnosis (2) • Data collection (2) • Incomplete paperwork/charts (2) • Lack of motivation/interest (2) • Treatment/management (2)

  30. Breadth of Problems

  31. Breadth of problems • Number of performance areas involved ranged from 1 to 23 • Average: 7 • See breakdown of individual RIT on handout

  32. Remediation • Ranged from “none” or “told to improve” to 12 or more interventions for a single resident • Most commonly reported: Increased meetings with advisor/mentor

  33. Outcomes of RIT (residency) • 26 graduated from the program (70.3%) • 11 did not graduate (29.7%) • Left voluntarily: 6 • Changed specialty: 2 • Dismissed: 3 • More complete follow up not complete • Have initiated licensure status and board certification F/U but too little data to report

  34. Outcomes

  35. Conclusions? • RIT more common than suspected based on lit. • Increased prevalence compared to other studies • ? Over-diagnosis • For “mild cases” • ? Truly higher prevalence • Due to population? • Due to program characteristics? • All RIT who had concerns in more than 8 areas had ongoing concerns at graduation or did not complete program • RIT who struggled in all 3 performance areas also either did not complete or graduated with ongoing concerns • Professionalism issues very common

  36. Problems Encountered • Information in files varies between programs • Missing data • Esp. test scores (fortunately, may be able to acquire) • Even within programs, varying data • Classification of single most important area for major problem residents • We collected data for an additional group of residents: those with ITE performance less than 10th %ile, even if no other problems, will follow those as well (n=4 right now)

  37. To come! • Full data reporting (approx 270) • Including USMLE and ITE data • Predictive model? • Follow up data (post-graduation)

  38. Special Thanks to: • Nicole Roberts, PhD data spelunker extraordinaire • Reed Williams, PhD • Steve Verhulst, PhD • Jerry Kruse, MD, MSPH • SIU Family Medicine Residency Program Directors: • Janet Albers, MD • John Bradley, MD • Tom Miller, MD • Penny Tippy, MD • Wiley Jenkins, PhD

  39. Referenced Articles • 1. Reamy BV, Harman JH. Residents in trouble: an in-depth assessment of the 25-year experience of a single family medicine residency. Fam Med 2006;38(4):252-7. • 2. Yao DC, Wright SM. National survey of internal medicine residency program directors regarding problem residents. JAMA 2000;284:1099-1104. • 3. Williams RG et al. The nature of general surgery resident performance problems (publication pending)

More Related