Ready, Set, GO! Power Prep for the ACGME Site Survey!

1. Ready, Set, GO!Power Prep for the ACGMESite Survey! Shirley Schlessinger, MD University of Mississippi Medical Center

2. Workshop Overview • Optimizing your “PIF-manship” • Day of Reckoning: The visit Itself • Behind the Scenes (ACGME / RRC) • Avoiding the MOST Common Citations • How the GME Office Can Help!

3. Optimizing Your “PIF-manship” • Why it matters • Understanding the rules • Timeline for completion • Attachments and Supporting Documents • Avoiding the most common errors *****Group Exercise: Common PIF Competency Ideas

4. The PIF is the KEY! • Majority of ultimate citations result directly from information provided in the PIF (others primarily from resident interview, resident survey, and document review) • A well-written PIF can minimize a multitude of sins! • A platform to prepare all survey day participants (best completed as a group effort for optimal buy-in…) • Ideally functions as a self-study to help ID areas of “opportunity”

5. Understand the RULES: • Review your CURRENT program requirements before beginning work on your PIF • All PIF questions track DIRECTLY to a program requirement (common or specialty) • There ARE many PIF questions with RIGHT and WRONG answers! • READ (and follow!) THE DIRECTIONS!!!!!!!! (“one” examples means ONE, “describe” means DESCRIBE) • NEVER send extra attachments or unsolicited information with the PIF • Be CLEAR, CONCISE, & CONSISTENT!

6. Timeline for PIF completion: • Many experts recommend beginning formal writing at the point of Internal Review (response to last citations, etc.) • On-going prep is best (track data, keep files organized, example folders of key requirements, etc. • ALWAYS at least 6-12 months before anticipated review date

7. Timeline Reminders: • Site surveyor must receive PIF and attachments 2 weeks before site visit • DIO must review and sign off before document can be mailed to surveyor • Optimally DIO reviews polished draft 4-6 weeks before survey, recommends changes, and signs final copy 3 weeks before survey date *** • Dept Chair, Chief Residents, Key Faculty should read and edit PIF prior to sending to DIO

8. SO… • Anticipated Date of Next Review: 4/2010 • Send to Site surveyor: 3/15/2010 • Send draft to DIO: 2/15 - 3/1/2010 • Draft for CR/key Faculty review: 2/1 – 3/1/2010 • 90-120 Day survey date notice: 12/15/2009 • Best you start writing by: 10 – 11/ 2009

9. Getting started: • Acgme.org • Web-ads sign-in and print out PIF • Part 1 is pre-populated program description; check for accuracy, make changes as needed thru web-ads • Part 2 is questions and narrative specific to your programs • Tackle the PIF in small bites / sections • JUST DO IT!!! (procrastination compromises product!)

10. Common PIF Errors: • Unanswered questions / Missing information • Failure to follow directions • Spelling / Grammar / Structure Structure errors • Including unformatted faculty CV’s, manuscripts, etc.!!!!!! • PIF faculty CV’s with publications from the 70’s & 80’s, (last 5 years ONLY!!!) • Pages incorrectly numbered • Inconsistent data • Inaccurate block diagrams • Failure to fully explain unique approached to satisfying requirements • Failure to TELL the TRUTH!!!

11. Common “inconsistencies”: • # residents in ACGME database = number of names on resident list = #of residents listed in PIF • Months / FTE’s at each participating site = number of rotations on block diagram • # of resident evaluations in folders = frequency of resident reviews reported • Institution mentioned in narrative is not referenced elsewhere • Faculty List / Faculty Credentials = faculty CV’s attached • Procedures listed for individual residents = numbers of total program procedures

12. Attachments: • May be slightly different for different RRC’s • Usually: • Policy for resident supervision • Moonlighting policy • Duty Hours Policy • Competency Assessment Tools • Evaluation tools • Overall Educational Goals for the Program • Sample Goals & Objectives for a rotation • Program Letters of Agreement

13. Supporting Documentation: • Policy for Supervision of residents • Program Policies for duty hours and work environment • Moonlighting policy • Documentation of internal review (DIO provides) • Overall Program Goals • Competency Based Goals & Objectives by rotation and level of trainee • Current Program Letters of Agreement • Files of current residents (sample transfer & problems) • File of recent program graduate (final summative eval, “competent to practice independently”)

14. Supporting Documentation (cont’d): • Evaluations of residents at end of rotation • Examples of completed 360* evaluations • Sample of written semi-annual evaluation by PD • Completed confidential evals of faculty by residents • Completed confidential evals of program by residents • Completed confidential evals of program by faculty • Minutes of annual program evaluations and written improvement plan • Resident duty hour tracking

15. Supporting Documentation (cont’d): • Complete Program Curriculum • Examples of scholarly activity (resident and faculty) • Documentation of program improvement projects • Conference schedules with documentation of attendance • Board Exam Results of graduates • Copies of alumni surveys if done • Trainee handbooks • Any other examples of “special” things you do!

16. Day of Reckoning: The Site Visit • The site visitor • Preparation • Program Director Prep • Program Administrator Prep • Faculty / Chair Prep • RESIDENT PREP • Last Minute Reminders • The Schedule

17. The Site Visitor: • Professional profile of your surveyor available on acgme.org • Confirms and clarifies the PIF (Does NOT make accreditation decision!) • Documents the data and the processes • Writes fair, objective, and accurate report • Does NOT make recommendations regarding accreditation action • Will have a specific structure they wish to follow for the day---ACCOMMODATE them!!!!

18. Survey Day Prep - General • Make sure everyone meeting with the surveyor has read the PIF (and agrees with its content!!!) • Design the day’s schedule as directed by your site surveyor • Communicate any changes or problems directly to your surveyor • Identify and schedule a room for the survey day which can accommodate all interviews • Have three copies of PIF for surveyor • Decide who will hold / answer pagers for participants!

19. Survey Day Prep – PD/PA • Know your Program Requirements inside / out, forwards & backwards! • Know your PIF/ prep your people • Have all supportive documentation, well-organized and optimally ordered by PIF topics • Have interview room neat and stocked with water • Surveyor may also wish to tour lounge and call rooms (make sure they are clean, too!)

20. Survey Day Prep - Residents • Hold peer election in larger programs to select who will meet with the surveyor (Surveyor will tell you how many; usually 10-12) • Provide PIF to residents for review 2-4 weeks prior • Meet with residents 1-2 weeks prior and review anticipated surveyor questions • Make sure they understand how previous citations have been addressed • Make certain ALL “non-compliant” issues in ACGME resident surveys have been addressed, and be sure residents feel efforts have been effective for all remedies

21. Survey Day Prep - Faculty • Select faculty per site surveyor directions • Provide PIF to faculty for review 2-4 weeks prior • Meet with faculty 1-2 weeks prior and review anticipated surveyor questions • Make sure they understand how previous citations have been addressed • Pagers must be OFF (or absent) during the meeting

22. The Typical Schedule • PD / PA meeting with review of PIF / Documents • Chair Meeting • DIO Meeting • Key Faculty Meeting • Resident Meeting • Wrap Up meeting with PD

23. Behind the Scenes: ACGME / RRC • Site Surveyor completes a written report- just the facts, designed to verify PIF information, summarize document review, and clarify issues raised in the ACGME resident survey • Surveyor report is forwarded to designated RRC to be added to next open agenda (agendas usually finalized 2 months in advance of actual meeting) • RRC committee members (1-3) are assigned PIF and surveyor report for review & to present at meeting (actual surveyors do not participate) • Committee discusses and determines status and cycle length • E-mail notice received within 2 weeks of meeting re status decision and cycle length • Full Letter of Report (LOR) with citations received 8-12 weeks after RRC meeting

24. The Common Problem Areas • Nationally Most Common Citations • UMC Most Common Citations • The Nine Red Flags

25. Most Common Citations: • Duty Hour Violations • Service versus Education Issues • Evaluation problems • Board Pass Rate Citations • Written Curriculum Inadequacies

26. UMC Most Common Citations: • Qualifications and Number of Faculty (14) • Procedural Experience (13) • Patient Care Experience (12) • Institutional Support (10) • Evaluation of Program (9) • Scholarly Activities (9) • Responsibilities of Program Director (9) • Performance on Board Exams (8)

27. Nine “Red Flags” in Accreditation Surveys • From ACGMe-Bulletin February 2008 • Barbara Bush, William Robertson, Ingrid Philibert authors • Key issues to AVOID in program & site surveys • These problems most likely to result in adverse accreditation citations

28. Red Flag #1: Lack of Program Leadership • PD and Faculty fail to advocate for residents on important education and patient care issues • Lack of response to issues raised by residents/fellows • Too much reliance on communication and preparation PIF by program coordinator or other staff members • Repeat citations on successive reviews

29. Red Flag #2: Lack of Program Infrastructure for Teaching and Evaluation • Insufficient clinical or didactic curriculum • Insufficient systems for evaluation of residents, faculty or program

30. Red Flag #3: Lack of Appropriate Volume and Variety of Patients • Insufficient volume or balance of patients (diagnoses, clinical problems, acuity and demographics) • Disputes with other disciplines affecting numbers of patients available to the teaching program • Too many residents, fellows, other learners competing for same patient populations

31. Red Flag #4: Problems with Resident Recruitment or Retention • High Resident Turn-over • Unfilled resident positions • Poor record for graduates sitting for and passing board exam (reflects poorer quality applicants / trainees?) • May be due to geography, program or institutional reputation, interest in specialty, etc.

32. Red Flag #5: Lack of Dedicated Teachers • Faculty unwillingness or inability to devote added time required for effective teaching (at the bedside and in operating room, during conferences, rounds, and other didactics) • Problem may present with low numbers of board-certified faculty or not enough key faculty • Too much or too little supervision • Failure to provide meaningful feedback and evaluation • Fellows doing all the teaching

33. Red Flag #6: Lack of Meaningful Didactics • Didactics don’t cover the essential body of knowledge required by RRC (basic science and clinical) • Frequent cancellation of conferences • Lack of sufficient faculty attendance or participation in conferences • Over-reliance on residents or fellows to organize and present at conferences

34. Red Flag #7: Lack of Financial and Human Resources • Inadequate or outdated facilities • Excessive clinical demands on faculty / PD • Excessive clinical demands on faculty including PD • Excessive “services needs” / Residents need to “cover” too many hospitals • Inadequate number of administrative and ancillary staff for size of program • Lack of funding for program

35. Red Flag #8: Service has a Higher Priority than education • Undue reliance on residents to provide service including clinical services that cannot run without the presence of residents • Residents being “pulled” to “cover” services regularly • Duty hour violations affecting a significant percentage of the residents • Residents being required to provide coverage or cross-coverage on inpatient units during their ambulatory, subspecialty, or research rotations.

36. Red Flag #9: Lack of Preparation for the Accreditation Process • On site survey day, a program leader or faculty member who does not understand, argues about the standards, or lacks “buy-in” for the requirements • Poorly prepared PIF • Obvious errors, inconsistencies or failure to follow instructions • Missing documents • PIF that arrives late to site visitor

37. Not Germane to ALL: (BUT Frequent Area of Citation in many specialty areas) Too MUCH or too LITTLE Scholarly Activity on the part of Faculty or Residents

38. How the GME Office Can Help! • Many required topics are covered in orientation annually • We provide notification of campus wide activities that offer opportunities to meet program requirements • Evaluation Tools ; Faculty Development Opportunities • Problem Resident / Faculty Intervention • Anonymous Resident Complaint Line • internal reviews/ Annual Survey/ Duty Hours Survey designed to help you maintain compliance and identify problems early • Resident Focus Groups convened as needs identified • PIF REVIEW!!!! (in draft form!) • Mock Surveys on request or consultant visits

39. Keys to Success: • Start early • Review and know Program and Institutional Requirements • Correct ALL previous citations • Have on-going program improvement processes • Good communication and preparation with residents and faculty • Convey the strengths and unique attributes of your program clearly to the surveyor!