The Residency Review Committee

1. The Residency Review Committee Jerry Vasilias, PhD, Executive Director Caroline Fischer, Associate Executive Director

2. Setting the Stage Overview to the RRC Review of RRC work in 2006 Frequent Citations New ACGME Changes Preparing for the site visit ACGME�s Strategic Priorities Website

3. RRC Composition 3 appointing organizations - AAP, ABP, AMA 13 voting members 4 from each appointing org + 1 resident member Members serve a term of 6 years -- except resident Generalists, subspecialists, Chairs and DIOs Critical Care Medicine, Hematology/Oncology, Pediatric Emergency Medicine, Gastroenterology, Neurology, Nephrology, Rheumatology Geographic distribution � CA, CT, DC, GA, MI, MD, OH, OR, PA Ex-officio members from each appointing organization (non-voting)

4. RRC Review of Programs Peer Review � 2 reviewers Reviewers use the following information when they review a program to determine whether it is in compliance with the requirements: Program Information Forms (PIF) prepared by the program The site visitor�s report Board scores Program Directors: this is an open book test Reviewers present program to Committee Independent reviews followed by �reconciliation� Committee assigns accreditation status along with review cycle, range of 1-5 years

5. Summary of Activities in 2006 The RRC meets twice a year � Spring and Fall Workload per meeting in 2006: approximately 30 core programs 120 subspecialty programs 15 progress reports

6. Accreditation Decisions in 2006 -- Core Pediatrics

7. Most Frequent Citations in 2006 �Core Pediatrics

8. New ACGME Changes Requirements have been updated with new Common Requirements Appear in bold font in document Specialty specific content has not changed However, during update some language in competencies was inadvertently removed; the document on the website/distributed today is correct. E-mail status of programs on RRC agenda 2-3 days after meeting will receive email w/ status and review cycle. Proposed adverse actions will receive notice as well E-mail that letter will be posted on Accreditation Data System (ADS) Letter is posted one business day after the email notification Printed copies of letters will not be provided E-mail notification of site visit date

9. New ACGME Changes More information will be collected/communicated through ADS. Common PIF = Common questions all specialties required to complete in ADS. Along with information already collected in ADS (institutions used for training and resident info) will collect: Information on faculty/teaching staff Evaluation (resident, faculty and program) Duty hours Board scores ADS to be used for changes in complement, PD, institution +/-, VW Responses to previous citations will be done in ADS. 1st part of PIF. Progress reports not yet in ADS.

10. New ACGME Changes Common competency questions inserted in all specialties� PIFs (common but not hard-wired into ADS). Typical question asks PD to identify learning activity used to achieve a particular outcome/competency Describe one learning activity in which residents engage to identify strengths, deficiencies and limits in their knowledge and expertise� Describe� develop competence in communicating effectively with patients and families across a broad range of SES and cultural backgrounds Questions like these in PIF since late fall of 05 PD Guide to the Common Requirements PD Virtual Handbook Webpage redesign

11. Update on Case Logs All core Pediatrics programs have been required to have their residents enter data in the ACGME�s case log system RRC has stated that it will not use for accreditation purposes until after its collected 3 years of data Can use this system to track continuity � not required Committee will review 3 years worth of data at its upcoming meeting

12. Preparing for the Site Visit: The Five Stages that Precede the Site Visit

13. Preparing for the Site Visit: The Five Stages that Precede the Site Visit 1st Stage: Denial ��No, way! They�re not coming already?� 2nd stage: Bargaining � �We can get a postponement!� 3rd stage: Anger --�We really have to go ahead with this!� 4th Stage: Depression --�I have sooooo much else to do? How will this get done?� 5th Stage: Acceptance -- �We�ll be ready.� With apologies to Elisabeth Kubler Ross

14. ACGME Expectations for Site Visits Conducted in accord with established policies Provide accurate, meaningful data for the RRC Note the PIF is the focus of the site visit (not the program) Verify/clarify info in the PIF Address all relevant aspects of the program Are educational and non-adversarial

15. ~ 6 to 18 months before the visit Residents complete ACGME Resident Survey (all programs with 4 or more, surveyed every two years) ACGME expects programs to reach a 70% threshold Summary of results become available to PD and DIO only if reach 70% response rate Intent of the survey (1) focus and direct the resident interview, and (2) identify potential areas of non-compliance with duty hour standards for RC follow-up Site Visit to Site Visit � An Ideal Time Line

16. On-going Review and know the program requirements ACGME web site, RRC specific updates and notices Contact RRC executive director or staff for clarification Attend meetings/workshops 1/2 way between last and next review Internal review occurs Involves a broad, representative group Candidly explores strengths and weaknesses Identifies improvements for implementation Site Visit to Site Visit: An Ideal Time Line

17. The next survey date on your last notification letter is an �approximate� Site visit will take place between 2 � 4 months around the approximate date 3 - 4 months before the actual visit You will receive an email notification announcing actual site visit date You should begin your PIF If you haven�t already done so� Site Visit to Site Visit: An Ideal Time Line

18. 4 - 6 weeks before the visit Site Visitor will contact you to set up schedule for site visit date Here are a few of things the site visitor will probably ask for: List of required documents (policy manual, G&O�s etc.) One large conference room Be prepared to setup meetings with PD, faculty, residents, DIO Large programs: 10-12 residents should be peer selected Small programs: all residents should be present Sometimes: tour of facilities If possible, no beepers and available for the duration 14 days (10 business days) before the site visit Mail PIF to the site visitor Site Visit to Site Visit: An Ideal Time Line RRC Notification Letter Type of site visitor (Field Staff or SSV) Approximate date of next site visit Field staff scheduling letter Letter from Ingrid Philibert, specifying the date Specialist Site Visitor scheduling letter Letter from Rochella Ratliff, specifying date report must be received RRC Notification Letter Type of site visitor (Field Staff or SSV) Approximate date of next site visit Field staff scheduling letter Letter from Ingrid Philibert, specifying the date Specialist Site Visitor scheduling letter Letter from Rochella Ratliff, specifying date report must be received

19. Site Visit to Site Visit: An Ideal Time Line **** The Site Visit Occurs **** 1 - 6 weeks after the visit Site visit report is completed Report is sent to RRC Staff 2 - 8 months after the visit Program is assigned to a RRC reviewers The RRC meets, reviews your program and makes an accreditation decision and assigns review cycle date

20. Preparing for the Visit: The site visit date The Question: What if the site assigned site visit date is a problem? The Answer: There is never a good time for a site visit. However, if it�s a major problem: Ingrid Philibert: (312) 755-5003 or iphilibert@acgme.org Jim Cichon: (312) 755-5015 or jcichon@acgme.org Penny Lawrence: (312) 755-5014 or pil@acgme.org Do not call the site visitor, the RRC staff, or the RRC Chair. Call soon after you receive your email/letter announcing the visit Requesting a postponement does not guarantee one

21. # of residents in program - # of names on resident list # of months/FTEs at each institution/or in specific rotation - # of months on block diagram # of patients per resident in clinic - overall clinic volume (and no explanation that some patients are not seen by residents) # of beds available - patient census # of performance evaluations - frequency of reviews Institution referenced in narrative or block diagram, not mentioned anywhere else Faculty list/faculty credentials - faculty CVs Accuracy and source(s) of data PIF Corrections Copy to SV should be final Revisions before site visit Provide SV with changed pages and highlight changes Revisions on the day of site visit = OK (if possible) Revisions after the site visit Only by written addendum to PIF Sent to RRC team at least 6 weeks before RRC meeting # of residents in program - # of names on resident list # of months/FTEs at each institution/or in specific rotation - # of months on block diagram # of patients per resident in clinic - overall clinic volume (and no explanation that some patients are not seen by residents) # of beds available - patient census # of performance evaluations - frequency of reviews Institution referenced in narrative or block diagram, not mentioned anywhere else Faculty list/faculty credentials - faculty CVs Accuracy and source(s) of data PIF Corrections Copy to SV should be final Revisions before site visit Provide SV with changed pages and highlight changes Revisions on the day of site visit = OK (if possible) Revisions after the site visit Only by written addendum to PIF Sent to RRC team at least 6 weeks before RRC meeting

22. The Day of the Visit: ACGME Interviews ACGME interviews program director, faculty, residents and others. Site visitors �triangulate� response to verify and clarify the PIF. Compares responses in the PIF to interview data and actual observations (e.g., call rooms, libraries, exam rooms etc.) Compares participants responses to each other Discrepancies presented to the program director for clarification and resolution

23. The Day of the Visit: Resident Interview The resident interview explores the educational program from the residents� perspective (as �consumers�) Residents are peer selected (if small program, all residents) Site visitor seeks consensus from residents Sample Questions: Have you seen/received a copy of program�s educational goals and objectives? How and how often are you evaluated? Do you evaluate your faculty and your rotations? How are you supervised? If you had �due process� concerns, where would you find the procedures? Whom would you contact? What are the programs strengths? Areas for improvement? What are the reasons you chose this program?

24. The Day of the Visit: PIF Corrections The PIF submitted to site visitor is considered �final.� However,�.. If there are revisions on the day of the visit Fix and provide SV with changed pages Highlight changes made Replace pages in copies to go to ACGME If there are revisions after the site visit � only if absolutely necessary Always done as addendum to the PIF

25. After the Site Visit: The site visitor�s report What goes into the report? Review of the program history Review of previous actions and actions taken to correct Review of institutional issues that pertain to the program Clarification and verification of the PIF Specialty specific content What does not go into the report? NO opinions or non-factual based comments �Just the facts�nothing but the facts��

26. After the visit: If you have questions� About the PIF/Case logs -- technical questions Visit the ACGME web site at www.acgme.org Contact ACGME Help Desk: helpdesk@acgme.org or 312/755-7464 For Part I of PIF off of WebADS: reng@acgme.org For Case logs: oplog@acgme.org About PIF content/Program Requirements For PIF questions: dbraun@acgme.org or (312) 755-5492 For program requirements questions: jvasilias@acgme.org or (312) 755-7477; cfischer@acgme.org or (312) 755-5044 Look on Peds webpage and FAQ About the site visit/After the site visit Call the site visitor or the Department of Field Activities Look at the site visit FAQ Complete site visitor evaluation

27. ACGME�s Strategic Plan In November 2005, ACGME endorsed 4 strategic priorities. 1) Foster innovation and improvement in the learning environment 2) increase the accreditation emphasis on educational outcomes 3) Increase efficiency and reduce burden from accreditation 4) Improve communication and collaboration with key internal and external stakeholders Many of the changes ACGME has implemented in the recent past and will continue to roll out in the future can be seen from the vantage point of the 4 strategic priorities.

28. Innovations and Improvements to the Learning Environment Committee on Innovation and the Learning Environment (CILE) Recommendation #4: Develop a set of ACGME-supported pilots to assist the RC in advancing innovation in the learning environment and meeting ACGME�s strategic priorities. CILE Pilots to help in refining/revising select common requirement standards, specifically duty hours and competencies. RRC will decide whether to participate ACGME�s Learning Innovation and Improvement Project: studying the characteristics of institutions that foster innovation and sharing w/ others Conversations between RRC and R3P

29. Increase Emphasis on Outcomes Common Requirements are a move towards more outcome-focused requirements and less process. Competency based G & O�s Fuller/fleshed out language within the competencies Evaluation of program written to focus on how evaluation can improve program RRC �Companion Document� RRC front-runner in including competencies as organizing structure for requirements and competency questions in PIF A �different� type of citation: constructive/improvement based citations related to the competencies

30. Increase Efficiency and Reduce Burden New Common Requirements: less overall requirements; outcomes focused E-mail accreditation status of programs on agenda w/in 2-3 days E-mail to inform you that notification letter is posted on ADS. Standard letters of notification Increased information being collected through ADS Revised glossary of terms PD Guide to Common Program Requirements PD Virtual handbook RRC removed request for documentation of continuity numbers; site visitor will verify

31. Improve Communication and Collaboration Redesigned webpage Email updates when necessary Post presentations on website Newsletter will be implemented soon Conversations related to innovation Conversations related to the case log system jvasilias@acgme.org cfischer@acgme.org dbraun@acgme.org

32. www.acgme.org Staff contact information ACGME Policies & Procedures Competencies/Outcomes Project List of accredited programs ADS Duty hours Information/FAQ Affiliation Agreements FAQ General information on site visit process and your site visitor Pediatrics Webpage Resident complement increase Program Requirements and PIFs Case Log System Archive of RRC Updates/Email Communications Pediatrics FAQ

33.

34. Completing a PIF is �Preparing for a site visit is � Simple Complicated Complex