Core Competency Assessment in Emergency Medicine from Design to Implementation

1. Core Competency Assessment in Emergency Medicinefrom Design to Implementation National Hispanic Medical Conference Christian Arbelaez, MD, MPH Associate Director, Office for Multicultural Faculty Careers Center for Faculty Development and Diversity Assistant Residency Director, Emergency Medicine Brigham and Women’s Hospital Washington, DC 2011

2. Educational purpose • To describe a comprehensive core competencies assessment program designed and implemented in our residency program. • The goals of the program are to provide the residents and residency program a robust feedback loop on the achievement of core competence education in Emergency Medicine.

3. Outline • Design • Implementation • 1 year Review

4. Design • Residency needs assessment • Review of the guidelines • Stakeholder meetings • Resource prioritization

5. Design • Design a comprehensive yet practical assessment curriculum • Program centered on multiple assessments balanced between simple and complex observational tools, and repeated at different times to assess a resident’s developmental progression over their training, need for resident remediation, and programmatic change.

6. Design • A semi-annual assessment form and online faculty evaluations were developed to document expected PGY-based core competencies and graduated responsibilities. • In-training examinations scores • 360-degree evaluation by resident peers and the clinical nursing leadership group. • Procedural competency • Endotracheal intubation • Central venous line placement • OSCE was used for the ultrasound assessment. • A resuscitation competency log was kept throughout each year. • A resident portfolio was generated to keep each of these documents • Self-reflection and Self-Assessment

7. Implementation • We developed an implementation and roll-out plan with reasonable set deadlines to meet the intern start date at the beginning of the academic year. • One of the key components was a clear and transparent communication plan for the residents, faculty, and all staff involved in assessment of the residents.

8. Implementation • During the resident meetings we presented the program objectives and expectations, outlined any changes in the electronic data management system, distributed semi-annual assessment forms, handed out the procedure cards, and provided examples of portfolio entries. • The residents also receive notification via the data management system to complete 360-degree peer evaluations. • The faculty meeting focused on the global faculty ratings and the need for direct observation and constructive feedback after the selected procedures. • The residents also met with the program manager to review their portfolios prior to their semi-annual review.

9. One-Year Review • After a year of implementation, we critically reviewed the assessment program and drafted a report outlining the individual components supported with resident data. • We then developed an action plan for phase 2 of our implementation plan and produced a residency promotion criterion.

10. One-Year Review • In order for promotion, at the PGY 3 end of year evaluation, the following are the basic requirements needed in the resident’s portfolio: • a completed resident learning contract • a completed Professionalism checklist • completed semi-annual evaluations • one Chief Complaint CC assessment per year (including one oral boards) • a completed Procedural/Resuscitation Competency achieving the ACGME targets by the end of PGY 3 • yearly 360 degree evaluations by the Clinical Nursing Group and Resident Peer-Peer, and • one self-assessment per year. • The end of the PGY 3 evaluation will be performed by one of the program directors and each of these items will be reviewed for completion with the resident. Accordingly, the resident will be promoted to the PGY 4 year or will be promoted with a remediation clause and a signed contract of expectations prior to granting of graduation the following year.

11. Conclusions • Residency programs are expected to be fully compliant and integrated with the ACGME core competency assessment guidelines by 2011. • Residency programs should be proactive in designing and implementing an evaluation and assessment program that best fits their needs based on institutional strengths and available resources. • We offer a comprehensive assessment program designed and successfully implemented at our institution that is sensitive to the residents, faculty, and residency program needs for accountability, growth, remediation, and improvement.

12. Questions/Contact Information Christian Arbelaez, MD, MPH 617-732-8947 carbelaez@partners.org