The Challenge and Importance of Evaluating Residents and FellowsDebra Weinstein, M.D. PHS GME Coordinators Retreat March 25, 2011
What keeps us up at night? • Recruiting top candidates • Protecting time for education • Duty hours compliance • Evaluating residents; providing feedback; dealing with poor performance
Why is this so important? • Duty to trainees • promote professional development • help each reach his/her greatest potential • Duty to society • ensure minimum standards of competence • promote development of accurate self-assessment, knowledge of individual limits
How do we know there’s a gap? • Internal reviews • ACGME resident survey • ACGME citations • Individual cases
Where are we falling short? • Many assigned evaluations are never done • Quality of evaluations is variable • Important questions not asked • Only ratings / no commentary • Intentional avoidance of critique • Evaluations not consistently reviewed with trainees • 360-degree evaluations not implemented
Goals • Protect current and future patients • Ensure appropriate balance of supervision and independence during training • Provide accurate references • Develop physician skills in self-assessment, understanding individual limits • Optimize professional development and satisfaction of individual trainees • Focus on exceptional talent and potential as well as weaknesses • Utilize performance reviews to individualize education and career counseling
Trainee assessment must be able to identify the spectrum of problems encountered • Insufficient academic progress: knowledge, judgment, decision making • Insufficient technical skill • Poor professionalism: dishonest, irresponsible, poor work ethic, inappropriate behavior • Interpersonal conflicts • Substance abuse • Mental health issues • Learning disabilities; ADD • Other: poor organizational skills, carelessness, lack of productivity, etc.
PHS Evaluation/Feedback Task Force: Key Recommendations • Stronger and more specific PHS policy • Steering Committee to oversee and champion the effort • Dedicated champion to implement and assess impact • Faculty development, resident/fellow training, improved infrastructure • Incentives/mandates for faculty
Faculty Development Initiatives Institution • ABIM Hospital Visit Program, May 2011 Department • “Train the trainer” sessions • Grand rounds, retreats, new hire orientation Individual • 1:1 coaching, role play
Resident/Fellow Training • shift in culture: seek, accept and utilized feedback • Formative, not punitive! • peer evaluation • self-evaluation
OSCE(Objective Structured Clinical Exam) • Objective: 1) Teaching 2) ?Early identification “at risk” trainees • Focus: Communication/interpersonal skills; patient safety • Plan: • 100 incoming intern participates during 2011 orientation program • Evaluate cost, feasibility, outcomes in considering future utilization
Targets for improved infrastructure • Templates for 360-degree evaluations, can be customized by program • Automated compliance reports from web-based evaluation system • Electronic vehicle to immediately communicate a concern
How will we measure progress? • Pre/post survey of faculty and trainees • knowledge, attitudes, confidence • Pre/post comparison of evaluation completion rates, and quality • Internal reviews and accreditation reports • Discussion with program directors, coordinators, trainees and faculty
Key Elements of An Improved Assessment System • Sufficient contact between trainees and faculty to allow for assessment • Consistent completion of high quality evaluations • Effective feedback to trainees • Early identification of performance problems with prompt intervention • Accessible resources for remediation
Coordinators’ Role:Work with Program Director to define this! May include: • Address the need for faculty contact/continuity in scheduling • Monitor evaluation completion rates; address non-compliance • Help to design and implement plan for obtaining 360-degree evaluations • Utilize and customize evaluation templates • Help to ensure that residents understand the program’s system for assessment: how is this communicated?