Benchmarking Clinicians Farrokh Alemi, Ph.D.
Why should it be done? • Hiring, promotion, and management decisions • Help clinicians improve
Intrusion in clinician’s practice? • Managers understand patient outcomes • Practice profiles are constructed after the fact, when the patient is gone. • Both the patient and managers can use benchmarked data • Poor clinicians are bad for the patient as well as for the organization
How should the analysis be done? • Compare clinician to average peer • Compare clinician to average peer taking care of same kinds of patients • Compare clinician to expectations on admission • Compare clinician and peers on patients matched on certain features In benchmarking, a clinician's performanceis compared to an expected value.
Compare Clinician to Average Peer • Calculate peer providers and the clinician’s average and standard deviation • Compare using test of hypothesis with unequal means • Problem: • Maybe misleading as providers see different kinds of patients and the clinician with more severely ill patients will naturally have worst outcomes
Example Data • 123 internal medicine residents at the New York-Presbyterian Hospital in New York City. • The outcomes examined included following outcomes: • Patients' satisfaction measured by telephone interviews of at least 10 patients • Disease-management profiles for average of 7 diabetes and 11 hypertensive patients. • Patient's condition • Frequency of use of various medications • Faculty-evaluations on seven dimensions: • History taking • Physical examination • Differential diagnosis • Diagnostic and/or treatment plan • Health care maintenance • Act compassionately • Team player
Comparing Clinicians to Expected Prognosis at Admission • Assess patients severity • Predict prognosis • Calculate pair-wise student-t comparison of observed and expected values
Is it reasonable to benchmark clinicians? • Measurement distorts goals • Measurement leads to defensive behavior • No adequate measure of severity maybe available • Too much time spent on measurement and too little on improvement
How Should Benchmarked Data Be Presented? • Before the meeting • Schedule a feedback time and date as soon as possible. • Check your data to make sure there are no errors. • Add text, charts or graphics. Supplement numeric data with anecdotal information and the customer's voice (e.g. a short audio from a patient). • Distribute handouts ahead of meeting to participants
How Should Benchmarked Data Be Presented? • At the meeting • Make it clear that the evaluation is confidential • Make a brief introduction of the purpose of the session • Acknowledge the limitation of the practice profiling method • Present the data and not the conclusions • Explicitly ask for clinician's evaluation of the data after each section of the report is presented • DO NOT defend the practice profiling method, the benchmarking effort or any aspect of your work • Thank the clinicians for their time and describe next steps
How Should Benchmarked Data Be Presented? • After the meeting • Summarize the comments and append it to the report. • Describe resources available. • Send a written report to each clinician. • Ask the clinicians to comment on: • What worked well and what needs improvement? • Do they plan to change their practice and in what way? • Was it worthwhile? • Set the time of next benchmarking report. • .
Take Home Lesson Expected Outcomes Can Be Benchmarked Using Severity of Patients’ Illness