Quantitative Evaluation of Drug Name Safety Using Close-to-Reality Simulated Pharmacy Practice

1. CCEB Quantitative Evaluation of Drug Name Safety Using Close-to-Reality Simulated Pharmacy Practice Sean Hennessy, PharmD, PhD Assistant Professor of Epidemiology & Pharmacology Center for Clinical Epidemiology and Biostatistics University of Pennsylvania School of Medicine

2. Outline • Big-picture view of drug name evaluation • Improving the process by making it quantitative • Model for measurement in mock pharmacy setting • Research agenda

3. A Big-Picture View of Drug Name Evaluation • Outcome • accept • reject Qualitative Evaluation Process Name

4. A Big-Picture View of Drug Name Evaluation • Outcome • accept • reject Qualitative Evaluation Process

5. A Big-Picture View of Drug Name Evaluation • Outcome • accept • reject Qualitative Evaluation Process Quantitative Name

6. Outline • Big-picture view of drug name evaluation • Improving the process by making it quantitative • Model for measurement in mock pharmacy setting • Research agenda

7. Advantages of Quantitative Approach • Explicit and systematic • Uses fuller range of information • Transparency of data & assumptions • Acknowledges uncertainty • Identifies knowledge gaps

8. What underlies this binary (yes/no) decision? A Big-Picture View of Drug Name Evaluation • Outcome • accept • reject Quantitative Evaluation Process Name

9. Safe name Unsafe name Rating

10. Is this enough? A Big-Picture View of Drug Name Evaluation Quantitative Evaluation Process • Outcome • accept • reject • Rating • probability of error Name

11. Bates DW. Drug Safety 1996;15:303-10. Are All Medication Errors Created Equal?

12. Are These Equally Bad? • erythromycin clarithromycin • chloramphenicol chlorambucil

13. A Big-Picture View of Drug Name Evaluation Quantitative Evaluation Process • Outcome • accept • reject • Rating • probability of error • consequences of error • probability of AE Name

14. Probability of Adverse Event • Includes adverse outcomes from not getting intended drug • From placebo-controlled trials • ADE depends on identity of drug that is mistakenly substituted • Measured empirically, as discussed later • Frequency of ADEs in recipients of mistakenly substituted drug • From pharmacoepidemiologic studies

15. A Big-Picture View of Drug Name Evaluation Quantitative Evaluation Process • Outcome • accept • reject • Rating • probability of error • consequences of error • probability of AE • disutility of AE Name

16. Disutility • The value of avoiding a particular health state, usually expressed on a scale from 0 to 1 • Measured empirically by asking patients standardized questions

17. Disutility of Outcomes for Occult Bacteremia • Blood draw 0.0026 • Hospitalization 0.0079 • Meningitis recovery 0.0232 • Deafness 0.1379 • Minor brain damage 0.2607 • Severe brain damage 0.6097 • Death 0.9823 Benett JE, et al. Arch Ped & Adoles Med 2000;154:43-48.

18. A Possible Quantitative Rating Perror Consequenceserror =Perror PAEerror  Disutility of AE

19. Rating Consequences of error Probability of error

21. A Big-Picture View of Drug Name Evaluation Quantitative Evaluation Process • Outcome • accept • reject • Rating • probability of error • consequences of error • probability of AE • disutility of AE Name • What settings? • outpatient pharmacy • inpatient pharmacy • physician office • inpatient unit • nursing administration • patient home administration • etc.

22. Outline • Big-picture view of drug name evaluation • Improving the process by making it quantitative • Model for measurement in mock pharmacy setting • Research agenda

23. Potential Model for Name Evaluation: Mock Pharmacy Practice

24. A Big-Picture View of Drug Name Evaluation Quantitative Evaluation Process • Outcome • accept • reject • Rating • probability of error • consequences of error • probability of AE • disutility of AE Name

25. Close-to-RealitySimulated Pharmacy Practice • New or existing simulated pharmacies • Use per diem practicing pharmacists or late-year pharmacy students • Cost vs. realism • List test drugs in computerized drug info source • List test drugs in prescription entry program • Put test drugs on pharmacy shelf

26. Pharmacy Practice Lab for Testing Drug Names • Simulate pharmacy practice by presenting Rx’s (phone, hand-written, computer-generated) for real and test drug • Add Rx volume, noise, interruptions, 3rd party reimbursement issues, Muzak, etc. • Pharmacist enters and fills prescription • Measure the rate of name mix-ups at all stages of filling process, and which drug was mistakenly substituted

27. Getting from EvaluationRating • For probability of error, use point estimate or upper confidence limit (CL)? Maximum value statistically compatible with data; function of measured rate & sample size

28. Getting from EvaluationRating • For probability of error, use point estimate or upper confidence limit (CL)? • Using upper CL encourages bigger studies • What coverage for CLs? (95%? 90%? 80%?) • Base on what seems reasonable using real data?

29. Potential Advantages vs. Expert Opinion • Yields empiric estimates of error rate, and of which drugs are mistakenly substituted • Better face validity • Validity can be tested by examining known “bad” names • Makes knowledge & assumptions explicit

30. Obstacles & Limitations-1 • Hawthorne effect? • Initial improvement in a process of production caused by the obtrusive observation of that process • Technical challenges

31. Obstacles & Limitations-2 • Need large sample sizes • Use routinely, or just to validate qualitative approaches? • Worth the added cost?

32. Outline • Big-picture view of drug name evaluation • Improving the process by making it quantitative • Model for measurement in mock pharmacy setting • Research agenda

33. Research Agenda • Feasibility • Cost • Reliability • Validity • Utility