Quantitative Research Methods for the Antibiotic Steward

1. Quantitative Research Methods for the Antibiotic Steward Pranita D. Tamma, MD, MHS Associate Professor, Pediatrics Director, Pediatric Antibiotic Stewardship Program Johns Hopkins University School of Medicine

2. Objectives • Recognize how to define relevant exposures and confounders in observational studies • Review how to determine relevant outcomes in observational studies • Review the pros and cons of propensity scores in observational studies

3. Exposure Outcome Something the patient was exposed to [antibiotic, microorganism] Individual attribute [Immunocompromised] Intervention [Antibiotic time-out tool] An event under consideration [Length of hospital stay, mortality]

4. Exposure Outcome Confounders

5. Confounders • Variables associated with the exposure of interest and a potential cause of the outcome of interest • Should not be an intermediate step in the causal pathway • Can lead to bias that distorts the magnitude of the relationship between the exposure and outcome

6. Combination antibiotic therapy 28-day mortality ICU admission Study question: Do patients with gram-negative bloodstream infections (BSI) who receive combination antibiotic therapy (β-lactam and aminoglycoside) have a reduced odds of 28-day mortality?

7. Combination antibiotic therapy 28-day mortality ICU admission

8. You Are Trying to Design a Study … • National guidelines make general recommendations about duration of antibiotic therapy for gram-negative BSI, but the optimal route of therapy remains undefined • Patients with gram-negative BSI generally receive parenteral therapy but it is unclear if therapy can be transitioned to oral therapy after an appropriate clinical response is observed without compromising patient outcomes • Limited observational data available and mostly limited to urinary tract infections • Benefits of oral therapy are well-recognized • Reducing infectious and noninfectious complications of catheters, improved mobility, eliminating discomfort associated with IV catheters, etc.

9. What is the Best Approach to Evaluate this Question? • Quasi-experimental study • Will phone calls from the stewardship team increase the proportion of patients transitioned to oral step-down therapy with gram-negative BSI? • Cohort study • Does early transition to oral therapy for gram-negative BSI increase the risk of poor outcomes?

10. Receipt of phone call from stewardship team Early transition to oral step-down therapy Hypothesis: A phone call from the stewardship team to prescribers around day 3-5 reviewing if patient is a candidate for transitioning to oral therapy

11. Interrupted-Time Series Analysis • Drawbacks: • Requires prolonged periods of time for evaluation • Changes in other interventions over time can introduce bias Analyzes rate during intervention Analyzes rate before intervention Accounts for pre-existing trends

12. Thinking More About a Quasi-Experimental Study… • Population characteristics should generally be consistent over time • Intervention should be uniformly applied • Need a defined implementation start date • Normalize count data • e.g., days of antibiotic therapy per 1,000 patient-days, C. difficile infection rates per 100 hospital admissions, etc. • Select outcome measurable across units of time using a consistent approach • Control group optional • e.g., days of IV antibiotic therapy for another syndrome, days of IV antibiotic use in a non-intervention unit, days of antibiotic therapy for gram-positive agents, etc. Shardell M, et al. ClinInfec Dis 2007; 45:901-7.

13. Oral step-down therapy Outcomes Objective: To compare the clinical outcomes of patients transitioned to oral step-down therapy versus continued parenteral antibiotic therapy for the treatment of gram-negative BSI

14. Description of Cohort • Multicenter observational cohort including adults 18 years of age and older hospitalized with monomicrobialEnterobacteriaceae BSI from 2008-2014 • Includes 4,967 unique patients • Data collected by manual chart review • Data available • Demographics • Pre-existing medical conditions • Source of BSI & source control measures • Microbiology data including antibiotic MICs • Inpatient and discharge antibiotic data • Clinical outcomes data Tamma PD, et al JAMA Intern Med. 2019 doi: 10.1001/jamainternmed.2018.6226. [Epub ahead of print]

15. Define Your Eligibility Criteria

16. Define Your Eligibility Criteria Blood cultures obtained, in vitro active antibiotics initiated Day 7 Day 5 Day 1 Day 2 ?

17. Selecting Duration of Therapy for Eligibility Criteria 16 days

18. Define Your Eligibility Criteria Blood cultures obtained, in vitro active antibiotics initiated Day 7 Day 16 Day 5 Day 1 Day 2 Total duration of antibiotic therapy 7-16 days

19. Define Your Eligibility Criteria Excluded if never received IV therapy Excluded if transitioned to IV therapy after day 5 Day 7 Day 16 Day 5 Day 1 Day 2 • Eligible for oral step-down therapy: • Source control • Pitt bacteremia score ≤1 • Ability to consume and absorb enteral antibiotics • In vitro active oral antibiotic option available Cohort size reduced from 4,967 to 2,161 patients.

20. Oral step-down therapy Outcome

21. What Outcomes Appeal to Clinicians? • Most clinicians want to see improvements in patient-centered outcomes • They want to know that the status quo is harming patients, necessitating a change in practice • Or, a change in treatment practices will not worsen clinical outcomes • Ideally, your stewardship intervention or cohort study will show both of these

22. Components of Clinical Pulmonary Infection Score (CPIS) • Temperature • Peripheral white blood cell count • Tracheal sections • Oxygenation • Progression of pulmonary infiltrate • Culture of tracheal aspirate • A score >6 is suggestive of pneumonia Singh N, et al. Am J Respir Crit Care Med 2000;162:505-11..

23. CPIS ≤ 6 Randomize Median antibiotic duration: 10 days Median antibiotic duration: 3 days Standard Care (antibiotics for 10-21 days) Antibiotics for 3 days Re-evaluate at3 days CPIS > 6 CPIS ≤ 6 Treat as pneumonia Discontinue antibiotics Singh N, et al. Am J Respir Crit Care Med 2000;162:505-11..

24. CPIS ≤ 6 Randomize Standard Care (antibiotics for 10-21 days) Antibiotics for 3 days ICU length of stay: 15 days ICU length of stay: 3 days Re-evaluate at3 days CPIS > 6 CPIS ≤ 6 Treat as pneumonia Discontinue antibiotics Singh N, et al. Am J Respir Crit Care Med 2000;162:505-11..

25. CPIS ≤ 6 Randomize 35% developed subsequent antibiotic resistant infections 15% developed subsequent antibiotic resistant infections Standard Care (antibiotics for 10-21 days) Antibiotics for 3 days Re-evaluate at3 days CPIS > 6 CPIS ≤ 6 Treat as pneumonia Discontinue antibiotics Singh N, et al. Am J Respir Crit Care Med 2000;162:505-11..

26. Why Was this Study Successful? • Goal to improve patient outcomes and not decrease antibiotic use • Focused on a diagnosis with good evidence to support the treatment recommendations • Involved multiple opportunities for interventions that could be scalable to the comfort level of the treating clinician • Treating clinician ultimately made all treatment decisions • Showed that the stewardship outcome was safe • Reduced ICU stay • “Harm” caused by the status quo impacted the patients in the study • More likely to have subsequent drug-resistant infections

27. Process Measures • Antibiotic usage • Antibiotic costs • Clinical Measures • C. difficile infections • Central-line complications • End-organ toxicity • Mortality • Length of hospital stay • Infection recurrence • Antibiotic resistance • Balancing Measures • Hospital readmissions • Delays in patient discharges

28. Selecting Meaningful Clinical Outcomes Harm with the Status Quo No Harm with the Intervention Mortality Relatively rare outcome – especially attributable mortality Length of stay Most useful for studies where promoting IV to oral switch or decreased antibiotic duration Infection recurrence Hospital readmission • Antibiotic resistance • Lack of specificity with antibiograms • Time consuming to evaluate patient-level resistance • C. difficile infections • Relatively rare outcome • Lapses in infection control • May occur months after intervention • Higher associations with certain agents • Confounding due to overtesting • PICC complications • Infectious, thrombotic, mechanical • End-organ toxicity • Except for acute kidney injury, relatively rare outcome

29. Back to Our Study… Oral step-down therapy 30-day mortality From the literature: 30-day mortality for gram-negative BSI is approximately 10-15%.

30. Oral step-down therapy 30-day mortality Confounders

31. Two Patients with E. coli Bacteremia…. Likely to remain on IV therapy Likely to transition early to oral therapy Age = 75 years Renal transplant Intra-abdominal source Intensive care unit Pitt bacteremia score = 6 Age = 37 years Otherwise healthy Urinary source General ward Pitt bacteremia score = 1

32. Assessing Covariate Balance

33. Assessing Covariate Balance

34. Addressing Confounding by Indication • Impact of an intervention best assessed by randomizing treatment assignments to ensure patients are similar in the oral step-down and IV therapy groups • Randomization is not possible in observational studies so adjustment for other differences is necessary to obtain valid estimates of the associations between the exposure and outcome • Consider multivariable regression analysis or propensity score methods • Regression analysis determines how the estimate of the outcome changes when a variable changes, while the other variables are held constant • Propensity score methods allow for substantially more variables to be accounted for, increasing the ability to adjust for confounding

35. Using Propensity Score Methods to Account for Differences in the Exposed and Unexposed • Goal is to develop two groups that are similar in all characteristics expect the route of therapy prescribed • The propensity score is the probability a patient would transition to oral step-down therapy, based on characteristics of the patient, organism, and treating clinician • Propensity scores are estimated using multivariable logistic regression, in which patient characteristics are the predictors to determine the odds of being assigned to the oral step-down therapy group • These probabilities are estimated (ranging from 0 to 1) for each patient in the study population • These probabilities- the propensity scores- are then used to adjust for differences between the short-course and prolonged-course groups • A standardized bias >0.10 represent imbalance between the distribution of covariates

36. Propensity-Score Matching • Matching patients transitioned to oral step-down therapy and those that remained on IV therapy based on similar or identical propensity scores • Most commonly 1:1 nearest neighbor matching (can do 2:1, 3:1,..) • Can do some exact matching • Can do matching with replacement or without replacement: if enough good matches, match without replacement • Can impose a “caliper” to limit matches to be within some range of propensity score values (commonly 0.25 propensity score standard deviations) • Drawback: Generally reduces your sample size as patients without a match are excluded • Need a reasonably large cohort of patients

37. Residual confounding likely will always still persist!

38. Jitter Plot

39. Propensity-Score Stratification • Separating study participants into distinct strata with similar propensity score values • Generally 5 strata used • The association between oral step-down therapy and 28-day mortality is estimated within each stratum or pooled across strata to provide an estimate of this relationship • Drawback: Need enough oral step-down and IV patients within each strata

40. Propensity Score Weighting • Propensity scores used to calculate weights for each individual who are “down-weighted” or “up-weighted” to create a contrived population of oral-step down patients who look similar to the IV patients • When someone prescribed oral step-down therapy looks like the patients who received IV therapy, we clone that patient in the data (upweighting) • Someone who is already over-represented in the oral step-down group gets reduced in influence (downweighting) • Develop a new “pseudo-population” of oral step-down people who look similar to the IV therapy group • Does not sacrifice people who cannot be matched; everyone in cohort is included • Drawback: More challenging for clinicians to conceptualize as each person is no longer represented by a whole number anymore

41. Steps in Propensity Score Matching 1- Identify your cohort of patients with gram-negative bacteremia 2- Define your eligibility criteria 3- Define your exposure and outcome 3- Select covariates on which to match (i.e., what variables might influence the decision to transition to oral step-down therapy?) 4- Estimate the propensity score using logistic regression (the outcome is the odds of receiving oral step-down therapy and the exposures are the variables identified in #3) 5- Run the diagnostics to find the best matches for patients 6- Check for appropriate balance (covariate distribution) between the two propensity-score matched groups 7- Determine the odds ratio for 30-day mortality in the propensity-score matched sample (less prone to cheating)

42. Assessing Covariate Balance AFTER Matching Cohort size reduced from 4,967 2,161 1,478 patients.

43. Assessing Covariate Balance AFTER Matching

44. Outcomes

45. Contextualizing Your Findings • Highlight your main findings • Briefly review available literature on the topic and emphasize what gap in knowledge your study fills • Discuss the population your findings are generalizable to • Discuss the limitations with your study • If your study was observational, there will always be residual confounding • Discuss the next steps that need to be done to validate your findings or to expand the current work to subpopulations not included in your study

46. Summary • Take the time to define your question, primary exposure, primary outcome, eligibility criteria, and potential confounders up front • Consider focusing your interventions/cohort studies on an infectious diseases syndrome • Ask clinicians what outcomes data are important to them • Feed back data to providers • Publish your findings to guide others and to give the findings credibility • Know when to ask for help!