Incorporating Meta-Analysis in Decision Modeling: Methodology & Applications

1. The Incorporation of Meta-Analysis Results into Evidence-Based Decision Modelling Nicola Cooper, Alex Sutton, Keith Abrams, Paul Lambert, David Jones Department of Epidemiology & Public Health, University of Leicester. CHEBS, Multi-Parameter Evidence Synthesis Workshop, Sheffield, March 2002

2. Where we fit in with Tony’s intro • Process •  Model relationship between evidence & parameters • Consistency check • Uncertainty Panacea • Statistical error • ½ Evidence relates to parameters indirectly • Systematic errors • Data quality, publication bias, etc

3. METHODOLOGIC PRINCIPLE 1) Pooled estimates

4. METHODOLOGIC PRINCIPLE 1) Pooled estimates • 2) Distribution

5. METHODOLOGIC PRINCIPLE 1) Pooled estimates • 2) Distribution 3) Transformation of distribution to transition probability (if required) (i) time variables: (ii) prob. variables:

6. METHODOLOGIC PRINCIPLE 1) Pooled estimates • 2) Distribution 4) Application to model 3) Transformation of distribution to transition probability (if required) (i) time variables: (ii) prob. variables:

7. EXAMPLES • Net Clinical Benefit Approach • Warfarin use for atrial fibrillation • Simple Economic Decision Model • Prophylactic antibiotic use in caesarean section • Markov Economic Decision Model • Taxane use in advanced breast cancer

8. MODELLING ISSUES COMMON TO ALL EXAMPLES • Bayesian methods implemented using Markov Chain Monte Carlo simulation within WinBUGS software • Random effect meta-analysis models used throughout • All prior distributions intended to be ‘vague’ unless otherwise indicated • Where uncertainty exists in the value of parameters (i.e. most of them!) they are treated as random variables • All analyses (decision model and subsidiary analyses) implemented in one cohesive program

9. EXAMPLE 1: NET (CLINICAL) BENEFIT Net Benefit = (Risk level x Risk reduction) – Harm • Glasziou, P. P. and Irwig, L. M. An evidence based approach to individualizing treatment. Br.Med.J. 1995; 311:1356-1359.

10. RE-ANALYSIS OF WARFARIN FOR NON-RHEUMATIC ATRIAL FIBRILLATION • Evidence that post MI, the risk of a stroke is reduced in patients with atrial fibrillation by taking warfarin • However, there is a risk of a fatal hemorrhage as a result of taking warfarin • For whom do the benefits outweigh the risks?

11. METHOD OUTLINE • Perform a meta-analysis of the RCTs to estimate the relative risk for benefit of the intervention • Use this to check the assumption that RR does not vary with patient risk • Check harm (adverse events) is constant across levels of risk (use RCTs and/or data from other sources) & estimate this risk • Place benefit & harm on same scale (assessment of QoL following different events) • Apply model - need to predict patients risk (identify risk factors and construct multivariate risk prediction equations)

12. SOURCES OF EVIDENCE Net Benefit = (risk of stroke x relative reduction in risk of stroke) - (risk of fatal bleed x outcome ratio) Multivariate risk equations M-A of RCTs M-A of RCTs/obs studies QoL study

13. Singer,D.E. Overview of the randomized trials to prevent stroke in atrial fibrillation. Ann Epidemiol 1993;3:567-7.

14. EVALUATING THE TRADE-OFF BETWEEN STROKE AND HEMORRHAGE EVENTS IN TERMS OF QOL • QoL following a fatal bleed = 0 • Data available on QoL of patients following stroke • Glasziou, P. P., Bromwich, S., and Simes, R. J. Quality of life six months after myocardial infarction treated with thrombolytic therapy. The Medical Journal of Australia. 1994; 161532-536 Proportion with index greater than horizontal axis value Time trade-off index

15. 10 6 8 6 4 4 2 2 0 0 -2.95 -2.90 -2.85 -2.80 -2.75 -2.70 -2.65 -1.5 -1.0 -0.5 0.0 0.5 1.0 reduction in relative risk 300 250 0.4 200 150 0.3 100 0.2 50 0.1 0 0.002 0.004 0.006 0.008 0.010 0.012 0.014 0.0 risk of bleed per year 0 20 40 60 80 100 Outcome ratio Multivariate Risk Equation Data Net Benefit (measured in stroke equivalents) No. T hrombo - Clinical No. of % of embolism Mean Median Probability of risk patients cohort rate (% (s.e.) (95% Benefit > 0 Simulated PDF factors per year CrI) (95% CI)) 6 5 4 2 or 3 68 12 17.6 (10.5 - 0.0004 0.06 54.2 % 3 to 29.9) (0.15) ( - 0.29 to 2 0.20) 1 0 -0.8 -0.6 -0.4 -0.2 0.0 0.2 0.4 2 or 3 Clinical factors EVALUATION OF NET BENEFIT Risk of stroke Relative risk per year e.g. reduction for for 1 or 2 clinical strokes taking risk factors: warfarin (1 - RR): 6.0% (4.1 to 0.23 (0.13 to 0.41) 8.8) Multivariate risk Meta - analyses equations of RCTs ´ (risk of stroke relative reduction in risk of stroke) - ´ (risk of fatal bleed outcome ratio) = Meta - Analysis of Net Benefit QoL study RCTs / obs studies Outcome ratio (1/ QoL Risk of fatal reduction) Median bleed per 3.75 (1.07 to 50), year taking Mean 26.14,indicating warfarin : the number of strokes 0.52% (0.27 that are equivalent to to 0.84) one death

17. “TAKE-HOME” POINTS 1 • Net-benefit provides a transparent quantitative framework to weigh up benefits and harms of an intervention • Utilises results from two meta-analyses and allows for correlation induced where studies included in both benefit and harm meta-analyses • Credible interval for net benefit can be constructed allowing for uncertainty in all model parameters

18. EXAMPLE 2: SIMPLE DECISION TREE • Use of prophylactic antibiotics to prevent wound infection following caesarean section

19. METHOD OUTLINE • Cochrane review of 61 RCTs evaluating prophylactic antibiotics use for caesarean section • Event data rare: use “Exact” model for RR • Meta-regression: Does treatment effect vary with patients’ underlying risk (pc)? ln(RRadjusted ) = ln(RRaverage)+  [ln(pc) - mean(ln(pc))] • Risk of infection without treatment from ‘local’ hospitaldata (p1) • Derive relative risk of treatment effect for ‘local’ hospital (using regression equation with pc=p1) • Derive riskof infection if antibiotics introduced to ‘local’ hospital (p2) p2 = p1 * RRadjusted

20. UNDERLYING BASELINE RISK =0.24 (-0.28 to 0.81) No treatment effect Local hospital event rate

21. Mean (95% Credible Interval) Posterior distribution Relative Risk, RRadjusted 0.30 (0.21 to 0.40) Prob(wound infection/placebo), p1 0.08 (0.06 to 0.10) Prob(wound infection/antibiotics),p2 0.02 (0.015 to 0.034) RESULTS

22. Mean (95% Credible Interval) Posterior distribution Reduction in cost using antibiotics -£49.53 (-£77.09 to -£26.79) Number of wound infections avoided using antibiotics per 1,000 53.09 (42.12 to 73.37) Between study variance (random effect in M-A), 2 0.30 (0.05 to 0.74) RESULTS (cont.) RESULTS

23. COST-EFFECTIVENESS PLANE Treatment more effective & more costly Control dominates Treatment less effective & less costly Treatment dominates

24. SENSITIVTY OF PRIORS [1] Gamma(0.001,0.001) on 2 [2] Normal(0,1.0-6) truncated at zero on  [3] Uniform(0,20) on 

25. “TAKE-HOME” POINTS 2 • Incorporates M-A into a decision model adjusting for a differential treatment effect with changes in baseline risk • Meta-regression model takes into account the fact that covariate is part of the definition of outcome • Rare event data modelled ‘exactly’ (i.e. removes the need for continuity corrections) & asymmetry in posterior distribution propogated • Sensitivity of overall results to prior distribution placed on the random effect term in a M-A

26. EXAMPLE 3: USE OF TAXANES FOR 2ND LINE TREATMENT OF BREAST CANCER Stages 1 & 2 (cycles 1 to 3) Treatment cycles Stage 3 (cycles 4 to 7) Stage 4 (cycles 8 to 35) Post -Treatment cycles

27. METHOD OUTLINE • Define structure of Markov model • Identify evidence used to inform each model parameter using meta-analysis where multiple sources available • Transform meta-analysis results, where necessary, into format required for model (e.g. rates into transition probabilities) • Informative prior distributions derived from elicited prior beliefs from clinicians • Evaluate Markov model

28. META-ANALYSES

29. TRANSITION PROBABILITIES

30. METHODOLOGIC PRINCIPLE 1) Pooled estimates • 2) Distribution 4) Application to model 3) Transformation of distribution to transition probability (if required) (i) time variables: (ii) prob. variables:

31. ELICITATION OF PRIORS e.g. Response Rate Taxane Standard

32. RESPONSE RATE

33. COST-EFFECTIVENESS PLANE

34. “TAKE-HOME” POINTS 3 • Synthesis of evidence, transformation of variables & evaluation of a complex Markov model carried out in a unified framework (facilitating sensitivity analysis) • Provides a framework to incorporate prior beliefs of experts

35. FURTHER ISSUES • Handling indirect comparisons correctly • E.g. Want to compare A v C but evidence only available on A v B & B v C etc. • Avoid breaking randomisation • Necessary complexity of model? • When to use approaches 1,2,3 above? • Use of predictive distributions • Necessary when inferences made at ‘unit’ level (e.g. hospital in 2nd example) rather than ‘population’ level? • Incorporation of EVI

37. MODEL SPECIFICATION Bayesian random effects M-A model specification: ln(RR) Prior distributions: Warn et al 2002 Stats in Med (in press)