1 / 39

Nils-Olov Stålhammar

Health Economics II – 2010 Health Economic Evaluations Part V Lecture 3 Evaluations based on patient level data. Nils-Olov Stålhammar. NICE on Estimating Effectiveness of Interventions .

coyne
Download Presentation

Nils-Olov Stålhammar

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Health Economics II – 2010Health Economic Evaluations Part VLecture 3Evaluations based on patient level data Nils-Olov Stålhammar

  2. NICE on Estimating Effectiveness of Interventions • The most reliable evidence come from experimental studies with high internal and external validity that have inclusion and exclusion criteria that have been defined a priori. • A hierarchy that describes relative validity for estimating relative treatment effect: • Randomised Controlled Trials • Randomised = patients are randomised to one of two or more treatment arms • Controlled = a comparison is made to placebo or other active treatment • The RCT is often Double blind = neither the physician nor the patient knows which treatment arm the patient has been randomised to • Controlled observational studies • Can be prospective and randomised – the main differences vs. RCT are then that the study is open (not blinded) and treatment as in clinical practice • Can also be done as a retrospective database study • Observational studies without control • Expert opinion

  3. How can clinical trials be used? • Retrospective analysis based on a single clinical trial • Modelling analysis using data from clinical trials and other sources • Prospective analysis alongside a clinical trial (trial based evaluation) • Good data quality (at least high internal validity) • Good use of resources • Good timing • Probably the type of analysis with the highest credibility

  4. Preclinical phase Phase III Phase IV Phase I-II Cost-of-illness Methodological problems identified Early models Baseline info expanded Models continuously updated and refined Clinical trials may be used to test questionnaires Comprehensive evaluations - alongside clinical trials when appropriate Studies in clinical practice to defend price Health economics in drug development First time into man Registration and launch

  5. Conflicting characteristics of a typical clinical trial and a health economic study Clinical trial Health economic study • Efficacy - Cost and effectiveness • Double blind - Open? • Selected patient population - All patients likely to be treated • Compare to placebo - Compare to treatment likely to be replaced • Investigations not routinely - Investigations as in clinical done in clinical practice practice - Conducted in specialist clinics - The setting should be representative for clinical practice

  6. Conflicting characteristics of a typical clinical trial and a health economic study Clinical trial Health economic study • Rigid treatment protocols - Allow for variance in High compliance practice • No follow-up after discon- - Data is needed for all patients tinuation of treatment during the planned study period • Intermediate outcomes - Final outcomes are needed - Short study period - A rather long period must often be considered

  7. Factors influencing the decision to use or not to use a clinical trial for a prospective economic evaluation • To which extent are the desired criteria fulfilled? • How much can the clinical trial be modified without jeopardizing other objectives? • What are the alternatives? • modelling? (may be needed anyway) • later trials? • How strong is the need for HE data?

  8. Important aspects when designing a health economic evaluation • Alternatives to be compared • Patient population • Perspective; Costs to be included • Appropriate measure of outcome/effectiveness • How to measure and value costs and effectiveness • Study time period • How to address uncertainty

  9. Additional important aspects when designing a trial based health economic evaluation • Length of study period should be ‘sufficient’ • Extrapolation beyond study period often necessary (If a life threatening disease is studied then the HE-evaluation will typically cover the remaining life time) • Ideally, all patients in the RCT should be followed for the same time period • Sample size and power for the economic evaluation • Economic evaluation often require bigger sample size • Could lead to ethically challenging questions • Usually sample size is given – calculate the power

  10. Calculating the sample size H0 = no difference (for instance)  = The probability of rejecting H0 when it is true  = The probability of not rejecting H0 when it is false The power of a study = 1- Assuming that two variables have the same std. And are normally distributed, the required sample size N can be expressed as (1 = mean of variable 1):

  11. Additional important aspects when designing a trial based health economic evaluation • Record all costs or focus on selection of costs (those supposed to be affected by the treatments studied)? • ‘Noise’ vs. risk of excluding costs that are affected • Challenging to identify truly related costs • How to collect information about resource use • If directly from patients, consider ‘recalling bias’ • If data is collected from subset of patients/centres, do not allow ‘self-selection’ • Collect information on units of resource use, multiply with unit price/cost in the analysis • How to extrapolate beyond trial period – if needed

  12. Economic evaluation from the 4S study • Objective: To investigate whether simvastatin therapy in post-MI and angina patients with serum total cholesterol levels of 5.5 to 8.0 mmol/l is cost-effective • Costs: Costs of simvastatin and costs of CVD hospitalization • Effectiveness: Number of life years gained • Study period: Median 5.4 years • During study gain in life years: • Undiscounted 0.065 • Discounted 0.054

  13. End of study mortality benefit • Assumes no additional benefit after the end of the trial due to simvastatin = all patients face the same risk of dying • Both treatment groups have the same average life expectancy of 10 additional years (epidemiological data for 65 year old post MI patients); • Assuming constant rate of death, half of the survivors at end of study dead at 10 years and all patients are dead by 20 years • Note that other assumptions could be made re. effect after stop of treatment • Rebound effect – higher rate of death in previously treated pts • Pro-longed effect % Survival 1 Simvastatin Placebo 0 5 10 15 20 Years End of clinical study period

  14. Life Years Saved (LYS) Undiscounted Discounted During study gain in life years 0.065 0.054 End of study mortality difference * 0.312 0.186 Total 0.377 0.240 *) Assumes patients alive at end of study have life expectancy of 10 additional years

  15. Cost per Life Year Saved (LYS) = Net costs / life years saved = (Drug cost - hospital reductions) / life years saved = (21,100 SEK - 7,560 SEK) / 0.240 LYS = 13,540 SEK / 0.240 LYS = 56,400 SEK / life years saved (£5,500 /life year saved) [= 250,000 SEK / LYS during the clinical study] • Note: • Treatment is here supposed to stop at end of clinical trial even though this type of treatment typically is intended to be life long • Analysing continued treatment requires additional assumptions • Restriction of analysis may also be partly justified by low compliance in the long run and effect of discounting

  16. Additional important aspects when designing a trial based health economic evaluation • If a multinational trial, consider what can be pooled Tirilazadmesylate for treatment of aneurysmal subarachnoid hemorrhage Cost per death averted (trial involved Europe and Australia) Country Country specific Country specific Country specific cost* and mortality cost*, trial mortality prices only 1 $11,450 $5,921 $46,818 2 $60,358 $91,906 $57,636 3 $244,133 $90,487 $53,891 4 $181,259 $93,326 $69,145 5 ** ** $65,800 All $45,892 $45,892 $45,892 * = prices and resource use ** = dominant R J Willke, Nov 1997

  17. Additional important aspects when designing a trial based health economic evaluation • How to deal with missing data • To which extent can data be extrapolated from what has been observed? • How to address uncertainty • Methodological • Reference case, sensitivity analysis (one- and two-way analyses) • Sampling variation • Confidence intervals (Stochastic analysis) • Extrapolation beyond trial and to other settings • Modelling; Sensitivity analysis • Specify all above in a plan before the study starts

  18. Quantifying uncertainty – sampling variation • Challenging to calculate Confidence Interval for a ratio (ICER = C/E) • Fieller’s method; assumes that numerator and denominator follow a joint normal distribution • Non-parametric bootstrapping • Re-sampling from the original data to build an empirical estimate of the sampling distribution of the ICER • Draw samples and calculate mean C and E for treatment group and control group • Calculate ICER • Repeat 1-2 many times • Calculate CIs Upperlimit of CI Lowerlimit of CI ΔC ΔQALYs

  19. CEAC – Cost-effectiveness acceptability curve 100’/QALY ΔC 75’/QALY ΔQALYs P that intervention is cost-effective CEAC 1 75’ 100’ Threshold WTP

  20. Alternative: Net Monetary Benefit (NMB) The variance of NMB can be easily defined ICER for A NMB of A vs. Existing care NMB Upper 95% CI RT or WTP Neg. value of increm. cost of A Lower 95% CI ΔE=RT

  21. Net Monetary Benefit (NMB) When multiple mutually exclusive alternatives: The alternative with the highest (average) NMB is most cost-effective Average NMB makes sense, average C-E ratio does not

  22. Acceptability curve based on Net Monetary Benefit (NMB) • Bootstrapping from primary data (RCT) will show that option X is cost-effective in Px% of all ’trials’ for a given WTP • Changing WTP will change Px • P for multiple alternatives must sum to 1 P Z Y X WTP

  23. Sensitivity Analyses

  24. Sensitivity Analyses

  25. Sensitivity Analyses

  26. Intermittent treatment of GERD The clinical study Randomized, double-blind, 3 different drug therapies, 1 year follow-up, 6 countries The economic evaluation Cost-effectiveness analysis, direct and indirect costs, one analysis for each country, days without medication and symptom free days

  27. Intermittent treatment of GERDMain contents of the economic protocol 1. AIM Compare cost and effect during one year in the three treatment arms. Both direct and indirect costs. Number of days without medication and number of symptom free days. One analysis for each country. 2. METHOD 2.1 GENERAL CONSIDERATIONS Data on effect and quantities of used resources will be collected within the trial. Quantities will with two exceptions be pooled. The exceptions are sick-leave and mode and time of transport. Country specific prices from other sources. ITT analysis. When observations are missing, values must be imputed. Both stochastic analysis and sensitivity analysis will be done. Cont.

  28. 2.2 THE EFFECT 2.2.1 Days without medication 2.2.2 Days without symptoms All days between two consecutive visits if the patient is symptom free at the first of these two visits plus half the number of days between two consecutive visits if the patient is symptom free at the second visit only. 2.3 THE COSTS 2.3.1 The quantities Medication, visits, endoscopies. To avoid protocol driven costs, visits in the non-treatment phase are included only if symptoms suggestive of a relapse have reoccurred at such a visit. Mode of transport and travelling time are collected at two visits for the calculation of the typical cost of transport. These quantities will not be pooled. Number of hours absent from work due to sick-leave and visits. Will not be pooled. Cont.

  29. 2.3.2 The prices Country specific prices/costs will be specified in the report. 2.4 IMPUTING VALUES Three groups of patients: I) completing according to protocol II) withdrawn but followed III) lost to follow up / complete discontinuation For group III) values will be imputed based on observations in group II) after withdrawal. The monthly imputed cost and effect will be assumed to be the same in all three treatment arms. 2.5 STATISTICAL ANALYSIS Statistical analysis with CI for incremental c-e ratios. 2.6 SENSITIVITY ANALYSIS 2.6.1 Uncertainty with regard to costs 2.6.2 Uncertainty with regard to incomplete data 3. COMMUNICATION OF RESULTS

  30. Intermittent treatment of GERD Main lessons (remaining questions) • Do not underestimate the workload! • There will always be patients LTFU and it is not obvious how values should be imputed when observations are missing • Two measures of effectiveness? • Trial-wide or country specific data?

  31. A suggested checklist for assessing economic evaluations 6. Were costs and consequences valued credibly? 7. Were costs and consequences adjusted for differential timing? 8. Was an incremental analysis of costs and consequences of alternatives performed? 9. Was a sensitivity analysis performed? 10. Did the presentation and discussion of study results include all issues of concern to users? 1. Was a well-defined question posed in answerable form? 2. Was a comprehensive description of the competing alternatives given? 3. Was there evidence that the programs´ effectiveness had been established? 4. Were all the important and relevant costs and consequences for each alternative identified? 5. Were costs and consequences measured accurately in appropriate physical units? Drummond et al. 1997

  32. Checklist for conducting a health economic analysis alongside a clinical trial, version 1.2“Stalhammar’s 13 points” 1. Is the planning period sufficiently long? - will there be time to set up a model to identify cost drivers? - will there be time to develop and test CRF-questions? - will there be time to make the economic study an integrated part of the clinical study? 2. Are relevant alternatives being compared? - are the alternatives relevant for clinical practice? - is current practice among the alternatives? - is ‘no treatment’ a relevant alternative? Cont.

  33. 3. Does patient management reflect clinical practice? - ‘clinical practice treatment protocol’ or treatment at doctor’s discretion - a question of sample size? 4. Is the study period sufficiently long? - modelling beyond end of study period? - is the study period unnecessarily long (=reduced power)? - is the study period of the same length for all patients? Cont.

  34. 5. What kind of resource consumption should be assessed? - perspective of the analysis? - focus on disease related cost drivers (sometimes difficult to define but a broader focus requires a larger sample size) 6. How should costs be assessed? - billing data (possible in US) - resource consumption multiplied with prices - how should prices be gathered? - use prices from the year of the study - resource consumption from CRF is preferable, can indirectly come from patient or other sources (non-study ‘sites’) Cont.

  35. 7. Are there any protocol driven costs? - will some arm(s) be favoured if these costs are disregarded? - will the related (disregarded) procedure affect subsequent costs and outcomes? 8. How should effectiveness be measured? - if one-dimensional, which dimensions of outcomes will be disregarded? are they important? - if disease specific, to which extent will comparisons with other disease areas be possible? - how should data on effectiveness be captured? - if QALYs, patient or population weights? Cont.

  36. 9. How to adjust for baseline differences? - if different employment rates; total cost per employed patient? - if different travelling cost per visit; use the same average travelling cost for all visits irrespective of arm? 10. Cost and effectiveness for patients lost to follow-up? - can costs and effectiveness be based on patient characteristics at time of discontinuation? - pre-specify a basic rule, use others in a sensitivity analysis - minimise discontinuations; can patients be followed ‘outside’ of the clinical study? Cont.

  37. 11. If international study, which data can be pooled across countries? - test for homogeneity? - the assumption that efficacy data can be pooled is usually the rational for the multinational study 12. How to assess uncertainty? - statistical analysis of stochastic data; confidence interval for CE-ratio - sensitivity analysis of deterministic data; new CIs - present as acceptability curve and scatter diagram 13. How much of the analysis should be specified in advance? - as much as possible! - increases credibility

  38. Old exam question Randomised Controlled Trials (RCT) are generally regarded as the best source for evidence on a drug’s (pharmaceutical’s) efficacy. Well performed RCTs are generally a necessary condition for the successful licensing of a drug, i.e. approval to market and sell the drug. Quite often the RCTs are also used to collect economic data. However, there are several problems that researchers face when conducting an economic evaluation as part of a trial that has been designed primarily for achieving licensing (i.e. to study efficacy and safety). Describe at least five such problems. It should be clear from your answer why each aspect you mention is a problem.

  39. Old exam question a) Rearrange the cost-effectiveness decision rule (i.e. that the cost for an extra unit of effectiveness should be lower than a certain threshold value) so that it illustrates the decision rule for i) the Net Monetary Benefit (NMB) of the programme, and ii) the Net Health Benefit (NHB)of the programme. b) What is the main advantage of the NMB approach compared to working with incremental cost-effectiveness ratios? c) Draw a figure where the NMB and its upper and lower limits of the confidence intervals (CI) are presented as a function of the threshold value. Illustrate the situation where the lower limit for the CI for the incremental cost-effectiveness ratio is defined, but where the upper limit is not defined.

More Related