slide1 n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Gerhardt Pohl Eli Lilly and Company PowerPoint Presentation
Download Presentation
Gerhardt Pohl Eli Lilly and Company

Loading in 2 Seconds...

play fullscreen
1 / 41

Gerhardt Pohl Eli Lilly and Company - PowerPoint PPT Presentation


  • 123 Views
  • Uploaded on

Towards a Complete Solution for Cost/Effectiveness in Oncology: Handling Heterogeneity, Variability and Censoring. Gerhardt Pohl Eli Lilly and Company. Objective.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Gerhardt Pohl Eli Lilly and Company' - george


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
slide1

Towards a Complete Solution for Cost/Effectiveness in Oncology: Handling Heterogeneity, Variability and Censoring

Gerhardt Pohl

Eli Lilly and Company

objective
Objective

We view a “complete” solution to the problem of calculating the Incremental Cost Effectiveness Ratio (ICER) in oncology as one simultaneously addressing the key issues of (1) Heterogeneity (2) Variability and (3) Censoring.

The talk will discuss a stratified, bootstrap approach and draw links to the concept of “local propensity.”

what is an icer
What is an ICER??
  • Incremental Cost Effectiveness Ratio
  • Ratio of difference in mean cost divided by difference in mean effectiveness.

where T denotes the new treatment and C the control

impact
Impact
  • ICER is the primary tool used in cost-effectiveness comparisons by HTA (Health Technology Assessment) bodies around the world.
  • ICER’s allow comparisons of treatments between therapies and across disease states to allow appropriate choices in national health expenditures.
icer graphically
ICER Graphically

 $

More Expensive and Less Effective

More Expense for More Effectiveness

ICE Plane

 QALY

(Quality Adjusted Life Years)

Cheaper but Less Effective

Cheaper and Better

nice thresholds
NICE Thresholds

National Institute for Health and Clinical Excellence

Not Approvable

£50,000/QALY (End of Life Status)

£30,000/QALY

Approvable

“End of Life Status” is granted only for treatments which are life-extending (>3months) for patients (<7,000) with short life expectancy (<24 months)

analysis goal
Analysis Goal

Create a bootstrapped display of variability in the ICE plane where each iteration is based on risk-adjusted and censored estimates of the cost and survival.

Frick KD, et al. “Modeled cost-effectiveness of the experience corps Baltimore based on a pilot randomized trial.” Journal of Urban Health 2004; 81:106-117.

propensity score pictorially
Propensity Score Pictorially

More likely to receive red treatment

Comorbidities

Propensity Score Strata

More likely to receive blue treatment

Age

a downside of propensity scoring
A Downside of Propensity Scoring
  • Patients with the identical propensity score may have very different covariate levels.

Young

and Sick

Comorbidities

Old

But Healthy

Age

blocking
Blocking
  • Grid the factor space into blocks (unordered strata) of similar patients.
  • This may be thought of as a many-to-many matching directly in the covariate space.

Stratum 1

Stratum 2

Stratum 3

Comorbidities

Stratum 4

Stratum 5

Stratum 6

Stratum 7

Stratum 8

Stratum 9

Age

general approach
General Approach
  • Whatever the original dimension of the covariate space, this reduces the problem to cross-classification of treatments versus strata.

Stratum 1

Stratum 2

Stratum 3

Stratum 4

Stratum 5

Stratum 6

Stratum 7

Stratum 8

Stratum 9

calculate within stratum treatment differences
Calculate Within-Stratum Treatment Differences

Cost:

Effectiveness:

How to pool across strata?

stratum weighting
Stratum Weighting

For overall mean difference, pool relative to size of strata:

Definition of Stratified ICER:

pros and cons
Pros and Cons

Blocking

Propensity Score Matching

Reduces complexity of covariate space down to one dimension

Can maintain structure/ordering of covariate levels

Can borrow information across blocks

Potentially uses full richness of continuous data

  • Non-parametric
  • Provides better matches of underlying covariates
  • Able to capture complex interactions of covariates and likelihood of treatment
variability in the icer estimate
Variability in the ICER Estimate
  • Many methods have been proposed to incorporate variability into ICER based inference:
    • Univariate Sensitivity Analyses (Tornado Diagrams)
    • Confidence Intervals (Fieller’s Theorem, 2-dimensional boxes, ellipses, wedges, bootstrapping…)
    • Simulation
    • Cost-Effectiveness Acceptability Curves
    • Net Monetary Benefit
    • And various combinations thereof…
  • Fundamental technical issue is that the ratio of 2 normal variables is not normal. (Nor very tractable! For example, what if confidence interval for the denominator includes zero.)
  • Evidence in literature suggests that Bootstrapping provides robust and consistent inference. Bootstrapping is also relatively assumption free and easily, explained heuristically .
bootstrapping the stratified icer
Bootstrapping the Stratified ICER
  • Pre-specify strata (factors and cutoff levels).
  • Sample individual patient cost/effectiveness pairs.
  • Draw samples with replacement proportional to the size of the treatment groups -- Ignore stratification in drawing the samples.
  • Use the fixed, pre-specified boundaries to divide each sample into strata.
  • Calculate stratified difference in cost, stratified difference in effectiveness and stratified ICER for each sample.
interpreting the bootstrap samples
Interpreting the Bootstrap Samples

Windshield Wiper Regions

Proportion in Each Quadrant of ICE Plane

$

$

ICER

1.0%

0.5%

Upper 95% Conf. Int.

QALY

+

+

Lower 95% Conf. Int.

48.0%

50.5%

QALY

effects of bootstrapping against fixed strata boundaries
Effects of Bootstrapping against Fixed Strata Boundaries
  • The fraction of patients treated with one or the other treatment within a stratum changes from sample to sample.
  • This captures (some of) the component of variability due to estimating the propensity.
  • Fixed boundaries prevent technical issues that violate assumptions necessary to assure convergence of the bootstrap. (Abade and Imbens. “On the failure of the bootstrap fro Matching Estimators”. Econometrica, Vol. 76, Issue 6, pp. 1537-1557, Nov. 2008.)
stratification redux
Stratification Redux
  • Whatever the original dimension of the covariate space, reduce the problem to cross-classification of treatments versus strata.

Stratum 1

Stratum 2

Stratum 3

Stratum 4

Stratum 5

Stratum 6

Stratum 7

Stratum 8

Stratum 9

local propensity score
Local Propensity Score
  • Define the “Local Propensity Score” as the fraction of patients treated with treatment A within each stratum.
  • Blocking can be thought of as fitting a step function for the estimated propensity.
some algebra
Some Algebra

Sum by blocks

Within-block Average

Sum of Weights is

Total Sample Size

consequences of equivalence
Consequences of Equivalence
  • (For a class of statistics…)
  • Consider two individuals from different blocks but with same propensity score. They enter IPW statistic identically with same weight.
  • Therefore, even if matched via propensity score, the summary statistic remains the same.
  • Stated another way, matching within strata then calculating local propensity is equivalent to deriving local propensity and then matching via that propensity.
the downside of ps does not apply to stratified local propensity scoring
The Downside of PS Does Not Apply to Stratified Local Propensity Scoring
  • Underlying differences in covariates are irrelevant to the summary statistic.

Young

and Sick

Comorbidities

Old

But Healthy

Age

three possible weightings
Three Possible Weightings

Weighting Within-Stratum Differences

Marginal wrt

Treatment:

Marginal wrt

Control:

Marginal wrt

Population:

three possible weightings1
Three Possible Weightings

Weighting Within-Stratum Differences

IPW/Causal Inference

Marginal wrt

Treatment:

Average Treatment among

Treated (ATT)

Marginal wrt

Control:

Average Treatment among

Controls (ATT for Control Group)

Marginal wrt

Population:

Average Treatment Effect (ATE)

relationship among weightings
Relationship among Weightings

Average Treatment among

Treated (ATT)

Average Treatment among

Controls (ATT for Control Group)

Average Treatment Effect (ATE)

I.e., ATE is a convex combination of the ATT weightings for Treatment and Control proportional to the size of the groups.

implications
Implications

ATT and ATE weightings are the same,

if and only if, there is uniform propensity to treat

with regard to strata.

Therefore, differences under ATT and ATE weightings

inform on the impact of directed prescribing

(a.k.a. propensity).

population based weightings are not the same as anova weightings
Population-Based Weightings Are Not the Same as ANOVA Weightings
  • SAS Type I:
  • SAS Type II:
  • SAS Type III:

Differ, so not collapsible to function of

Inversely proportional to variance of

Uniform across strata

censored survival
Censored Survival
  • A recent paper reviewed the survival component of 45 Health Technology Assessments (HTA) submitted to National Institute for Health and Clinical Excellence (NICE) in the cancer disease area.
  • Nicholas R. Latimer. “Survival Analysis for Economic Evaluations alongside Clinical Trials—Extrapolation with Patient-Level Data: Inconsistencies, Limitations, and a Practical Guide”. Medical Decision Making, published online 22 January 2013.
  • A variety of methods were noted as having been used to estimate mean survival in the assessments
    • restricted means analyses, i.e., area under the K-M curve up until a certain point
    • parametric modeling (exponential, Weibull, Gompertz, etc.)
    • Partial Likelihood Regression/Proportional Hazards (which accounts for heterogeneity)
    • Reliance on estimates external to the study
  • STRONG PREFERENCE of author for parametric modeling : “a lifetime horizon is usually advocated, particularly for interventions that affect survival. Therefore, in the presence of censoring, extrapolation is required to predict the complete survival impact of the new intervention, which may be summarized as the mean survival benefit.
is censoring really so bad
Is Censoring Really So Bad?
  • “In 17 (38%) [H]TAs, extrapolation was not performed, with the survival analysis based purely on the observed trial data (restricted means analysis). Appropriately, this was generally only the case when there was relatively little censoring in the survival data from the trial.”
  • Consider a study with 1,000 patients followed until death. Now, add data from 1,000 more patients that have incomplete censored data. The supplemented data has 50% censoring but more information content. Is it really worse than a study one half the size with no censoring?
our approach
Our Approach
  • The restricted K-M mean provides the advantage of using all data and being non-parametric. So long as time horizon is adequately long to fully characterize survival function, censoring should not be a problem.
  • We calculate the K-M mean within each stratum as area under the curve up until the last death. This is conservative as regards the ICER as it tends to underestimate the denominator.
censored costs
Censored Costs
  • D. Y. Lin; E. J. Feuer; R. Etzioni; Y. Wax. “Estimating Medical Costs from Incomplete Follow-Up Data”. Biometrics, Vol. 53, No. 2. (Jun., 1997), pp. 419-434.
  • Basic concept is to calculate expected cost via conditioning, i.e., as the sum all intervals of the probability of survival to start of an interval times the average cost incurred during the interval by patients alive at the start of the interval.
calculation of censored cost
Calculation of Censored Cost

Input Dataset

One Record per Patient

Total Cost in Interval (E.g., Weekly Cost)

Time to Death or Censoring

calculation of censored cost1
Calculation of Censored Cost

Input Dataset

One Record per Patient

Calculate Averages

Kaplan-Meier

Average Period Cost among Patients Alive at Start of Time Period

Probability of

Survival until

Time Period

One Record per Time Interval

calculation of censored cost2
Calculation of Censored Cost

Input Dataset

One Record per Patient

Calculate Averages

Kaplan-Meier

Average Period Cost among Patients Alive at Start of Time Period

Expected (Censoring-Adjusted) Cost

Probability of

Survival until

Time Period

One Record per Time Interval

pulling it all together
Pulling It All Together
  • To adjust for heterogeneous risk stratify.
  • Address censoring by calculating censored mean survival and expected cost within strata.
  • Pool strata relative to stratum size.
  • Bootstrap at individual patient level against the pre-specified strata boundaries to incorporate variability arising from outcomes and propensity estimates.
  • All together, this yields a complete method to calculate ICER in oncology.