statistical challenges in the validation of surrogate endpoints l.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Statistical challenges in the validation of surrogate endpoints PowerPoint Presentation
Download Presentation
Statistical challenges in the validation of surrogate endpoints

Loading in 2 Seconds...

play fullscreen
1 / 35

Statistical challenges in the validation of surrogate endpoints - PowerPoint PPT Presentation


  • 66 Views
  • Uploaded on

Statistical challenges in the validation of surrogate endpoints. Marc Buyse International Drug Development Institute (IDDI), Brussels Limburgs Universitair Centrum, Diepenbeek, Belgium marc.buyse@iddi.com FDA Industry Workshop, September 22-23, 2004. Outline. Need for surrogates

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Statistical challenges in the validation of surrogate endpoints' - melinda


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
statistical challenges in the validation of surrogate endpoints

Statistical challenges in the validation of surrogate endpoints

Marc Buyse

International Drug Development Institute (IDDI), Brussels

Limburgs Universitair Centrum, Diepenbeek, Belgium

marc.buyse@iddi.com

FDA Industry Workshop, September 22-23, 2004

outline
Outline
  • Need for surrogates
  • Definitions
  • Validation criteria
    • Single trial
    • Several trials (meta-analysis)
  • Case studies
    • PSA and survival (advanced prostatic cancer)
    • 3-year PFS and 3-year OS (early colorectal cancer)
why do we need surrogates
Why do we need surrogates?
  • Practicality of studies:
    • Shorter duration
    • Smaller sample size (?)
  • Availability of biomarkers:
    • Tissue, cellular, hormonal factors, etc.
    • Imaging techniques
    • Genomics, proteomics, other-ics

Ref: Schatzkin and Gail, Nature Reviews (Cancer) 2001, 3.

validity of a surrogate endpoint
Validity of a surrogate endpoint

Evidence that biomarkers predict clinical effects

  • Epidemiological
  • Pathophysiological
  • Biological
  • Statistical

What are the conditions required to show this?

Ref: Biomarkers Definition Working Group, Clin Pharmacol Ther 2001, 69: 89.

definitions
Definitions
  • Clinical endpoint: a characteristic or variable that reflects how a patient feels, functions, or survives
  • Biomarker: a characteristic that is objectively measured and evaluated as an indicator of normal biological processes, pathogenic processes, or pharmacologic responses to a therapeutic intervention
  • Surrogate endpoint: a biomarker that is intended to substitute for a clinical endpoint. A surrogate endpoint is expected to predict clinical benefit (or harm or lack of benefit or harm)

Ref: Temple, JAMA 1999;282:790.

single trial
Single trial

Parameters of interest

  • effect of treatment on surrogate endpoint ()
  • effect of treatment on true endpoint ()
  • effect of surrogate on true endpoint ()
  • adjusted effect of treatment on true endpoint (S)
  • adjusted effect of surrogate on true endpoint (Z)

Ref: Buyse and Molenberghs, Biometrics 1998;54:1014.

slide7

Surrogateendpoint

Treatment

Trueendpoint

correlation of endpoints is not enough
Correlation of endpoints is not enough

Key point: “A correlate does not a surrogate make”

   0 is not a sufficient condition for validity

Ref: Fleming and DeMets, Ann Intern Med 1996, 125: 605.

a first formal definition and criteria
A first formal definition and criteria

Prentice’s definition

H0S :  = 0  H0T : = 0

Prentice’s criteria

An endpoint can be used as a surrogate if

  • it predicts the final endpoint (  0)
  • it fully captures the effect of treatment upon the final endpoint (  0 and S = 0)

Ref: Prentice, Statist in Med 1989;8:431.

a first formal definition and criteria10
A first formal definition and criteria

Problems with Prentice’s approach

  • rooted in hypothesis testing
  • require significant treatment effects
  • overly stringent
  • criteria not equivalent to definition (except for binary endpoints)
  • one can never prove the null (S = 0)

Ref: Buyse and Molenberghs, Biometrics 1998;54:1014.

the proportion explained
The proportion explained

Freedman’s “proportion explained” is defined as

PE = 1 - S /

  • if S =, PE = 0 and the surrogate explains nothing
  • if S =0, PE = 1 and the surrogate explains the entire effect of treatment on the true endpoint

Ref: Freedman et al, Statist in Med 1989;8:431.

the proportion explained12
The proportion explained

Problems with the proportion explained

  • PE is not a proportion (can be <0 or >1)
  • PE confuses two sources of variability, one at the individual level, the other at the trial level:

PE = Z /

  • PE can be anywhere on the real line, depending on precision of S and T…

Ref: Molenberghs et al, Controlled Clin Trials 2002;23:607.

statistical validation of surrogate endpoints
Statistical validation of surrogate endpoints

“The effect of treatment on a surrogate endpoint must be reasonably likely to predict clinical benefit”

Ref: Biomarkers Definitions Working Group, Clin Pharmacol Ther 2001;69:89.

the relative effect
The relative effect

Interest now focuses on the two components of PE:

  • the surrogate must predict the true endpoint (Z  0)
  • the relative effect, defined as

RE = /

allows prediction of the effect of treatment on the true endpoint () based on the effect of treatment on the surrogate ()

Ref: Buyse and Molenberghs, Biometrics 1998;54:1014.

prediction of true endpoint from surrogate endpoint
Prediction of true endpoint from surrogate endpoint

Endpoints observed on individual patients

R² indicates quality of regression

True Endpoint

Slope = 

Surrogate Endpoint

prediction of treatment effect one trial
Prediction of treatment effect: one trial

Treatment effect observedin the trial

1

.5

Slope = /

0

Treatment Effect onTrue Endpoint ()

Regression through origin; only one point!

-.5

-1

-1

0

1

Treatment Effect on Surrogate Endpoint ()

several trials
Several trials

For a marker to be used as a surrogate, we need “repeated demonstrations of a strong correlation between the marker and the clinical outcome”

Ref: Holland, 9th EUFEPS Conference on “Optimising Drug Development: Use of Biomarkers”, Basel, 2001.

prediction of treatment effect several trials
Prediction of treatment effect: several trials

Treatment effects observedin all trials

1

.5

Slope = /

0

Treatment Effect onTrue Endpoint ()

-.5

R² indicates quality of regression

-1

-1

0

1

Treatment Effect on Surrogate Endpoint ()

validation criteria using several trials
Validation criteria using several trials

Parameters of interest

  • effect of treatment on surrogate endpoint ()
  • effect of treatment on true endpoint ()
  • effect of surrogate on true endpoint ()
  • measure of association between surrogate endpoint and true endpoint (R²individual)
  • measure of association between effects of treatment on surrogate endpoint and on true endpoint (R²trial)

Ref: Buyse et al, Biostatistics 2000;1:49;

Gail et al, Biostatistics 2000;1:231.

technical difficulties the endpoints are not normally distributed
Technical difficulties: the endpoints are not normally distributed

In practice, endpoints are often of the following type : response, survival, longitudinal. Such endpoints are not normally distributed, and therefore complex modelling is required to characterize the association between endpoints (“individual level association”).

At the trial level, however, simple linear models are still adequate to characterize the association between treatment effects on the endpoints (“trial level association”).

Refs:

Molenberghs et al, Stat Med 20:3023, 2001;

Burzykowski et al, J Royal Stat Soc A 50: 405, 2001;

Renard et al, J Applied Statist 30:235, 2002.

a case study in advanced prostatic cancer the trials
A case study in advanced prostatic cancer:the trials
  • Two multicentric trials for patients in relapse after first-line endocrine therapy (596 patients)
  • Unit of analysis for treatment effects: country (19 units)
  • Patients randomized between two treatments:
    • Experimental (retinoic acid metabolism-blocking agent)
    • Control (anti-androgen)

Ref: Buyse et al, in: Biomarkers in Clinical Drug Development (Bloom JC, ed.): Springer-Verlag, 2003.

a case study in advanced prostatic cancer the endpoints
A case study in advanced prostatic cancer:the endpoints

Potential surrogate endpoints:

  • Longitudinal PSA measurements taken at pre-defined time points
  • PSA response (decrease of at least 50%)
  • Time to PSA progression (TPP)

True endpoint:

  • Overall survival
a case study in advanced prostatic cancer
A case study in advanced prostatic cancer

Surrogateendpoint

Treatment

Experimental

Rz

Control

Trueendpoint

psa response as surrogate for survival
PSA response as surrogate for survival

Very weak association between treatment effects

R² = 0.05

ttp as surrogate for survival
TTP as surrogate for survival

Weak association between treatment effects

R² = 0.22

longitudinal psa as surrogate for survival
Longitudinal PSA as surrogate for survival

Moderate association between treatment effects

R²trial = 0.45

individual level and trial level measures of association

Individual-level association between PSA and survival

[95% C.I.]

Trial-level association between treatment effects on PSA and survival [S.E.]

PSA response

Survival odds ratio = 5.5 [2.7 - 8.2]

R²trial = 0.05 [0.13]

Time to PSA progression

Survival odds ratio = 6.3 [4.4 – 8.2]

R²trial = 0.22 [0.18]

Longitudinal PSA

Coefficient of determination R²(t) > 0.84 at all times t

R²trial = 0.45 [0.18]

Individual-level and trial-level measures of association
a case study in early colorectal cancer the trials
A case study in early colorectal cancer:the trials
  • Fifteen collaborative group trials for patients after resection of colorectal tumor (12,915 patients)
  • Unit of analysis for treatment effects: 18 comparisons between 33 treatment arms
  • Patients randomized between various 5-FU regimens and/or control
a case study in early colorectal cancer the endpoints
A case study in early colorectal cancer:the endpoints

Potential surrogate endpoint:

  • 3-year disease-free survival

True endpoint:

  • 5-year overall survival

Ref: Sargent et al, Proceedings ASCO (Abstract # 3502), 2004.

Acknowledgement: the following slides are based on Dr Daniel Sargent’s presentations to ODAC on May 5 and at ASCO on June 6

overview of validation approaches
Overview of validation approaches
  • Single trial
    • full capture (Prentice)
    • proportion explained (Freedman et al)
    • relative effect (Buyse & Molenberghs)
    • likelihood reduction factor (Alonso et al)
  • Several trials (meta-analysis)
    • concordance (Begg & Leung)
    • correlation of effects (Daniels & Hughes)
    • trial-level measures of association (Gail et al)
    • individual- and trial-level measures of association (Buyse et al)
    • predicted treatment effect (Baker)
    • surrogate threshold effect (Burzykowski & Buyse)
conclusions on surrogate validation
Conclusions on surrogate validation
  • Ideally, statistical validation requires the following:
    • data from randomized trials
    • replication at the trial or center level
    • at least some observations of T
    • large numbers of observations
    • range of therapeutic questions (Z1, Z2, …)
  • Hence:
    • individual patient data meta-analyses are needed
    • access to such data is a problem when they are proprietary

Ref: Burzykowski, Molenberghs and Buyse (eds.), “The Evaluation of Surrogate Endpoints”, Springer-Verlag (in press).