statistical issues in clinical studies using laboratory endpoints n.
Skip this Video
Loading SlideShow in 5 Seconds..
Statistical Issues in Clinical Studies Using Laboratory Endpoints PowerPoint Presentation
Download Presentation
Statistical Issues in Clinical Studies Using Laboratory Endpoints

Loading in 2 Seconds...

play fullscreen
1 / 29

Statistical Issues in Clinical Studies Using Laboratory Endpoints - PowerPoint PPT Presentation

  • Uploaded on

Statistical Issues in Clinical Studies Using Laboratory Endpoints. ACOSOG June 5, 2003 Montreal. Elizabeth S. Garrett, PhD Sidney Kimmel Comprehensive Cancer Center Johns Hopkins University. Outline. Motivating example: Endostatin trial Surrogate markers

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about 'Statistical Issues in Clinical Studies Using Laboratory Endpoints' - evangelina

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
statistical issues in clinical studies using laboratory endpoints

Statistical Issues in Clinical Studies UsingLaboratory Endpoints


June 5, 2003


Elizabeth S. Garrett, PhD

Sidney Kimmel Comprehensive Cancer Center

Johns Hopkins University

  • Motivating example: Endostatin trial
  • Surrogate markers
  • Design issues in laboratory studies
  • Defining outcomes
  • Issues relating to measurement
motivating example dose finding endostatin trial
Motivating Example: Dose Finding Endostatin Trial*
  • Endostatin is angiogenesis inhibitor.
  • Complex biology: finding best dose will require extensive study.
  • Cytostatic versus cytotoxic agent.
  • Outcomes of interest are biomarkers:
    • Tumor blood flow
    • Tumor metabolism
  • Outcomes were measured at baseline, 28 days and 56 days.

* Herbst, Abbruzzese et al. Development

of Biologic Markers of Response and

Assessment of Antiangiogenic Activity

in Clinical Trial of Human Recombinant

Endostatin. JCO v. 20, 2002.

motivating example dose finding endostatin trial1
Motivating Example: Dose Finding Endostatin Trial*
  • Why is this study different than other dose finding studies?
    • Standard Phase I trials find MTD (maximum tolerated dose) with safety as primary outcome.
    • Primary outcome is ‘efficacy-related’.
    • Outcomes are ‘surrogate’ outcomes.
    • Measuring outcomes is more invasive and more costly than standard safety or efficacy trials.
    • Measurement of outcomes can be complicated.
surrogate markers
Surrogate Markers
  • If tumor blood flow and tumor metabolism change as we expect, what can we conclude?
  • Is it true, then, that endostatin is efficacious?
  • Not necessarily… closely tied are these markers and clinical response?
  • Surrogate outcomes: outcomes in the causal pathway of true outcome.
surrogate markers1
Surrogate Markers
  • Replace a distal endpoint (response) by proxy endpoint (tumor metabolism and/or blood flow).
  • Benefits of using surrogate markers
    • Reduction in sample size
    • Reduction in trial duration
    • Reduction in cost
    • Reduction in time to evaluate new therapies
  • Their use is NOT AS EASY AS IT SOUNDS…
  • Use of a marker as surrogate for outcome requires that you first identify one...
what is a surrogate marker
What is a surrogate marker?
  • Defining Characteristic:
    • a marker must predict clinical outcome, in addition to predicting the effect of treatment on clinical outcome
  • Operational Definition
    • establish an association between marker & clinical outcome
    • establish an association between marker, treatment & clinical outcome, in which marker mediates relationship between clinical outcome and treatment
surrogate markers2
Surrogate Markers

[ Clinical Outcome | Treatment & Marker]


[ Clinical Outcome | Marker]

Effect of the treatment on the clinical outcome is completely explained by the effect of the marker

Knowing the status of the marker, the

treatment adds nothing more to explaining

the clinical outcome.

surrogate markers3
Surrogate Markers

1) establish an association between marker & clinical outcome.




2) establish an association between marker, treatment &

clinical outcome, in which marker completely mediates

relationship between clinical outcome and treatment.





not surrogate markers
NOT Surrogate Markers









correlative studies
Correlative Studies
  • It is still valid to look at markers that you would expect to be correlated with the clinical outcome.
  • But, we do not want to be overconfident by saying that they are true ‘surrogates.’
  • Correlative studies might include:
    • Pharmacokinetics
    • Pharmacodynamics
    • Other biologic markers that can be measured in serum, biopsy samples, etc.





design issues
Design Issues
  • Novel laboratory studies may not be suited for standard phase I and phase II clinical trials.
standard phase 1 clinical trial designs
Standard Phase 1 Clinical Trial Designs
  • Historically, MTD is of interest
  • ‘3+3’: aims for dose with <33% toxicity
    • Treat 3 patients at a dose.
    • If no toxicities, escalate.
    • If 1 toxicity, treat 3 more at that level.
    • If 2 or more, de-escalate to lower dose.
  • ‘3+3’ or variants are most commonly seen.
  • Not relevant to cytostatic and other agents (e.g. vaccine trials).
  • Need different approach.
alternative designs
Alternative Designs
  • Continual Reassessment Method (CRM)
    • ‘Response’ can be defined as toxicity or efficacy endpoint.
    • Target level of response is chosen.
    • Mathematical model for dose-response is assumed.
    • Dose levels are not fixed in advance: later doses are determined by responses to previous doses.
    • Many variants, but generally need to pre-specify a minimum and maximum dose.
    • Problem: we might not know what our ‘target’ response is.
alternative designs1
Alternative Designs
  • Dose-ranging study
    • Choose fixed doses and treat a fixed number of patients at each of the doses.
    • Allows estimation of dose-response curve.
    • Choice of Herbst, et al. for endostatin trial.
    • Can be used when safety is not of great concern (i.e. probability of toxicities is small).
  • For both CRM and dose-ranging, we tend to get better estimates of dose-response curves.
  • Toxicity almost always gets included as (at least) a secondary outcome.
selecting a phase i design
Selecting a Phase I Design
  • Don’t assume the standard ‘phase 1’ trial designs are appropriate for assessing laboratory endpoints.
  • Dose-ranging studies are more likely candidates.
  • Treat enough patients at each dose level to get ‘reasonable’ estimate of response (i.e. at least 3).
  • Think carefully about dose levels….
    • Want enough dose levels to understand/estimate the curve
    • Don’t want so many that it will take you a long time to finish the study.
  • Patient population
    • Endostatin trial: broad patient population
    • 3 NSCLC, 8 melanoma, 2 HNSCC, 3 breast, 2 colon, 2 renal cell, 4 sarcoma, 1 thyroid.
    • Generalizability versus accrual versus specificity
choosing dose levels
Choosing dose levels
  • Can be hard to know exactly how to select doses.
  • If there is too much ‘space’ between doses, it can be difficult to estimate true curve.
  • If you expect, ‘monotonicity’, then you can infer intermediate doses.
  • But, what if not monotonic? Endostatin example.
choosing the timing
Choosing the timing
  • Standard Phase I
    • Patients are under surveillance for short term toxicities.
    • Patients contact study team when a toxicity occurs after discharge.
    • Not an issue of ‘when to look.’
  • In laboratory studies
    • Usually a pre-post setting: how does the measure compare after treatment to before treatment.
    • Baseline (before treatment) measure is needed.
    • Post-treatment measures:
      • When is treatment at its most potent?
      • How often should response be measured?
      • Expensive? Invasive?
      • Is it sufficient to look at clinic visit times?
phase ii designs
Phase II Designs
  • Phase II designs: efficacy
  • Standard efficacy outcomes
    • Complete response
    • Partial response
    • Overall survival
  • For cytostatic agents
    • Progression
    • Progression-free survival
    • Laboratory outcomes…
defining outcomes in laboratory studies
Defining outcomes in laboratory studies
  • They are usually messy
  • More common binary outcomes have nice properties:
    • “Looking for 40% response vs. 20% response”
  • Laboratory outcomes are not so nice:
    • Often skewed.
    • Often have ‘undetectable’ range.
    • Often do not know what to expect.
    • This makes it hard to plan (i.e. sample size, power).
  • Novel assays: Not obvious what expected changes would be without treatment.
    • How much fluctuation would we expect to see?
expected fluctuations
Expected Fluctuations

But, with multiple

patients, these

changes would

tend to average to


However, if half

of patients have

increase and half

have decrease, we

might conclude that

treatment is 50%





expected fluctuations1
Expected Fluctuations
  • Follow-up times are compared to baseline
    • That puts a lot of stock in baseline measure
    • Why not consider a ‘burn-in’ period?
  • If baseline is inaccurately measured, all comparisons will be incorrect.
measurement issues
Measurement Issues


  • Regardless of natural fluctuations…
  • How accurately are we measuring our outcome?
  • How accurate can we measure blood flow?
  • Are we on target?
measurement issues1
Measurement Issues
  • Why would it be measured inaccurately?
    • Often LONG protocol to get ‘final’ measure.
    • Lots of room for errors!
  • Although the classic efficacy outcome of ‘response’ is relatively soft, it has been objectively defined.
  • Laboratory endpoints require assays and other measurements, and also assumptions.
  • Often, assay is being developed along with the trial.
assay properties
Assay Properties
  • Sensitivity: does the assay detect abnormalities in cases where abnormalities exist?
  • Specificity: does the assay find normal levels in normal cases?
  • “It has been shown that at high flow rates, measuring blood flow by PET underestimates blood flow.”

bias in results.

more measurement issues
More Measurement Issues
  • Reliability of assay
    • How reproducible are the results?
      • Two samples taken from the same patient on the same day from different lesions?
      • Two samples taken from the same patient on the same day from the same lesion?
      • One sample analyzed twice using the same method?
    • Subjectivity
    • Inter-rater agreement
    • Intra-rater agreement
    • In what ways can ‘error’ come into the procedure?

Great study + bad assay = bad study

measurement issues2
Measurement Issues
  • How can we know the reliability of the measures?
  • Preliminary studies (pre-clinical).
  • Build it into the study design!
    • Reliability substudy:
      • Inter-rater agreement
      • Intra-rater agreement
    • Incorporate burn-in period
    • Take multiple measures
    • Run the assay (test) more than once
final comments
Final Comments
  • Novel trials often need creative/novel study designs
  • Understanding the properties (i.e. reliability, sensitivity) of your assay are crucial.
  • Try to build reliability testing into study design
  • Think carefully about:
    • Are your markers are truly surrogates?
    • Which doses you should choose (minimum, maximum, and increments)?
    • How often you should measure the outcome?

Statisticians usually can provide guidance, but not necessarily

answers to these questions!

Answers need to be based on experience with the novel agent and

the underlying biology.