alex dmitrienko ph d eli lilly and company fda industry statistics workshop september 2004 l.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Gatekeeping Testing Strategies in Clinical Trials PowerPoint Presentation
Download Presentation
Gatekeeping Testing Strategies in Clinical Trials

Loading in 2 Seconds...

play fullscreen
1 / 27

Gatekeeping Testing Strategies in Clinical Trials - PowerPoint PPT Presentation


  • 342 Views
  • Uploaded on

Alex Dmitrienko, Ph.D. Eli Lilly and Company FDA/Industry Statistics Workshop September 2004. Gatekeeping Testing Strategies in Clinical Trials. Outline. Gatekeeping strategies

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Gatekeeping Testing Strategies in Clinical Trials' - paiva


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
alex dmitrienko ph d eli lilly and company fda industry statistics workshop september 2004
Alex Dmitrienko, Ph.D.

Eli Lilly and Company

FDA/Industry Statistics Workshop

September 2004

Gatekeeping Testing Strategies in Clinical Trials

outline
Outline
  • Gatekeeping strategies
    • account for the hierarchical structure of multiple analyses (comparisons) and preserve the overall false positive rate
  • Clinical trial examples
    • registration trials with multiple primary and secondary endpoints (trials in patients with depression and acute lung injury)
    • dose-finding trials (trial in patients with hypertension)
primary versus secondary findings
Primary versus secondary findings
  • Findings with respect to secondary endpoints provide much useful information
    • useful to prescribing physicians and patients
  • FDA guidance “Clinical studies section of labeling for prescription drugs and biologics”
    • “The CLINICAL STUDIES section should present those endpoints that are essential to establishing the effectiveness of the drug (or that show the limitations of effectiveness) and those that provide additional useful and valid information about the activities of the drug”
primary versus secondary findings4
Primary versus secondary findings
  • Dilemma
    • regulatory agencies and pharmaceutical companies have long debated what secondary findings should be included in the product label
    • regulatory agencies are concerned that pharmaceutical companies tend to present favorable data and ignore unfavorable data
  • Gatekeeper strategies offer one potential solution to the dilemma
types of gatekeeping strategies
Types of gatekeeping strategies
  • Two types of gatekeeping strategies
    • sequential strategies have been used for 10 years
    • parallel strategies were introduced by Dmitrienko, Offen and Westfall (2003)
  • Introduce general gatekeeping framework
    • based on the closed testing principle (Marcus et al, 1976)
    • focus on strategies derived using Bonferroni’s test
    • easily extended to more powerful tests that account for the correlation among the endpoints (Dunnett’s test, resampling tests)
example 1 one primary endpoint
Example 1: One primary endpoint
  • Depression trial
    • Experimental drug is compared to placebo
    • Single primary endpoint
      • 17-item Hamilton depression rating scale (HAMD 17 score)
    • Trial is declared successful if the drug is superior to placebo
    • Two important secondary endpoints
      • response rate based on the HAMD 17 score
      • remission rate based on the HAMD 17 score
    • Can the secondary findings be included in the product label?
sequential gatekeeping strategy
Sequential gatekeeping strategy
  • Step 1: Perform the primary analysis
  • Step 2: Perform the secondary analyses with an adjustment for multiplicity if the primary analysis yielded a significant result

Family 1 (gatekeeper)

Family 2

A1: HAMD17

A2: Response rate

A3: Remission rate

sequential gatekeeping strategy8
Sequential gatekeeping strategy
  • Primary analysis: No adjustment for multiplicitySecondary analyses: Stepwise Holm’s test
  • All primary and secondary findings are significant at 5% level
example 2 multiple primary endpoints
Example 2: Multiple primary endpoints
  • Clinical trial in patients with acute lung injury
    • Experimental drug is compared to placebo
    • Two primary endpoints
      • number of days patients are off mechanical ventilation (vent-free days)
      • 28-day all-cause mortality rate
    • Trial is declared successful if the drug is superior to placebo with respect to either endpoint
    • Two important secondary endpoints
      • number of days patients are out of ICU (ICU-free days)
      • overall quality of life at the end of the study
    • Can the secondary findings be included in the product label?
parallel gatekeeping strategy
Parallel gatekeeping strategy

Family 1 (gatekeeper)

Family 2

  • Step 1: Perform the primary analyses with an adjustment for multiplicity
  • Step 2: Perform the secondary analyses with an adjustment for multiplicity if at least one primary analysis yielded a significant result

A1: Vent-free days

A3: ICU-free days

A4: Quality of life

A2: Mortality

parallel gatekeeping strategy11
Parallel gatekeeping strategy

Family 1 (gatekeeper)

Family 2

  • Step 1: Perform the primary analyses with an adjustment for multiplicity
  • Step 2: Perform the secondary analyses with an adjustment for multiplicity if at least one primary analysis yielded a significant result

A1: Vent-free days

A3: ICU-free days

A4: Quality of life

A2: Mortality

parallel gatekeeping strategy12
Parallel gatekeeping strategy

Family 1 (gatekeeper)

Family 2

  • Step 1: Perform the primary analyses with an adjustment for multiplicity
  • Step 2: Perform the secondary analyses with an adjustment for multiplicity if at least one primary analysis yielded a significant result

A1: Vent-free days

A3: ICU-free days

A4: Quality of life

A2: Mortality

statistical details
Statistical details
  • Sequential families of null hypotheses
    • Null hypotheses H11, H12, H21, H22 grouped into two families F1={H11, H12} and F2 ={H21, H22}
  • Parallel gatekeeping testing procedure
    • Overall Type I error rate is no greater than  (e.g., 0.05)
    • The null hypotheses in F2 can be tested if at least one hypothesis in F1 has been rejected [parallel testing]
    • Adjusted p-values for the hypotheses in F1 do not depend on the p-values associated with the hypotheses in F2 [primary analyses cannot depend on secondary analyses]
closed testing
Closed testing
  • Closed family of hypotheses
    • Consider all 15 intersections of the null hypotheses
      • {H11or H12 or H21or H22}, {H11or H12 or H21}, etc
    • Specify a test for each intersection hypothesis that controls the Type I error rate, e.g., Bonferroni test
    • Establish implication relationships
      • Intersection hypothesis {H11or H12} implies H11 and H12, etc
    • Tests for original hypotheses
      • Reject a null hypothesis if all intersection hypotheses implying it have been rejected
parallel gatekeeping strategy16
Parallel gatekeeping strategy
  • Two scenarios
    • significant improvement in the mean number of ventilator-free days and 28-day all-cause mortality
    • significant improvement in 28-day all-cause mortality but not in mean number of ventilator-free day
  • Weighted analyses
    • primary endpoints are unequally weighted to reflect their relative importance (likelihood of success)
      • weight=0.9 for number of ventilator-free days
      • weight=0.1 for 28-day all-cause mortality
parallel gatekeeping strategy17
Parallel gatekeeping strategy
  • Scenario 1
    • significant improvement in the mean number of ventilator-free days and 28-day all-cause mortality
  • Primary analyses: Bonferroni’s testSecondary analyses: Stepwise Holm’s test
  • All primary and secondary findings are significant at 5% level
parallel gatekeeping strategy18
Parallel gatekeeping strategy
  • Scenario 2
    • significant improvement in 28-day all-cause mortality but not in mean number of ventilator-free day
  • Primary analyses: Bonferroni’s testSecondary analyses: Stepwise Holm’s test
  • The primary mortality analysis and secondary quality of life analysis are significant at 5% level
example 3 dose finding study
Example 3: Dose-finding study
  • Clinical trial in patients with hypertension
    • Four doses of an experimental drug are compared to placebo
      • doses are labeled as D1, D2, D3 and D4
    • Primary endpoint
      • reduction in diastolic blood pressure
    • Objectives of the study
      • find the doses with a significant reduction in diastolic blood pressure compared to placebo
      • study the shape of the dose-response curve
example 3 dose finding study20
Example 3: Dose-finding study

Family 1 (gatekeeper)

Family 2 (gatekeeper)

Family 3

  • Step 1: Compare doses D3 and D4 to placebo
  • Step 2: Compare doses D1 and D3 to placebo if at least one comparison at Step 1 is significant
  • Step 3: Perform various pairwise dose comparisonsif at least one comparison at Step 2 is significant

A1: D4 vs. P

A3: D2 vs. P

Pairwise comparisons

A2: D3 vs. P

A4: D1 vs. P

example 3 dose finding study21
Example 3: Dose-finding study

Family 1 (gatekeeper)

Family 2 (gatekeeper)

Family 3

  • Step 1: Compare doses D3 and D4 to placebo
  • Step 2: Compare doses D1 and D3 to placebo if at least one comparison at Step 1 is significant
  • Step 3: Perform various pairwise dose comparisonsif at least one comparison at Step 2 is significant

A1: D4 vs. P

A3: D2 vs. P

Pairwise comparisons

A2: D3 vs. P

A4: D1 vs. P

example 3 dose finding study22
Example 3: Dose-finding study

Family 1 (gatekeeper)

Family 2 (gatekeeper)

Family 3

  • Step 1: Compare doses D3 and D4 to placebo
  • Step 2: Compare doses D1 and D3 to placebo if at least one comparison at Step 1 is significant
  • Step 3: Perform various pairwise dose comparisonsif at least one comparison at Step 2 is significant

A1: D4 vs. P

A3: D2 vs. P

Pairwise comparisons

A2: D3 vs. P

A4: D1 vs. P

parallel gatekeeping strategy24
Parallel gatekeeping strategy
  • Doses D2, D3 and D4 are significantly different from placebo at 5% level
summary
Summary
  • Gatekeeping strategies can be successfully used in
    • pivotal trials with multiple primary and secondary endpoints
    • dose-finding studies
  • Registration trials
    • a priori designation of gatekeeping strategy allows additional data useful to physician and patient to be presented in the product label
  • Dose-finding studies
    • efficient tests of dose-response relationship
extensions
Extensions
  • More powerful gatekeeping tests
    • based on more powerful tests, e.g., Simes test
    • based on tests accounting for the correlation among the endpoints (exact parametric tests such as Dunnett’s test and approximate resampling-based Westfall-Young tests)
  • Software implementation
    • SAS programs for gatekeeping tests can be found in Dmitrienko, Molenberghs, Chuang-Stein, Offen. (2004). Analysis of Clinical Trials: A Practical Guide. SAS Publishing, Cary, NC.
references
References
  • Papers
    • Dmitrienko, Offen, Westfall. (2003). Gatekeeping strategies for clinical trials that do not require all primary effects to be significant. Statistics in Medicine. 22, 2387-2400.
    • Marcus R, Peritz E, Gabriel KR. (1976). On closed testing procedure with special reference to ordered analysis of variance. Biometrika. 63, 655-660.
    • Westfall, Krishen. (2001). Optimally weighted, fixed sequence, and gatekeeping multiple testing procedures. Journal of Statistical Planning and Inference. 99, 25-40.