controversy what controversy an attempt to structure the debate on adaptive designs l.
Skip this Video
Loading SlideShow in 5 Seconds..
Controversy? What controversy? An attempt to structure the debate on adaptive designs PowerPoint Presentation
Download Presentation
Controversy? What controversy? An attempt to structure the debate on adaptive designs

Loading in 2 Seconds...

play fullscreen
1 / 20

Controversy? What controversy? An attempt to structure the debate on adaptive designs - PowerPoint PPT Presentation

  • Uploaded on

Controversy? What controversy? An attempt to structure the debate on adaptive designs. Marc Vandemeulebroecke MCP, Vienna, 11.7.2007. Vandemeulebroecke Controversy? What controversy? 1. Examples (1): Sequential analysis.

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about 'Controversy? What controversy? An attempt to structure the debate on adaptive designs' - edalene

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
controversy what controversy an attempt to structure the debate on adaptive designs

Controversy? What controversy?An attempt to structure the debate on adaptive designs

Marc Vandemeulebroecke

MCP, Vienna, 11.7.2007


Examples (1): Sequential analysis

A. Wald (1945): „The National Defense Research Committee considered these developments sufficiently useful for the war effort to make it desirable to keep the results out of the reach of the enemy.“

F. J. Anscombe (1963): „‘Sequential analysis‘ is a hoax.“

P. Armitage (1989): „If [...] ‘sequential analysis is a hoax‘, then its proponents are amongst the most persuasive swindlers, and their clients amongst the most gullible consumers, in the history of scientific endeavour.“

Vandemeulebroecke Controversy? What controversy?



Repeated confidence intervals (C. Jennison, B. Turnbull 1989)

such that for all

Example (2): Group sequential designs

P. Armitage: „ingenious derivation“

S. J. Pocock: „a minor change of presentation style“

Vandemeulebroecke Controversy? What controversy?



Test H0: with ; interim:

Change to observe 

Example (3): Adaptive/flexible designs

C.-F. Burman, C. Sonesson (2006): Are flexible designs sound?

„The weighted test violates inference principles [...] and may lead to unreasonable results.“

M. Proschan: „Such methods, if extremely abused, produce illogical conclusions, but that is no more a condemnation of adaptive methods than Jack the Ripper is a condemnation of cutlery.“

P. Bauer: „Is wine sound? It is the way to drink it that matters.“

Vandemeulebroecke Controversy? What controversy?



Example (4): Terminology

A. J. Phillips, O. N. Keene (2006): „The terms ‘flexible design’ and ‘adaptive design’ are used interchangeably.”

C. Jennison, B. Turnbull (2006): „The adjectives ‘flexible’ and ‘adaptive’ are not synonymous. Indeed adaptive designs can range from being totally inflexible to very flexible.”

Vandemeulebroecke Controversy? What controversy?




  • Imagine a study protocol:
  • Double-blind multicenter trial in Crohn‘s Disease
  • Active vs placebo 1:1
  • Endpoint: CDAI after 4 weeks
  • IA at half of the sample size, switch to 2:1 if encouraging
  • Trial conduct:
  • IA disappointing, but new budget available
  •  Continue 1:1, ad hoc increase of sample size
  • Final analysis correctly adjusted and significant
  • In stage 2, patients are more severe, and placebo is doing
  • much worse

Vandemeulebroecke Controversy? What controversy?



Issue 1: Feasibility

  • Practical, logistical or operational issues regarding the conduct of the study
  • Speed of recruitment vs. speed of response
  • Availability of specialized software
  • „Supply chain management“: Data capture/cleaning/ processing; real-time randomization and drug supply
  • Training/SOPs/“charter“
  • Investigate behavior of procedure by simulation, play through scenarios with decision makers / IDMC

Vandemeulebroecke Controversy? What controversy?



Issue 2: Validity

  • Statistical/mathematical property.
  • Correct type I error
  • Point estimates available and sensible (e.g., mean/median unbiased)
  • Confidence intervals available, sensible and correct

Vandemeulebroecke Controversy? What controversy?



Issue 3: Integrity

  • Hard to quantify. Concern: „operational bias“
  • Consistency/homogeneity across trial stages (A. Koch: identification problem)  MCP: T. Friede, A. Faldum...
  • Jeopardizing integrity means jeopardizing credibility.
  • Measures to maintain trial integrity:
    • avoid unblinding if possible
    • control dissemination of results
    • control scope and amount of adaptations

Vandemeulebroecke Controversy? What controversy?



Issues 2/3: Validity/Integrity

V. Dragalin (2006, PhRMA):

„To maintain study validity means providing correct statistical inference [...], ensuring consistency between different stages of the study, and minimizing operational bias.

To maintain study integrity means providing convincing results to a broader scientific community; preplanning, as much as possible, based on intended adaptations; and maintaining the blind of interim analysis results.“

H. Schäfer (2006): internal validity - external validity

Vandemeulebroecke Controversy? What controversy?



Issue 4: Interpretability

  • Only interpretable results are communicable.
  • Intregrity and interpretability have the main impact of the persuasiveness of the study results.
  • Endpoints meaningful
  • Point estimates and confidence intervals available and sensible
  • Jeopardizing integrity (e.g., across-stage consistency or amount of adaptations) will also reduce interpretability.

Vandemeulebroecke Controversy? What controversy?



Issue 5: Efficiency

  • Often, adaptation comes at the cost of efficiency.
  • terms of power / expected information
  • Reason: non-sufficient statistics
  • Any adaptive design can be outperformed by a nonadaptive design - and vice versa!
  • „Statistical efficiency is not always the priority“ (Koch 2006)
  • „Response-dependent choice of group sizes can itself be a source of improved efficiency“ (Schmitz 1993, Jennison/Turnbull 2006)

Vandemeulebroecke Controversy? What controversy?



Nature of change

Prespeficied rule

Completely free

Not dep. on trial-internal data


Adaptable features

+ stage sizes

+ other

Issue 6: Flexibility

  • Two main questions:
  • Should there be an a priori selection of study features that may be adapted? If so, now many and which?
  • Should the design changes follow prespecified rules or may they be done ad hoc? Should they depend on information internal or external to the trial?

Group sequential

„Schmitz designs“

Spending functions

p-value combinations

Vandemeulebroecke Controversy? What controversy?




  • Issues
  • Feasibility: „Only“ operational issues
  • Validity: „Only“ mathematical issues
  • Integrity: Requirements?
  • „Operational bias“? Homogeneity across stages?
  • Interpret‘y: Requirements?
  • Efficiency: Weigh against flexibility (sponsor‘s choice)
  • Flexibility:

A priori selection of adaptable features? How many/which?

Adaptations ad hoc or following prespecified rules? Depending on internal or external information ?

Vandemeulebroecke Controversy? What controversy?



Some references

Biom J 48 (2006, special issue), in particular: O‘Neil, Hung et al., Koch, and discussants: Section on „Regulatory perspectives“, pp. 559-622

Burmann, Sonesson (2006):Are flexible designs sound? Biometrics 62, pp. 664-683 (with discussion)

EMEA (2006):Reflection paper on methodological issues in confirmatory clinical trials with flexible design and analysis plan. CHMP/EWP/2459/02

Fleming, DeMets (1993): Monitoring of clinical trials: Issues and Recommendations. Contr Clin Trials 14, pp. 183-197

Gallo et al. (PhRMA, 2006):Adaptive designs in clinical drug development - an executive summary of the PhRMA working group. J Biopharm Stat 16, pp. 275-283 (with discussion)

Jennison, Turnbull (2006):Adaptive and nonadaptive group sequential tests. Biometrika 93, pp. 1-21

Phillips, Keene (PSI, 2006):Adaptive designs for pivotal trials: discussion points from the PSI adaptive design expert group. Pharm Stat 5, pp. 61-66

PhRMA (2006):Full white paper. DIJ 40, pp. 421-484

Posch, Bauer, Brannath (2003):Issues in designing flexible trials. Stat in Med 22, pp. 953-969

Stat in Med 25 (2006, special issue), pp. 3229-3408

...and many thanks to Norbert Benda!

Vandemeulebroecke Controversy? What controversy?



V. Dragalin (2006): “a multistage study design that uses accumulating data to decide how to modify aspects of the study without undermining the validity and integrity of the trial. [...] preplanning, as much as possible, based on intended adaptations.”

Appendix: Terminology

PhRMA: Definition of “adaptive design”

P. Gallo et al (2006): “a clinical study design that uses accumu-lating data to decide how to modify aspects of the study as it continues, without undermining the validity and integrity of the trial. [...] In such trials, changes are made ‘by design’, and not on an ad hoc basis.”

Vandemeulebroecke Controversy? What controversy?



Novartis slide deck: “Adaptive designs allow for initial uncertainties in trial design to be confirmed/adapted during the trial. The integrity of the trial is maintained and the evidence for the same hypothesis before and after the adaptation is combined.”


Other definitions of “adaptive design”

EMEA (2006): “A study is called ‘adaptive’ if statistical methodology allows the modification of a design element [...] at an interim analysis with full control of the type I error.”

Vandemeulebroecke Controversy? What controversy?




  • Other terms
  • flexible
  • adaptive combination
  • flexible group sequential
  • flexible adaptive
  • adaptive group sequential
  • internal pilot
  • self designing
  • seamless adaptive
  • design automates
  • ....

(List stolen from J. Röhmel‘s talk at BfArM Discussion on Adaptive Designs, 2006)

Vandemeulebroecke Controversy? What controversy?




Jennison, Turnbull (2006): „The strength of adaptive redesign lies in coping with the unexpected, in particular responding to external information that could not have been anticipated at the start of a study.“

M. Frisén (2006): „One should use planned adaptive designs when one expects that it will be necessary to adapt the design. If one unexpectedly has strong reasons to change the plans, one should be very careful.“

P. Gallo et al. (2006): „Should or should it not be viewed that adaptive designs might remedy inadequate planning?“

Vandemeulebroecke Controversy? What controversy?