1 / 11

# - PowerPoint PPT Presentation

Point/Counterpoint – Single Phase 3 trial. POINT: A single phase 3 trial is often insufficient Brian Smith, Amgen, Inc. Disclaimer. The views expressed herein represent personal views and do not necessarily represent the views or practices of Amgen. On “significance” and replication.

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.

## PowerPoint Slideshow about '' - penda

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

### Point/Counterpoint – Single Phase 3 trial

POINT: A single phase 3 trial is often insufficient

Brian Smith, Amgen, Inc.

• The views expressed herein represent personal views and do not necessarily represent the views or practices of Amgen.

If one in twenty does not seem high enough odds, we may, if we prefer it, draw the line at one in fifty (the 2 per cent. point), or one in a hundred (the 1 per cent. point). Personally, the writer prefers to set a low standard of significance at the 5 per cent. point, and ignore entirely all results which fail to reach this level. A scientific fact should be regarded as experimentally established only if a properly designed experiment rarely fails to give this level of significance.

— Sir Ronald Aylmer Fisher

'The Arrangement of Field Experiments', The Journal of the Ministry of Agriculture, 1926, 33, 504.

How Likely Would an Ineffective Treatment Make it to Market Given 2 Phase 3, p-value < 0.05 trials?

• Remember

• Type 1 error (one sided) – Probability that a treatment which does not work will appear to work after the experiment is completed

• This probability is typically 2.5% for a Phase 3 trial

• Power – For a treatment that has a particular true effect x, the probability that the treatment will appear to work after the experiment is completed

• This probability is typically 90% for a Phase 3 trial

How Likely Would an Ineffective Treatment Make it to Market Given 2 Phase 3, p-value < 0.05 trials?

• Simplifying assumptions for illustration

• The treatment either does not work or

• The treatment has true effect x

• Two phase 3 studies are to be done in sequence

• Historically only 15% of compounds for this therapeutic area are effective

• After the first Phase 3 trial, you have success.

• what is the probability that the treatment is effective?

• 0.15 * 0.9 / (0.15 * 0.9 + 0.85 * 0.025) = 0.864

How Likely Would an Ineffective Treatment Make it to Market Given 2 Phase 3, p-value < 0.05 trials?

• After the second Phase 3 trial, you have success.

• what is the probability that the treatment is effective?

• 0.864 * 0.9 / (0.864 * 0.9 + 0.136 * 0.025) = 0.996

• This is just a simple application of Bayes Theorem

• Suppose you just wanted to do a single trial and if positive you wanted to be equally confident that the treatment worked, what would α be?

• 0.15 * 0.9 / (0.15 * 0.9 + 0.85 * α) = 0.996

• α = 0.00069

Some other Priors Given 2 Phase 3, p-value < 0.05 trials?

What do we Learn? Given 2 Phase 3, p-value < 0.05 trials?

• Positive Results (p-value < 0.025) from 1 trial is not nearly enough evidence of a positive effect

• You could reduce α and increase sample size if want single trial

If you wanted to calculate this probability Given 2 Phase 3, p-value < 0.05 trials?

• Use full Bayesian approach

• Prior distribution for first Phase 3 study

• After trial find posterior distribution

• Use this as prior distribution for second Phase 3 study

• After trial find posterior

• Calculate probability of treatment difference

• Thus, the calculations that I performed are more for illustration purposes

What Else Do We Know? Given 2 Phase 3, p-value < 0.05 trials?

• Phase 3 is equally about safety endpoints

• Really, don’t you just feel better when results replicate?

Conclusions Given 2 Phase 3, p-value < 0.05 trials?

• Replication is an under appreciated concept in evidence generation

• Do you have to have 2 trials?

• No, but you should generate the same amount of evidence with one

• Bayesian perspective really helps make sense of these sort of questions.