Demonstrating the impact of statistics in the preclinical area using Bayesian analysis with informat...
This presentation is the property of its rightful owner.
Sponsored Links
1 / 20

8 th -10 th October 2014 PowerPoint PPT Presentation

  • Uploaded on
  • Presentation posted in: General

Demonstrating the impact of statistics in the preclinical area using Bayesian analysis with informative priors. 8 th -10 th October 2014. Non Clinical Statistics Conference. Ros Walley, John Sherington, Joe Rastrick, Gill Watt. Outline. Strategy to demonstrate impact

Download Presentation

8 th -10 th October 2014

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript

8 th 10 th october 2014

Demonstrating the impact of statistics in the preclinical area using Bayesian analysis with informative priors

  • 8th-10th October 2014

  • Non Clinical Statistics Conference

Ros Walley, John Sherington, Joe Rastrick, Gill Watt



  • Strategy to demonstrate impact

    • Features of Bayesian designs

    • Selling points

  • Preclinical setting

    • Contrast with Clinical

    • Related issues

  • Case study

  • First steps

  • Types of control groups

  • Bayesian methodology

  • Summary to date

Strategy to demonstrate impact

Strategy to demonstrate impact

Features of Bayesian designs

  • Explicit way of combining information sources

  • Forces early agreement as to relevance of all information sources

  • Reduce costs and resources (animal numbers) through informative priors/predictive distributions

  • Reduce costs and resources through interim analysis

  • Allows more relevant statements to be made at the end of the study e.g. the probability the response rate for drug A is more than 10% better than drug B

    • Ranking compounds

    • Comparing a combination with its components

  • Flexibility in estimation. E.g. one can analyse on the log scale and estimate differences on the linear scale

  • Allows for a wide variety of models to be fitted and can address issues such as lack of convergence or outlier-prone data

  • Model averaging, model selection

Strategy to demonstrate impact1

Strategy to demonstrate impact

“Selling points of Bayesian methods”

  • High impact

    • Focus on in vivo studies that are run again and again

    • Saving even a few animals per study results in large savings, easily demonstrated

  • Ground-breaking

    • Quick search in the literature suggestslittle use in vivo except some focused applications:

      • PK & PK/PD models

      • SNPS/genes – pathway analysis. Including a Nature reviews article called “The Bayesian revolution in genetics”

      • Lookup proteins

  • Complements clinical strategy

  • *

Preclinical setting

Preclinical setting

Contrast with clinical

Preclinical setting1

Preclinical setting

Related issues to consider

Different output: internally & externally

Getting hold of the data; in the right format

Bayesian designs for repeated studies

Rely entirely on historic data for one group

Allied designs: modelled approach, QC charts

Supplement a group with prior info  analysis issues

Case study

Case study


  • Mouse model to study inflammatory response after a challenge.

  • Measures a number of cytokines at 2.5 or 3hrs.

  • Includes test compounds + three control treatments:

    • Negative group (no challenge).

    • ‘Positive’ group (challenge, no treatment).

    • Comparator with known efficacy

  • Main comparisons of interest:

    • Test compound vs. +ve group (untreated) – Does test compound reduce the response?

  • Data on 19 studies available (reasonably consistent protocol).

  • Most frequent in-vivo assay in this therapeutic area.

  • Data typically presented an experiment at a time, in PRISM

First steps

First steps

Strategy to smooth the introduction of Bayesian methods

  • QC charts to demonstrate reproducibility over time

  • Promoting interval estimates rather than a focus on p-values

  • Before committing to reducing animals, show “what would have happened if we had done this in the last study” by removing observations from the data set

  • Advertising and speaking to stakeholders

  • Publication strategy:

  • A statistical paper

    • case studies with details removed

  • For each assay, publish the meta-analysis of the historic data and the planned future data analysis

    • preferably in the selected journal to be used for data for new compounds

  • Publish data relating to specific compounds

    • having de-risked by the preceding 2 publications

  • First steps1

    First steps

    QC charts for control groups

    Well received.

    Introduces the idea of expt-to-expt variation.

    Intuitively, the relevance of the historic controls depends on the size of the study to study variation.

    High expt-to-expt variation

    Low expt-to-expt variation

    Bayesian analysis can use the historic control information, down-weighting it according to the amount of experiment-to-experiment variation

    Types of control groups

    Types of control groups

    • Not used for formal statistical comparisons. Example uses:

      • To ensure challenge is working; to establish a “window”; to check consistency with previous studies; to convert values to %.

      • Replace group with a range from a predictive distribution

    • Used for formal comparison vs. test compounds/doses

      • Used as the comparison in t-tests ..etc

      • Combine down-weighted historic data with the current experiment

    Bayesian methodology

    Bayesian methodology


    • Analysehistoric control treatment group data, excluding the last study.

      • Bayesian meta-analysis

    • Analyse the last study

      • Show what would have happened if we had “bought into” the Bayesian approach; omit animals if necessary

    • Possible options for future studies:

      • Omit all/some animals from all/some control groups.

      • Use historic data as prior information combined with observed data in a Bayesian analysis.

      • Use historic data to give a predictive distribution for control group. i.e. don’t include that treatment group in current study.

    • Statistical model based on meta-analytic predictive methodology in Neuenschwanderet al

    Bayesian meta analytic predictive approach

    Bayesian meta-analytic predictive approach

    Statistical model based on Neuenschwander et al (2010)

    • Historic data:, h=1…H:

      Observed study data: h | θh, σa2~ N(θh, σa2)

      Study means: θ1…θH ~ N(μ, τ2)

    • Predictions for next study, denoted by *:

      True study mean θ* ~ N(μ, τ2)  prior for next study

      Observed study mean*| θ*, σa2~ N(θ*, σa2/n*)  predictive dn for next study

    • Priors and hyperpriors

  • μ has vague normal prior

  • Τhas vague half normal prior (sensitivity analysis carried out)

  • σa2 has vague gamma prior

  • Bayesian methodology 2

    Bayesian methodology (2)

    “Replacing” control groups with predictive distributions

    Bayesian analysis of historic control data

    New study data: 8 per group (for the other groups)

    Traditional analysis of current study

    QC-chart like limits for control group

    Overlay Bayesian analysis in data presentations

    Suitable for control groups not used in statistical comparisons

    Case study1

    Case study

    Predictive distribution

    Bayesian methodology 4

    Bayesian methodology (4)

    Full Bayesian analysis

    This is a simplification of the exact analysis

    Bayesian analysis of historic control data

    New study data: 8 per group

    Bayesian analysis of current study

    Effectively N control animals with mean, m

    Suitable for any control groups but requires a Bayesian analysis for each data set.

    Results and


    Case study2

    Case Study

    Full Bayesian analysis

    The Bayesian analysis gives narrower confidence intervals. It is comparable to using 3 extra animals (11 instead of 8)


    [email protected]/kg


    [email protected]/kg


    [email protected]/kg


    [email protected]/kg

    Summary to date

    Summary to date

    • Predictive approach developed for two models

      • Biologists very positive; implementation being considered

      • Potential saving: one control group per study

    • Full Bayesian approach developed for one model

      • Modest savings in numbers of animals

      • Software issue; potentially lengthening turnaround times for rapid screens

      • Biologists suggested starting with something slightly more low-key

    • QC charts provided an excellent introduction to between and within study variation

    • Bayesian methods can reduce required resource (animal numbers)

    • To have the greatest impact, focus on studies repeated very frequently

    • Even for well controlled in vivo experiments study-to-study variation is not negligible

    • ,



    • Neuenschwander, B., Capkun-Niggli, G., Branson, M. and Spiegelhalter, DJ. Summarizing historical information on controls in clinical trials., Clin Trials 2010 7: 5

    • Di Scala L, Kerman J, Neuenschwander B. Collection, synthesis, an interpretation of evidence: a proof-of-concept study in COPD. Statistics in Medicine 2013; 32: 1621–1634.

    • Evans, M, Hastings, N and Peacock, B (2000). Statistical Distributions. 3rd Edition. New York: Wiley

    • Beaumont and Rannala, The Bayesian revolution in genetics. Nature Reviews Genetics 5, 251-261 (April 2004)



  • Login