1 / 30

OVERVIEW - PowerPoint PPT Presentation

  • Uploaded on

STATISTICAL CONSIDERATIONS IN POSTMARKETING SAFETY EVALUATION A. Lawrence Gould Merck Research Laboratories West Point, PA [[email protected]] FDA/Industry Workshop 29 September 2006 Washington, DC. OVERVIEW. Spontaneous AE Reports.

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about ' OVERVIEW' - amandla

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

STATISTICAL CONSIDERATIONS IN POSTMARKETING SAFETY EVALUATIONA. Lawrence GouldMerck Research LaboratoriesWest Point, PA [[email protected]]FDA/Industry Workshop29 September 2006Washington, DC


Spontaneous ae reports
Spontaneous AE Reports EVALUATION

  • Clinical trial safety information is limited & relatively short duration

  • Safety data collection continues after drug approval

    • Detect rare adverse events

    • Obtain tolerability information in a broader population

  • Large amount of low-quality data collected

    • Not usable for trt comparisons or risk assessment

    • Unknown sensitivity & specificity

  • Evaluation by skilled clinicians & epidemiologists

  • Long history of research on issue

Information available postmarketing
Information Available Postmarketing EVALUATION

  • Previously undetected adverse and beneficial effects that may be uncommon or delayed, i.e., emerging only after extended treatment

  • Patterns of drug utilization

  • Effect of drug overdoses

  • Clinical experience with study drugs in their “natural” environment

The pharmacovigilance process
The Pharmacovigilance Process EVALUATION





Detect Signals

Generate Hypotheses

Insight from


Public Health

Impact, Benefit/Risk


Type A






Type B


Restrict use/


Change Label

Considerations issues an incomplete list
Considerations & Issues (An Incomplete List!) EVALUATION

  • Incomplete reports of events, not reactions

  • Bias & noise in system

  • Difficult to estimate incidence because no. of pats at risk, pat-yrs of exposure seldom reliable

  • Significant under reporting (esp. OTC)

  • Synonyms for drugs & events → sensitivity loss

  • Duplicate reporting

  • No certainty that a drug caused the reaction reported

  • Cannot use accumulated reports to calculate incidence, estimate drug risk, or compare drugs

Data mining

Data mining is a part of pharmacovigilance
Data Mining is a Part of Pharmacovigilance EVALUATION

  • Identify subtle associations (e.g., drug+drug+event) and complex relationships not apparent by simple summary

  • Identify potential toxicity early

  • Finding ‘real’ D-E associations similar to finding potential active compounds or expressed genes – not exactly the same (no H0) – more like model selection

  • Still need initial case review

    respond to reports involving severe, potential life-threatening events eg., Stevens-Johnson syndrome, agranulocytosis, anaphylactic shock

  • Clinical/biological/epidemiological verification of apparent associations is essential

Typical data display
Typical Data Display EVALUATION

Basic idea:

Flag when

R = a/E(a) is “large”

  • Some possibilities

  • Reporting Ratio: E(a) = nTD nTA/n

  • Proportional Reporting Ratio: E(a) = nTD c/nOD

  • Odds Ratio: E(a) = b  c/d

  • Need to accommodate uncertainty, especially if a is small

  • Bayesian approaches provide a way to do this

  • Currently used bayesian approaches
    Currently Used Bayesian Approaches EVALUATION

    • Empirical Bayes (DuMouchel, 1998) & WHO (Bate, 1998)

    • Both use ratio nij / Eij where

      nij = no. of reports mentioning both drug i & event j

      Eij = expected no. of reports of drug i & event j

    • Both report features of posterior dist’n of ‘information criterion’

      ICij = log2 nij / Eij = PRRij

    • Eij usually computed assuming drug i & event j are mentioned independently

    • Ratio > 1 (IC > 0)  combination mentioned more often than expected if independent

    Comparative example dumouchel 1998
    Comparative Example (DuMouchel, 1998) EVALUATION

    • No. Reports = 4,864,480, Mentioning drug = 85,304

    Result from 6 years of reports on lisinopril
    Result From 6 Years of Reports on Lisinopril EVALUATION

    Events w/Lower 5% RR Bnd > 2 (Bold  N  100)

    Accumulating information over time
    Accumulating Information over Time EVALUATION

    • Lower 5% quantiles of RR stabilized fairly soon

    Time sliced evolution of risk ratios
    Time-Sliced Evolution of Risk Ratios EVALUATION

    • See how values of criteria change over time within time intervals of fixed length

    Change in ICij for reports of selected events on A2A from 1995 to 2000

    tension =


    failure =

    heart failure

    kalemia =


    edema =


    Masking of ae drug relationships 1
    Masking of AE-Drug Relationships (1) EVALUATION

    • Company databases smaller than regulatory databases, more loaded with ‘similar’ drugs

      eg, Drug A is 2nd generation version of Drug B, similar mechanism of action, many reports with B

    • Elevated reporting frequency on Drug B could mask effect of Drug A

    • May be useful to provide results when reports mentioning Drug B are omitted

    Example 2 vaccine vaccine interaction
    Example 2: Vaccine-Vaccine Interaction EVALUATION

    • From FDA VAERS database, reports from 1990-2002

    • Intussusception is a serious intestinal malady observed to affect infants vaccinated against rotavirus

    • Look at reports of intussusception that mention rotavirus vaccine (RV) and DTAP vaccine

    • DTAP is a benign combination vaccine commonly administered to infants

    • Demonstration question: Intussusception very commonly reported with RV – but does the reporting rate depend on whether DTAP was co-administered?

    • Not easy to address using standard pharmacovigilance procedures

    Outline of analysis
    Outline of Analysis EVALUATION

    • Standard tools provide intussusception reporting rate for pairs of vaccines, and for vaccines singly

    • Result is a 3-way count table (corresponding to RV + or -, DTAP + or -, and intussusception + or -)

    • Use log-linear model to see if intussusception is mentioned with the two vaccines together more often than the separate vaccine-intussusception reporting associations would predict

    • Turns out that there is an association – Likelihood ratio chi-square is 17.41, 1 df, highly significant

    Comments EVALUATION

    • Intussusception seems to be reported more often than expected when RV and DTAP are given together than when RV is given without DTAP, after adjusting for individual vaccine-intussusception associations

    • Reports of intussception without RV are very rare, about 4.5/10,000 reports if RV is not mentioned

    • The joint effect of RV and DTAP on intussusception reporting is small, but does reach statistical significance

    • Not clear that apparent association means anything -- actual synergy between RV and DTAP seems unlikely, but explanation requires clinical knowledge

    A new bayesian approach gould biometrical journal 2006 to appear
    A NEW BAYESIAN APPROACH EVALUATION(Gould, Biometrical Journal 2006, to appear)

    Model for process generating observations
    Model for Process Generating Observations EVALUATION

    • ni = no. of reports mentioning i-th drug-event pair ~ Poisson (true for EB approach as well)

      f(ni | Ei, i) = fPois(ni ; iEi)

    • i drawn from a gamma(a0, b0) distribution or from a gamma(a1, b1) distribution

      • A model selection problem

      • Dist’ns reflect physician/epidemiologist’s judgment as to what range of  values corresponds to ‘signals’, and what does not

    Expected count

    under independence

    Association measure

    Prior model density of
    Prior/Model Density of EVALUATION

    • Bayes approach starts with a random mixture of gamma densities,

      f0( ; , a0, b0, a1, b1)

      = (1 - )fgam(; a0, b0) + fgam(; a1, b1)

      Use value of Ppost(g = 1) for inference

    • EB approach starts with expectation wrt  given p  nonrandom mixture of gamma densities,

      f0( ; p, a0, b0, a1, b1)

      = pfgam(; a0, b0) + (1-p)fgam(; a1, b1)

      Use quantiles of posterior dist’n of  for inference

    Analyst specifies parameter values

    Data determine parameter values

    Comments EVALUATION

    • Bayes and EB approaches both model strength of drug-event reporting assn as a gamma mixture

    • Diagnostic properties of Bayes method can be determined analytically or by simulation

    • Unknown separation of the true alternative dist’ns for  more important than prior dist’n used for analysis

    • Methods described here can be applied to other models – Scott & Berger (2005) used normal distributions – could also use binomial instead of Poisson, beta instead of gamma distributions to develop screening methods for AEs in clinical trials


    Discussion EVALUATION

    • Bayesian approaches may be useful for detecting possible emerging signals, especially with few events

    • MCA (UK) currently uses PRR for monitoring emergence of drug-event associations

    • Signal detection combines numerical data screening, statistical interpretation, and clinical judgement

    • Most apparent associations represent known problems

    • ~ 25% may represent signals about previously unknown associations

    • The actual false positive rate is unknown

    What next
    What Next? EVALUATION

    • PhRMA/FDA working group has published a white paper addressing many of these issues

      Drug Safety (2005) 28: 981-1007

    • Further refine methods, look for associations among combinations of drugs and events, timing of reports

    • Data mining is like screening, need to evaluate diagnostic properties of various approaches

    • Need good dictionaries: many synonyms  difficult signal detection

      • Event names: MedDRA may help

      • Drug names: Need a common dictionary of drug names to minimize dilution effect of synonyms