slide1 n.
Skip this Video
Download Presentation

Loading in 2 Seconds...

play fullscreen
1 / 30

OVERVIEW - PowerPoint PPT Presentation

  • Uploaded on

STATISTICAL CONSIDERATIONS IN POSTMARKETING SAFETY EVALUATION A. Lawrence Gould Merck Research Laboratories West Point, PA [] FDA/Industry Workshop 29 September 2006 Washington, DC. OVERVIEW. Spontaneous AE Reports.

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about 'OVERVIEW' - amandla

Download Now An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

STATISTICAL CONSIDERATIONS IN POSTMARKETING SAFETY EVALUATIONA. Lawrence GouldMerck Research LaboratoriesWest Point, PA []FDA/Industry Workshop29 September 2006Washington, DC

spontaneous ae reports
Spontaneous AE Reports
  • Clinical trial safety information is limited & relatively short duration
  • Safety data collection continues after drug approval
    • Detect rare adverse events
    • Obtain tolerability information in a broader population
  • Large amount of low-quality data collected
    • Not usable for trt comparisons or risk assessment
    • Unknown sensitivity & specificity
  • Evaluation by skilled clinicians & epidemiologists
  • Long history of research on issue
information available postmarketing
Information Available Postmarketing
  • Previously undetected adverse and beneficial effects that may be uncommon or delayed, i.e., emerging only after extended treatment
  • Patterns of drug utilization
  • Effect of drug overdoses
  • Clinical experience with study drugs in their “natural” environment
the pharmacovigilance process
The Pharmacovigilance Process





Detect Signals

Generate Hypotheses

Insight from


Public Health

Impact, Benefit/Risk


Type A






Type B


Restrict use/


Change Label

considerations issues an incomplete list
Considerations & Issues (An Incomplete List!)
  • Incomplete reports of events, not reactions
  • Bias & noise in system
  • Difficult to estimate incidence because no. of pats at risk, pat-yrs of exposure seldom reliable
  • Significant under reporting (esp. OTC)
  • Synonyms for drugs & events → sensitivity loss
  • Duplicate reporting
  • No certainty that a drug caused the reaction reported
  • Cannot use accumulated reports to calculate incidence, estimate drug risk, or compare drugs
data mining is a part of pharmacovigilance
Data Mining is a Part of Pharmacovigilance
  • Identify subtle associations (e.g., drug+drug+event) and complex relationships not apparent by simple summary
  • Identify potential toxicity early
  • Finding ‘real’ D-E associations similar to finding potential active compounds or expressed genes – not exactly the same (no H0) – more like model selection
  • Still need initial case review

respond to reports involving severe, potential life-threatening events eg., Stevens-Johnson syndrome, agranulocytosis, anaphylactic shock

  • Clinical/biological/epidemiological verification of apparent associations is essential
typical data display
Typical Data Display

Basic idea:

Flag when

R = a/E(a) is “large”

    • Some possibilities
  • Reporting Ratio: E(a) = nTD nTA/n
  • Proportional Reporting Ratio: E(a) = nTD c/nOD
  • Odds Ratio: E(a) = b  c/d
  • Need to accommodate uncertainty, especially if a is small
  • Bayesian approaches provide a way to do this
currently used bayesian approaches
Currently Used Bayesian Approaches
  • Empirical Bayes (DuMouchel, 1998) & WHO (Bate, 1998)
  • Both use ratio nij / Eij where

nij = no. of reports mentioning both drug i & event j

Eij = expected no. of reports of drug i & event j

  • Both report features of posterior dist’n of ‘information criterion’

ICij = log2 nij / Eij = PRRij

  • Eij usually computed assuming drug i & event j are mentioned independently
  • Ratio > 1 (IC > 0)  combination mentioned more often than expected if independent
comparative example dumouchel 1998
Comparative Example (DuMouchel, 1998)
  • No. Reports = 4,864,480, Mentioning drug = 85,304
result from 6 years of reports on lisinopril
Result From 6 Years of Reports on Lisinopril

Events w/Lower 5% RR Bnd > 2 (Bold  N  100)

accumulating information over time
Accumulating Information over Time
  • Lower 5% quantiles of RR stabilized fairly soon
time sliced evolution of risk ratios
Time-Sliced Evolution of Risk Ratios
  • See how values of criteria change over time within time intervals of fixed length

Change in ICij for reports of selected events on A2A from 1995 to 2000

tension =


failure =

heart failure

kalemia =


edema =


masking of ae drug relationships 1
Masking of AE-Drug Relationships (1)
  • Company databases smaller than regulatory databases, more loaded with ‘similar’ drugs

eg, Drug A is 2nd generation version of Drug B, similar mechanism of action, many reports with B

  • Elevated reporting frequency on Drug B could mask effect of Drug A
  • May be useful to provide results when reports mentioning Drug B are omitted
example 2 vaccine vaccine interaction
Example 2: Vaccine-Vaccine Interaction
  • From FDA VAERS database, reports from 1990-2002
  • Intussusception is a serious intestinal malady observed to affect infants vaccinated against rotavirus
  • Look at reports of intussusception that mention rotavirus vaccine (RV) and DTAP vaccine
  • DTAP is a benign combination vaccine commonly administered to infants
  • Demonstration question: Intussusception very commonly reported with RV – but does the reporting rate depend on whether DTAP was co-administered?
  • Not easy to address using standard pharmacovigilance procedures
outline of analysis
Outline of Analysis
  • Standard tools provide intussusception reporting rate for pairs of vaccines, and for vaccines singly
  • Result is a 3-way count table (corresponding to RV + or -, DTAP + or -, and intussusception + or -)
  • Use log-linear model to see if intussusception is mentioned with the two vaccines together more often than the separate vaccine-intussusception reporting associations would predict
  • Turns out that there is an association – Likelihood ratio chi-square is 17.41, 1 df, highly significant
  • Intussusception seems to be reported more often than expected when RV and DTAP are given together than when RV is given without DTAP, after adjusting for individual vaccine-intussusception associations
  • Reports of intussception without RV are very rare, about 4.5/10,000 reports if RV is not mentioned
  • The joint effect of RV and DTAP on intussusception reporting is small, but does reach statistical significance
  • Not clear that apparent association means anything -- actual synergy between RV and DTAP seems unlikely, but explanation requires clinical knowledge
model for process generating observations
Model for Process Generating Observations
  • ni = no. of reports mentioning i-th drug-event pair ~ Poisson (true for EB approach as well)

f(ni | Ei, i) = fPois(ni ; iEi)

  • i drawn from a gamma(a0, b0) distribution or from a gamma(a1, b1) distribution
    • A model selection problem
    • Dist’ns reflect physician/epidemiologist’s judgment as to what range of  values corresponds to ‘signals’, and what does not

Expected count

under independence

Association measure

prior model density of
Prior/Model Density of 
  • Bayes approach starts with a random mixture of gamma densities,

f0( ; , a0, b0, a1, b1)

= (1 - )fgam(; a0, b0) + fgam(; a1, b1)

Use value of Ppost(g = 1) for inference

  • EB approach starts with expectation wrt  given p  nonrandom mixture of gamma densities,

f0( ; p, a0, b0, a1, b1)

= pfgam(; a0, b0) + (1-p)fgam(; a1, b1)

Use quantiles of posterior dist’n of  for inference

Analyst specifies parameter values

Data determine parameter values

  • Bayes and EB approaches both model strength of drug-event reporting assn as a gamma mixture
  • Diagnostic properties of Bayes method can be determined analytically or by simulation
  • Unknown separation of the true alternative dist’ns for  more important than prior dist’n used for analysis
  • Methods described here can be applied to other models – Scott & Berger (2005) used normal distributions – could also use binomial instead of Poisson, beta instead of gamma distributions to develop screening methods for AEs in clinical trials
  • Bayesian approaches may be useful for detecting possible emerging signals, especially with few events
  • MCA (UK) currently uses PRR for monitoring emergence of drug-event associations
  • Signal detection combines numerical data screening, statistical interpretation, and clinical judgement
  • Most apparent associations represent known problems
  • ~ 25% may represent signals about previously unknown associations
  • The actual false positive rate is unknown
what next
What Next?
  • PhRMA/FDA working group has published a white paper addressing many of these issues

Drug Safety (2005) 28: 981-1007

  • Further refine methods, look for associations among combinations of drugs and events, timing of reports
  • Data mining is like screening, need to evaluate diagnostic properties of various approaches
  • Need good dictionaries: many synonyms  difficult signal detection
    • Event names: MedDRA may help
    • Drug names: Need a common dictionary of drug names to minimize dilution effect of synonyms