Meta-Analysis of Clinical Data for Regulated Biopharmaceutical Products: Answers to Frequently Asked Questions. Brenda Crowe, Research Advisor, Eli Lilly and Company With special thanks to Jesse Berlin. Midwest Biopharmaceutical statistics workshop May 21, 2013. Disclaimer.
Meta-Analysis of Clinical Data for Regulated Biopharmaceutical Products: Answers to Frequently Asked Questions
Brenda Crowe, Research Advisor, Eli Lilly and Company
With special thanks to Jesse Berlin
Midwest Biopharmaceutical statistics workshop
May 21, 2013
The views expressed herein represent those of the presenter and do not necessarily represent the views or practices of the presenter’s employer or any other party
SPERT = Safety Planning, evaluation and Reporting Team
Decisions on what to combine depend on the specific questions to be answered (duh)
Often there are several questions and these might require different subsets of studies or subjects
Type of control: placebo vs. active
Dose route or regimen
Concomitant (background) therapy
Methods of eliciting adverse events (e.g., active vs. passive).
Duration of treatment (and follow-up?)
Subgroups of patients based on age groups, geographies, ethnicity groups, or severity of disease, etc.
Phase 1 pharmacokinetic and pharmacodynamic studies (because short duration, healthy subjects or patients with incurable end-stage disease).
Studies that cannot / will not provide individual patient level data if required for analysis.
It is generally most appropriate to combine data from studies that are similar.
Strong similarity is not required for pooling, if the effects of treatment don’t depend on the trial characteristics being considered.
Even with strict criteria using previously collected data, bias could be introduced by retrospective adjudication
For many questions get same answer with IPD as with APD
For analyses that do not require patient-level data, including all relevant studies improves precision
May also reduce bias that could be introduced by limiting the analysis to those where patient-level data are available
However, there can be advantages to IPD
Much easier to detect interactions between treatment and patient-level characteristic with IPD than with APD
Allows specification of a common set of patient-level covariates so subgroup analyses across trials can be performed
Can define outcomes based on combinations of variables defining specific events but that may indicate a common mechanism, e.g., a combination of weight loss or appetite reduction
Complicated by having multiple looks over time and multiple (and an unknown number of) endpoints
Safety Planning, Evaluation, and Reporting Team (SPERT) defined “Tier 1 events” as those for which a prespecified hypothesis has been defined
E.g., to rule out an effect of a certain magnitude for assessing a particular risk (a noninferiority test – as for diabetes drugs)
Generally, should consider performing formal adjustment for multiple looks for Tier 1 events and for multiple endpoints for other events
Need to rule out a relative risk of 1.8 (for CV events) for conditional approval, and 1.3 for final approval
Confidence level for that specific outcome may need to be adjusted for multiple looks, which can be considered separately from non-Tier 1 events because it needs to be met for the drug to move forward
An event of interest: important regardless of the specific side effect profile and
Analogous to a primary analysis in the efficacy setting
Often have low power, lack of a priori definitions, and extraneous variability
Value in trying not to miss a safety signal, but remember that initial detection is not the same as proving that a given AE is definitively related to a given drug
Worry about reducing false negative findings in drug safety given the known limitations of our tools
Heterogeneity refers to differences among studies and/or study results.
Can be classified in 3 ways: clinical, methodological and statistical.
Differences among trials in their
Patient selection (e.g., disease conditions under investigation, eligibility criteria, patient characteristics, or geographic differences)
Differences among trials in their
Interventions (e.g., duration, dosing, nature of the control)
Outcomes (e.g., definitions of endpoints, follow-up duration, cut-off points for scales)
Study design (e.g., the mechanism of randomization).
Study conduct (e.g., allocation concealment, blinding, extent and handling of withdrawals and loss to follow up, or analysis methods).
Decisions about what constitutes clinical heterogeneity and methodological heterogeneity do not involve any calculation and are based on judgment.
Numerical variability in results, beyond expected by sampling variability
May be caused by
Known (or unknown) clinical and methodological differences among trials
Clinical heterogeneity may not always result in statistical heterogeneity.
If there is clinical heterogeneity but little variation in study results, may represent robust, generalizable treatment effects.
Cochran’s Q is a global test of heterogeneity
I2 is a measure of global heterogeneity
KEY POINT: They are informative, but rely on neither of these statistics
Apparent lack of overall heterogeneity does not rule out a specific source of heterogeneity
Conversely, large studies with clinically small variability can yield spuriously high statistical heterogeneity
In some situations, it may not be appropriate to produce a single overall treatment-effect estimate
Goal should sometimes (often) be to model and understand sources of heterogeneity
Risk differences more heterogeneous than odds ratios (OR) or relative risks (RR, a point that is also made in an FDA’s draft guidance for industry on noninferiority trials)
Can model on OR scale then convert to RD or RR to help with clinical interpretability
Constant OR implies effect size must vary for RD, so - must decide whether to estimate the baseline (control) event rate from the external data or from the data included in the actual meta-analysis (implications for variance estimation)
Specify a prior probability distribution
Today’s posterior becomes tomorrow’s prior
Flexibility to deal with heterogeneity through complex modeling
Available under both FE and RE (use Deviance Information Criterion to decide?)
Bayesian inferences are based on the full ‘exact’ posterior distributions (so useful for small numbers of events)
Meta-analysis increasingly used to address safety concerns in drug development.
Up-front thought allows teams to improve planning and enhance data capture, and enhances transparency and interpretation of the results.
Christy Chuang-Stein, and Mohan Beltangady. Reporting cumulative proportion of subjects with an adverse event based on data from multiple studies.Pharmaceut. Statist. 2010
Crowe, Xia, Berlin et al. Recommendations for safety planning, data collection, evaluation and reporting during drug, biologic and vaccine development: a report of the safety planning, evaluation, and reporting team. Clin Trials 2009; 6 430-440
Lièvre, Cucherat and Leizorovicz. Pooling, meta-analysis, and the evaluation of drug safety. Current Controlled Trials in Cardiovascular Medicine 2002
Olkin I, Sampson A. Comparison of meta-analysis versus analysis of variance of individual patient data.Biometrics. Mar 1998;54(1):317-322.