Sensitivity analysis of randomized trials with missing data
1 / 30

Sensitivity Analysis of Randomized Trials with Missing Data - PowerPoint PPT Presentation

  • Uploaded on

Sensitivity Analysis of Randomized Trials with Missing Data. Daniel Scharfstein Department of Biostatistics Johns Hopkins University Statistical Principles for Clinical Trials (FDA Guidance - E9).

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about 'Sensitivity Analysis of Randomized Trials with Missing Data' - lel

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Sensitivity analysis of randomized trials with missing data l.jpg

Sensitivity Analysis of Randomized Trials with Missing Data

Daniel Scharfstein

Department of Biostatistics

Johns Hopkins University

Statistical principles for clinical trials fda guidance e9 l.jpg
Statistical Principles for Clinical Trials(FDA Guidance - E9)

  • “It is important to evaluate the robustness of the results and primary conclusions of the trial.”

  • Robustness refers to “the sensitivity of the overall conclusions to various limitations of the data, assumptions, and analytic approaches to data analysis.”

Fda critical path initiative l.jpg
FDA Critical Path Initiative

  • FDA’s Critical Path white-paper “calls for a joint effort of industry, academia, and the FDA to identify key problems and develop targeted solutions.”

  • In response to this document, the Office of Biostatistics at CDER has identified missing data as one of these key areas (Robert O’Neil, personal communication).

Fda office of biostatistics at cder robert o neil l.jpg
FDA: Office of Biostatistics at CDER(Robert O’Neil)

  • “Virtually every drug/disease area in clinical trials has problems with patient data that are missing because patients dropped out, died, withdrew due toxicity or aggravation with the trial, failed to complete forms, and other reasons.”

  • “The success and failure of a trial and its interpretation often depends on how these missing data are dealt with at either the planning or analysis stage.”

  • “Current statistical methodologies proposed need to be evaluated for their ability to address informative treatment-related missing data …”

Fda office of biostatistics at cder robert o neil5 l.jpg
FDA: Office of Biostatistics at CDER(Robert O’Neil)

  • “… a concensus on missing data approaches needs to be developed in order to minimize the impact of failed studies and remove obstacles to ambiguous interpretation of product efficacy and safety conclusions.”

  • “There are no established set of diagnostics to evaluate the severity or impact of missing data …”

  • “there is no easily available computer software …”

Clinical trial registration icmje l.jpg
Clinical Trial Registration - ICMJE

  • “In return for the altruism and trust that makes clinical research possible, the research enterprise has an obligation to conduct research ethically and to report it honestly.”

  • “Registration is only part of the means to an end; that end is full transparency with respect to the performance and reporting of clinical trials.”

  • In my view, the evaluation of the robustness of trial conclusions is an integral part of honest and transparent reporting.

Slide7 l.jpg

  • Present a coherent sensitivity analysis paradigm for the presentation of results of clinical trials in which there is concern about informative missing data.

Actg 175 l.jpg
ACTG 175

  • ACTG 175 was a randomized, double bind trial designed to evaluate nucleoside monotherapy vs. combination therapy in HIV+ individuals with CD4 counts between 200 and 500.

  • Participants were randomized to one of four treatments: AZT, AZT+ddI, AZT+ddC, ddI

  • CD4 counts were scheduled at baseline, week 8, and then every 12 weeks thereafter.

Actg 1759 l.jpg
ACTG 175

  • One goal of the investigators was to compare the treatment-specific means of CD4 count at week 56 had all subjects remained on their assigned treatment through that week.

  • The interest is efficacy rather than effectiveness.

  • We define a completer to be a subject who stays on therapy and is measured at week 56. Otherwise, the subject is called a drop-out.

  • 33.6% and 26.5% of subjects dropped out in the AZT+ddI and ddI arms, respectively.

Actg 17510 l.jpg
ACTG 175

  • Completers-only analysis

Actg 17511 l.jpg
ACTG 175

  • The completers-only means will be valid estimates if, within treatment groups, the completers and drop-outs are similar on measured and unmeasured characteristics.

  • Missing at random (MAR), with respect to treatment group.

  • Without incorporating additional information, the MAR assumption is untestable.

  • It is well known from other studies that, within treatment groups, drop-outs tend to be very different than completers.

Sensitivity analysis paradigm l.jpg
Sensitivity Analysis Paradigm

  • Evaluate the robustness of the above conclusions to deviations from the untestable MAR assumption.

  • Three steps

    • Models

    • Estimation

    • Testing

Step 1 models l.jpg
Step 1: Models

  • For each treatment group, specify a set of models for the relationship between the distributions of the outcome for drop-outs and completers.

  • Index the treatment-specific models by an untestable, interpretable parameter (alpha), where zero denotes MAR.

  • alpha is called a selection bias parameter and it indexes deviations from MAR.

  • Pattern-mixture model

Slide15 l.jpg

Step 1: Models week 56 for drop-outs

  • Selection model for the probability of being a completer given the outcome.

  • The parameter alpha is interpreted as the log odds ratio of dropping out when comparing subjects whose log CD4 count at week 56 differs by 1 unit.

  • alpha>0 (<0) indicates that subjects with higher (lower) CD4 counts are more likely to drop-out.

  • alpha=0.5 (-0.5) implies that a 2-fold increase in CD4 count yields a 1.4 increase (0.7 decrease) in the odds of dropping out.

Slide16 l.jpg

Step 2: Estimation week 56 for drop-outs

  • For a plausible range of alpha’s, estimate the treatment-specific means by taking a weighted average of the mean outcomes from the completers and drop-outs.

Slide19 l.jpg

Step 3: Testing of

  • Test the null hypothesis of no treatment effect as a function of treatment-specific selection bias parameters.

  • For each combination of the treatment-specific selection bias parameters, form a Z-statistic by taking the difference in the estimated means divided by the estimated standard error of the difference.

Slide20 l.jpg

Step 3: Testing of

  • If the selection bias parameters are correctly specified, this statistic is normal(0,1) under the null hypothesis.

  • Reject the null hypothesis at the 0.05 level if the absolute value of the Z-statistic is greater than 1.96.

Slide23 l.jpg

Sensitivity Analysis Paradigm of

  • Extensions

    • Longitudinal and time-to-event outcomes

    • MAR or CAR (with respect to all observable time-independent and dependent data)

    • Sensitivity analysis with respect to alpha

Conjecture l.jpg
Conjecture of

  • There is information from previously conducted clinical studies to help in the analysis of the current trials.

  • Data from previous trials may be able to restrict the range of or estimate alpha.

Comments on other approaches l.jpg
Comments on Other Approaches of

  • Likelihood-based inference

  • LOCF

Likelihood based inference l.jpg
Likelihood-based Inference of

  • A parametric model for the outcome and a parametric for the probability of being a completer given the outcome.

  • For example, the outcome is log normal.

  • Estimated alpha’s are -2.6 (95% CI: [-3.0,-2.1]) and -2.8 (95% CI: [-3.3,-2.2]) in the AZT+ddI and ddI arms.

  • Estimated means are 303 (95% CI: [278,331]) and 297 (95% CI: [271,324]) in the AZT+ddI and ddI arms.

  • Need to be certain about log-normal assumption.

Slide28 l.jpg
LOCF week 56 for drop-outs

Bad idea

  • Imputing an unreasonable value.

  • Results may be conservative or anti-conservative.

  • Uncertainty is under-estimated.

Summary l.jpg
Summary week 56 for drop-outs

  • We have presented a paradigm for reporting the results of clinical trials where missingness is plausibly related to outcomes.

  • We believe this approach provides a more honest characterization of the overall uncertainty, which stems from both sampling variability and lack of knowledge of the missingness mechanism.

Dscharf@jhsph edu l.jpg week 56 for drop-outs

  • Scharfstein, Rotnitzky A, Robins JM, and Scharfstein DO: “Semiparametric Regression for Repeated Outcomes with Non-ignorable Non-response,” Journal of the American Statistical Association, 93, 1321-1339, 1998.

  • Scharfstein DO, Rotnitzky A, and Robins, JM.: “Adjusting for Non-ignorable Drop-out Using Semiparametric Non-response Models (with discussion),” Journal of the American Statistical Association, 94, 1096-1146, 1999.

  • Rotnitzky A, Scharfstein DO, Su TL, and Robins JM: “A Sensitivity Analysis Methodology for Randomized Trials with Potentially Non-ignorable Cause-Specific Censoring,” Biometrics, 57:30-113, 2001

  • Scharfstein DO, Robin JM, Eddings W and Rotnitzky A: “Inference in Randomized Studies with Informative Censoring and Discrete Time-to-Event Endpoints,” Biometrics, 57: 404-413, 2001.

  • Scharfstein DO and Robins JM: “Estimation of the Failure Time Distribution in the Presence of Informative Right Censoring,” Biometrika 89:617-635, 2002.

  • Scharfstein DO, Daniels M, and Robins JM: “Incorporating Prior Beliefs About Selection Bias in the Analysis of Randomized Trials with Missing Data,” Biostatistics, 4: 495-512, 2003.