A n Analytic Road Map for Incomplete Longitudinal Clinical Trial Data

1. Craig Mallinckrodt ICSA June 4, 2007Raleigh, NC An Analytic Road Map for Incomplete Longitudinal Clinical Trial Data

2. Context • Road map ≠ driving directions • provides info about alternatives • framework to plan route, doesn’t choose it • Route to valid analyses of incomplete longitudinal data can be problematic • Many advances in missing data theory and implementation • We can do better than LOCF • Need framework to plan use of newer methods

3. Outline • Missing data mechanisms • Modeling philosophies • Example of developing an analytic road map • Example of implementing an analytic road map • Today’s focus is on missing data. Concepts easily extended to include modeling of time and correlation

4. Starting Point • No universally best method for analyzing longitudinal data • This implies analysis must be tailored to the specific situation at hand • Consider the desired attributes of the analysis and the characteristics of the data

5. Missing Data Mechanisms • MCAR - missing completely at random • Conditional on the model, neither observed or unobserved outcomes are related to dropout • MAR - missing at random • Conditional on the model, observed outcomes are related to dropout, but unobserved outcomes are not • MNAR - missing not at random • Unobserved outcomes are related to dropout

6. Missing Data Mechanisms • MCAR - missing completely at random • Conditional on independent variables in the model, neither observed or unobserved outcomes of the dependent variable are related to dropout • MAR - missing at random • Conditional on the independent variables in the model, observed outcomes of the dependent variable are related to dropout, but unobserved outcomes are not

7. Consequences • Missing data mechanism is a characteristic of the data AND the model • Differential dropout by treatment indicates covariate dependence, not mechanism • Mechanism can vary from one outcome to another • Terms like ignorable missingness and informative censoring must also consider the method • What is ignorable with likelihood-based analysis is non-ignorable with GEE

8. Analytic Approaches for Continuous Outcomes • MCAR • ANOVA, GEE • MAR • Likelihood-based mixed-effects models (LB-MEM), Multiple imputation (MI), Propensity scoring, Weighted analyses • MNAR • Selection models, Pattern mixture models, Shared parameter models, sensitivity analyses • Note that if missing values are imputed other assumptions also need to be considered

9. Modeling Philosophies • Restrictive modeling • Inclusive modeling

10. Restrictive Modeling • Simple models with few independentvariables • Often include only the design factors of theexperiment

11. Inclusive Modeling • In addition to the design factors of the experiment, models include auxiliary variables • Auxiliary variables included to improve performance of the missing data procedure – expand the scope of MAR • baseline covariates • time varying post-baseline covariates

12. Rationale For Inclusive Modeling • MAR: conditional on the dependent and independent variables in the analysis, unobserved values of the dependent variable are independent of dropout • Hence adding more variables that explain dropout can make missingness MAR that would otherwise be MNAR

13. Inclusive Modeling with LB-MEM • Add auxiliary variables to analysis model • Adding baseline covariates is straightforward • Adding post-baseline time varying covariates that are also influenced by treatment often dilute the treatment effect • Could conduct a multivariate analysis where the auxiliary variable is a second dependent variable

14. Inclusive Modeling with MI • Include auxiliary variables in the imputation model • Typically analyze complete data sets using a restrictive model, but could use the same inclusive model to analyze the data • Similar approaches can be used in propensity scoring and weighted analyses where a dropout model is used to develop propensity score bins or inverse probability of dropout weights, followed by a second analysis step

15. Developing An Analytic Road Map:Example From A Depression Trial • Confirmatory clinical trial of an antidepressant. Primary analysis should be simple and dependable • MAR with restrictive modeling as primary • Use MAR with inclusive modeling and MNAR methods as sensitivity analyses • Use local influence to investigate impact of influential patients

16. Why MAR? • Data in clinical trials are seldom MCAR because the observed outcomes typically influence dropout (lack of efficacy) • Trials are designed to observe all the relevant information, which minimizes MNAR data • Hence in the highly controlled scenario of clinical trials missing data may be mostly MAR

17. Why not MNAR? • Rubin (1994): “…, even inferences for the data parameters generally depend on the posited missingness mechanism, a fact that typically implies greatly increased sensitivity of inference…” • Laird (1994): “estimating the unestimable can be accomplished only by making modeling assumptions, The consequences of model misspecification will be more severe in the non-random case.” • Molenberghs, Kenward & Lesaffre (1997): “conclusions are conditional on the appropriateness of the assumed model, which in a fundamental sense is not testable.”

18. Why Restrictive Modeling? • Historically favor simple models, with impact of other factors addressed via secondary analyses • No strong a priori evidence of important auxiliary variables • Cost of including unnecessary variables? • Avoids any potential confounding of auxiliary variables with design factors (e.g., treatment)

19. Implementing The Road Map: Example From A Depression Trial • 259 patients, randomized 1:1 ratio to drug and placebo • Response: Change of HAMD17 score from baseline • 6 post-baseline visits (Weeks 1,2,3,5,7,9) • Primary objective: test the difference of mean change in HAMD17 total score between drug and placebo at the endpoint • Primary analysis: LB-MEM

20. Primary Analysis: LB-MEM proc mixed; class subject treatment time site; model Y = baseline treatment time site treatment*time ; repeated time / sub = subject type = un; lsmeans treatment*time / cl diff; run; This is a full multivariate model, with unstructured modeling of time and correlation. More parsimonious approaches may be useful in other scenarios Treatment contrast 2.17, p = .024

21. Patient Disposition • Drug Placebo • Protocol complete 60.9% 64.7% • Adverse event 12.5% 4.3% • Lack of efficacy 5.5% 13.7% • Differential rates, timing, and/or reasons for dropout do not necessarily distinguish between MCAR, MAR, MNAR

22. Inclusive Modeling in MI:Including Auxiliary AE Data • Imputation Models • *Yih = µ +1 Yi1 +…+ h-1 Yi(h-1) + ih • Yih = µ + 1 Yi1 +…+ h-1 Yi(h-1) + 1 AEi1 +…+ h-1 AEi(h-1) + ih • Yih= µ + 1 Yi1 +…+ h-1 Yi(h-1) + 1 AEi1 +…+ h-1 AEi(h-1) +11 (Yi1 *AEi1 ) + …+i(h-1) (Yi(h-1) * AEi(h-1) ) + ih • Analysis Model MMRM as previously described

23. Result • MI results were not sensitive to the different imputation models Endpoint contrastMMRM 2.2MI Y+AE 2.3MI Y+AE+Y*AE 2.1 • Including AE data might be important in other scenarios. Many ways to define AE

24. MNAR Modeling • Implement a selection model • Had to simplify model: modeled time as linear + quadratic, and used ar(1) correlation • Compare results from assuming MAR, MNAR • Also obtain local influence to assess impact of influential patients on treatment contrasts and non-random dropout

25. Selection Model Results

26. Local Influence: Influential Patients

27. Individual Profiles with Influential Patients Highlighted

28. Investigating The Influential Patients • The most influential patient was #30, a drug-treated patient that had the unusual profile of a big improvement but dropped out at week 1 • This patient was in his/her first MDD episode when s/he was enrolled • This patient dropped out based on his/her own decision claiming that the MDD was caused by high carbon monoxide level in his/her house • This patient was of dubious value for assessing the efficacy of the drug

29. Selection Model: Influential Patients Removed

30. Implications • Comforting that no subjects had a huge influence on results. Impact bigger if it were a smaller trial • Similar to other depression trials we have investigated, results not influenced by MNAR data • We can be confident in the primary result

31. Discussion • MAR with restrictive modeling was a reasonable choice for the primary analysis • MAR with inclusive modeling and MNAR was useful in assessing sensitivity • Sensitivity analyses promote the appropriate level of confidence in the primary result and lead us to an alternative analysis in which we can have the greatest possible confidence

32. Opinions • Inclusive modeling has been under utilized • More research to understand dropout would be useful • Did not discuss pros and cons of various ways to implement inclusive modeling. Use the one you know? Be careful to not dilute treatment • The road map for analyses used in the example data is specific to that scenario and is not intended to be a general prescription

33. Conclusions • No universally best method for analyzing longitudinal data • Analysis must be tailored to the specific situation at hand • We can do better than LOCF etc. • Considering the missingness mechanism and the modeling philosophy provides the framework in which to choose an appropriate primary analysis and appropriate sensitivity analyses