Download
practical statistical reasoning in clinical trials for non statisticians n.
Skip this Video
Loading SlideShow in 5 Seconds..
PRACTICAL STATISTICAL REASONING IN CLINICAL TRIALS FOR NON-STATISTICIANS PowerPoint Presentation
Download Presentation
PRACTICAL STATISTICAL REASONING IN CLINICAL TRIALS FOR NON-STATISTICIANS

PRACTICAL STATISTICAL REASONING IN CLINICAL TRIALS FOR NON-STATISTICIANS

157 Views Download Presentation
Download Presentation

PRACTICAL STATISTICAL REASONING IN CLINICAL TRIALS FOR NON-STATISTICIANS

- - - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

  1. PRACTICAL STATISTICAL REASONING INCLINICAL TRIALS FOR NON-STATISTICIANS Presented on November 14, 2012 by: • Paul Wakim, PhD • Abigail G. Matthews, PhD Clinical Trials Network National Institute on Drug Abuse ─ National Institutes of Health ─ U.S. Department of Health and Human Services

  2. Presenters • Abigail G. Matthews, PhD Biostatistician NIDA CTN Data and Statistics Center EMMES Corporation • Paul Wakim, PhD Senior Mathematical Statistician NIDA CCTN

  3. Outline: • Introduction • Trial Design • Q&A • Analysis Plan • Trial Monitoring and Interim Analyses • Q&A • Primary Analysis • Subgroup Analyses • Q&A

  4. Goals • Improve communication between researchers and biostatisticians • Importance of collaboration • Role of the biostatistician in clinical trials research • Basic statistical concepts • Discussion with participants from all backgrounds NO technical information, and NO formulas

  5. Lack of Communication

  6. Lack of Communication

  7. Why is Communication So Important? • Biostatisticians cannot: • Propose research questions • Be subject-matter experts • Design a study without clinical input • Design statistical analyses without clinical input • Interpret results and place in clinical context • Investigators cannot: • Be knowledgeable about all statistical issues involved in sample size estimation and development of analysis plans • Implement the often complex statistical analyses involved in clinical trials • Interpret statistical analyses » Without communication, neither can do their jobs

  8. Role of a Biostatistician • Work with investigators on trial design • Insure design will yield results that answer research question of interest • Aid in defining primary outcome • Conduct sample size calculations • Write appropriate sections of protocol • Develop analysis plan • Identify interim analyses and procedures for trial monitoring • Design primary analysis • Specify methods for subset analyses, sensitivity analyses and other exploratory analyses

  9. Role of a Biostatistician (cont’d) • Implement trial monitoring and interim analyses • Develop monitoring reports for investigators, site staff, and sponsor, for example • Recruitment rates • Demographics • Availability of primary outcome • Prepare and present DSMB reports for open and closed sessions • Conduct interim analyses such as efficacy, futility and sample size re-estimation • Aid in preparation of IND Annual Reports

  10. Role of a Biostatistician (cont’d) • Implement analysis plan • Aid in creation of the final/clinical study report • Tables • Figures • Interpretation • Perform any additional analyses for manuscripts • Contribute to IND reports as necessary • Develop novel statistical methodologies to analyze clinical trial data more appropriately (if necessary)

  11. TRIAL DESIGN

  12. Trial Design • Basic designs • Primary outcome measure (a.k.a. primary endpoint) • Sample size and power analysis Clinical Trials Network National Institute on Drug Abuse ─ National Institutes of Health ─ U.S. Department of Health and Human Services Clinical Trials Network National Institute on Drug Abuse ─ National Institutes of Health ─ U.S. Department of Health and Human Services

  13. Basic Design: Superiority Clinical hypothesis: Experimental treatment is more effective than the control treatment Statistical hypotheses: Null hypothesis H0: Experimental – Control = 0 Alternative hypothesis H1: Experimental – Control ≠ 0 We expect (hope) to reject H0 in favor of H1 Clinical Trials Network National Institute on Drug Abuse ─ National Institutes of Health ─ U.S. Department of Health and Human Services

  14. Basic Design: Superiority Superior Inconclusive Inconclusive Inferior 95% confidence intervals around the difference: Experimental – Control High numbers (on the right) represent good outcome Diff.= 0 Based on Piaggio 2006

  15. Basic Design: Non-Inferiority Clinical hypothesis: Experimental treatment is not less effective than the control treatment Statistical hypotheses: Null hypothesis H0: Experimental – Control < – M Alternative hypothesis H1: Experimental – Control ≥ – M We expect (hope) to reject H0 in favor of H1 Clinical Trials Network National Institute on Drug Abuse ─ National Institutes of Health ─ U.S. Department of Health and Human Services

  16. Basic Design: Non-Inferiority Inconclusive Non-inferior(?) Inconclusive(?) Superior Non-inferior Inferior Non-inferior 95% confidence intervals around the difference: Experimental – Control High numbers (on the right) represent good outcome Diff.= -M Diff.= 0 Based on Piaggio 2006

  17. Basic Design: Equivalence Clinical hypothesis: Experimental treatment is as effective as the control treatment Statistical hypotheses: Null hypothesis H0: Experimental – Control < – M or Experimental – Control > + M Alternative hypothesis H1: – M ≤ Experimental – Control ≤ + M We expect (hope) to reject H0 in favor of H1 Clinical Trials Network National Institute on Drug Abuse ─ National Institutes of Health ─ U.S. Department of Health and Human Services

  18. Basic Design: Equivalence Inferior Inconclusive Inconclusive(?) Equivalent(?) Equivalent(?) Superior Equivalent Inconclusive(?) 95% confidence intervals around the difference: Experimental – Control High numbers (on the right) represent good outcome Diff.=-M Diff.=+M Diff.=0 Based on Piaggio 2006

  19. Primary Outcome Measure (aka primary endpoint) • Clinically meaningful • Simple vs. composite Clinical Trials Network National Institute on Drug Abuse ─ National Institutes of Health ─ U.S. Department of Health and Human Services Clinical Trials Network National Institute on Drug Abuse ─ National Institutes of Health ─ U.S. Department of Health and Human Services

  20. Three “Deadly Sins” in Measuring Clinical Trial Outcomes Treating ordinal data as categorical Creating dichotomies from continuous data Using change from baseline From Stephen Senn, 2011 Clinical Trials Network National Institute on Drug Abuse ─ National Institutes of Health ─ U.S. Department of Health and Human Services Clinical Trials Network National Institute on Drug Abuse ─ National Institutes of Health ─ U.S. Department of Health and Human Services

  21. Sample Size & Power Analysis • What the biostatistician needs and why: • Number of treatment groups • Superiority or non-inferiority or equivalence • One-sided or two-sided • Expected drop-out rate Clinical Trials Network National Institute on Drug Abuse ─ National Institutes of Health ─ U.S. Department of Health and Human Services Clinical Trials Network National Institute on Drug Abuse ─ National Institutes of Health ─ U.S. Department of Health and Human Services

  22. Expected Drop-Out Rate (amount of missing primary data) Expected drop-out rate   Sample size  Expected drop-out rate   Sample size  Increase the sample size to account for the expected amount of missing data in the primary analysis Clinical Trials Network National Institute on Drug Abuse ─ National Institutes of Health ─ U.S. Department of Health and Human Services Clinical Trials Network National Institute on Drug Abuse ─ National Institutes of Health ─ U.S. Department of Health and Human Services

  23. Sample Size & Power Analysis • What the biostatistician needs and why: • Number of treatment groups • Superiority or non-inferiority or equivalence • One-sided or two-sided • Expected drop-out rate • Smallest meaningful clinical difference to detect Clinical Trials Network National Institute on Drug Abuse ─ National Institutes of Health ─ U.S. Department of Health and Human Services Clinical Trials Network National Institute on Drug Abuse ─ National Institutes of Health ─ U.S. Department of Health and Human Services

  24. Smallest Meaningful Clinical Difference to Detect Difference to detect   Sample size  Difference to detect   Sample size  Clinical Trials Network National Institute on Drug Abuse ─ National Institutes of Health ─ U.S. Department of Health and Human Services Clinical Trials Network National Institute on Drug Abuse ─ National Institutes of Health ─ U.S. Department of Health and Human Services

  25. Sample Size & Power Analysis • What the biostatistician needs and why: • Number of treatment groups • Superiority or non-inferiority or equivalence • One-sided or two-sided • Expected drop-out rate • Smallest meaningful clinical difference to detect • Alpha, aka chance of Type I error, e.g. 5% Clinical Trials Network National Institute on Drug Abuse ─ National Institutes of Health ─ U.S. Department of Health and Human Services Clinical Trials Network National Institute on Drug Abuse ─ National Institutes of Health ─ U.S. Department of Health and Human Services

  26. Alpha aka probability of making a Type I error Non-technical definition (superiority trial): Chance of concluding that the experimental treatment is (more) effective when in fact it is not Technical definition: Probability of rejecting H0 when H0 is true Different perspectives: FDA, Pharmaceutical company Bottom line: Most commonly used value for α: 0.05 (two-sided) Clinical Trials Network National Institute on Drug Abuse ─ National Institutes of Health ─ U.S. Department of Health and Human Services Clinical Trials Network National Institute on Drug Abuse ─ National Institutes of Health ─ U.S. Department of Health and Human Services

  27. Alpha aka probability of making a Type I error Alpha   Sample size  Alpha   Sample size  Clinical Trials Network National Institute on Drug Abuse ─ National Institutes of Health ─ U.S. Department of Health and Human Services Clinical Trials Network National Institute on Drug Abuse ─ National Institutes of Health ─ U.S. Department of Health and Human Services

  28. Sample Size & Power Analysis • What the biostatistician needs and why: • Number of treatment groups • Superiority or non-inferiority or equivalence • One-sided or two-sided • Expected drop-out rate • Smallest meaningful clinical difference to detect • Alpha, aka chance of Type I error, e.g. 5% • Power to detect an effect, e.g. 80% or 90% Clinical Trials Network National Institute on Drug Abuse ─ National Institutes of Health ─ U.S. Department of Health and Human Services Clinical Trials Network National Institute on Drug Abuse ─ National Institutes of Health ─ U.S. Department of Health and Human Services

  29. Power to Detect an Effect Non-technical definition (superiority trial): Chance of concluding that the experimental treatment is (more) effective when in fact it is Technical definition: Probability of rejecting H0 when H0 is false (i.e. when H1 is true) Different perspectives: FDA, Pharmaceutical company Bottom line: Most commonly used value for power: between 0.80 & 0.90 Clinical Trials Network National Institute on Drug Abuse ─ National Institutes of Health ─ U.S. Department of Health and Human Services Clinical Trials Network National Institute on Drug Abuse ─ National Institutes of Health ─ U.S. Department of Health and Human Services

  30. Power to Detect an Effect Power   Sample size  Power   Sample size  Clinical Trials Network National Institute on Drug Abuse ─ National Institutes of Health ─ U.S. Department of Health and Human Services Clinical Trials Network National Institute on Drug Abuse ─ National Institutes of Health ─ U.S. Department of Health and Human Services

  31. Sample Size & Power Analysis • What the biostatistician needs and why: • Number of treatment groups • Superiority or non-inferiority or equivalence • One-sided or two-sided • Expected drop-out rate • Smallest meaningful clinical difference to detect • Alpha, aka chance of Type I error, e.g. 5% • Power to detect an effect, e.g. 80% or 90% • Variability of primary outcome measure Clinical Trials Network National Institute on Drug Abuse ─ National Institutes of Health ─ U.S. Department of Health and Human Services Clinical Trials Network National Institute on Drug Abuse ─ National Institutes of Health ─ U.S. Department of Health and Human Services

  32. Variability of Primary Outcome Measure Variability   Sample size  Variability   Sample size  Clinical Trials Network National Institute on Drug Abuse ─ National Institutes of Health ─ U.S. Department of Health and Human Services Clinical Trials Network National Institute on Drug Abuse ─ National Institutes of Health ─ U.S. Department of Health and Human Services

  33. Sample Size & Power Analysis • What the biostatistician needs and why: • Number of treatment groups • Superiority or non-inferiority or equivalence • One-sided or two-sided • Expected drop-out rate • Smallest meaningful clinical difference to detect • Alpha, aka chance of Type I error, e.g. 5% • Power to detect an effect, e.g. 80% or 90% • Variability of primary outcome measure • Correlation between measurements within the same cluster (aka Intra-Class Correlation or ICC) Clinical Trials Network National Institute on Drug Abuse ─ National Institutes of Health ─ U.S. Department of Health and Human Services Clinical Trials Network National Institute on Drug Abuse ─ National Institutes of Health ─ U.S. Department of Health and Human Services

  34. From Wikipedia Clinical Trials Network National Institute on Drug Abuse ─ National Institutes of Health ─ U.S. Department of Health and Human Services Clinical Trials Network National Institute on Drug Abuse ─ National Institutes of Health ─ U.S. Department of Health and Human Services

  35. From Wikipedia Clinical Trials Network National Institute on Drug Abuse ─ National Institutes of Health ─ U.S. Department of Health and Human Services Clinical Trials Network National Institute on Drug Abuse ─ National Institutes of Health ─ U.S. Department of Health and Human Services

  36. Correlation Between Measurements within the Same Cluster (e.g. repeated measures) Intra-class correlation   Sample size  Intra-class correlation   Sample size  Clinical Trials Network National Institute on Drug Abuse ─ National Institutes of Health ─ U.S. Department of Health and Human Services Clinical Trials Network National Institute on Drug Abuse ─ National Institutes of Health ─ U.S. Department of Health and Human Services

  37. Sample Size & Power Analysis • What the biostatistician needs and why: • Number of treatment groups • Superiority or non-inferiority or equivalence • One-sided or two-sided • Expected drop-out rate • Smallest meaningful clinical difference to detect • Alpha, aka chance of Type I error, e.g. 5% • Power to detect an effect, e.g. 80% or 90% • Variability of primary outcome measure • Correlation between measurements within the same cluster (aka Intra-Class Correlation or ICC) Clinical Trials Network National Institute on Drug Abuse ─ National Institutes of Health ─ U.S. Department of Health and Human Services Clinical Trials Network National Institute on Drug Abuse ─ National Institutes of Health ─ U.S. Department of Health and Human Services

  38. One Final Note About Sample Size • Cost, which has nothing to do with biostatistics, is most often a key factor in the final decision on sample size. Clinical Trials Network National Institute on Drug Abuse ─ National Institutes of Health ─ U.S. Department of Health and Human Services Clinical Trials Network National Institute on Drug Abuse ─ National Institutes of Health ─ U.S. Department of Health and Human Services

  39. Questions?

  40. ANALYSIS PLAN

  41. Purpose • Identify primary outcome measure a priori • Spell out analytic methods a priori • Remove criticism of data driven analyses In CTN: • Analysis plan must be finalized before data lock • Developed by DSC, but approved by Lead Node

  42. Key Components of an Analysis Plan • Population to analyze: Intent-to-Treat (ITT) vs. per-protocol (PP) analysis • Statistical test or model for primary outcome • Adjustment for multiple comparisons • Handling of missing data • Handling of outliers • Interim analyses • Sensitivity analyses • Secondary and subgroup analyses

  43. 1. Population Analyzed Intent-to-Treat (ITT) • ALL randomized participants are analyzed • “Once randomized, analyzed” • Participants with completely missing data are included Per-Protocol (PP) • Analyze a select subset of randomized participants as stated in protocol • For example, • Only participants who had at least 80% of study medication • Only participants who attended at least 50% of the expected TAU sessions

  44. 2. Statistical Test or Model Test • What statistical test should be used? • What time points are of interest? • Measure of treatment effect Modeling • Must have parameter(s) to test primary outcome and hypothesis • Longitudinal model/repeated measures, single time point or composite score • Consider inclusion of stratification factors, time by treatment interactions, additional covariates (e.g. level of baseline substance use) • Potential site effects

  45. 3. Adjustment for Multiple Comparisons Why? • Need to control the study-wise false positive rate (type I error) • If perform 100 tests, 5% will be significant by chance if α = 0.05 When? • More than one primary outcome • Multiple treatment comparisons (e.g. multiple doses vs. placebo) • Multiple time points of interest, but not longitudinal model

  46. 3. Adjustment for Multiple Comparisons (cont’d) How? • Bonferroni • Very conservative, but simple • Split type I error rate equally between all statistical tests • Stepwise procedures

  47. 4. Handling of Missing Data Based on the first 24 multi-site CTN trials on substance abuse conducted between 2001 and 2010, the percent of missing data for the primary outcome measure ranged from 2% to 60% (Wakim 2011). There are many methods of handling missing data with varying levels of complexity, e.g., • Simple: imputing missing abstinence data as positive • Complex: pattern mixture models

  48. Types of Missing Data • Missing Completely at Random (MCAR) • Whether an observation is missing or not is completely random • Participant does not attend visit due to snow storm • Missing at Random (MAR) • Unobserved data can be explained by observed data • Most common statistical methods will yield valid results under MAR • Missing Not at Random (MNAR) • Unobserved data cannot be explained by observed data • Participant does not attend study visit because they were using • Standard statistical methods cannot be used

  49. 5. Handling of Outliers An outlier is a value that is so far from the others that it appears to have come from a different population. The presence of outliers can invalidate many statistical analyses. Motulsky 2010

  50. 6. Interim Analyses • Specify type of interim analyses to be performed • Sample size re-estimation • Futility • Efficacy • Specify when analyses will be performed • e.g., sample size re-estimation when 50% of participants have completed active treatment • Specify frequency of these analyses • e.g., DSMB meetings every six months