PLANNING AND PERFORMING A RANDOMIZED CONTROLLED CLINICAL TRIAL

1. PLANNING AND PERFORMING A RANDOMIZED CONTROLLEDCLINICAL TRIAL

2. REPRODUCIBILITY IN RESEARCH • “Growing alarm about results that cannot be reproduced” Nature Supplement, Challenges in Irreproducible Research, October 7, 2015 • “Reproducibility, rigor, transparency and independent verification are cornerstones of the scientific method” NIH-Science-Nature Workshop on Reproducibility and Rigor of Preclinical Research Nature 2014;515:7

3. ENHANCING REPRODUCIBILITY AND RIGOR IN CLINICAL RESEARCH • Study design of high methodologic quality • Minimizes bias: better estimate of “truth”

4. ENHANCING REPRODUCIBILITY AND RIGOR IN CLINICAL RESEARCH • Study design of high methodologic quality • Minimizes bias: better estimate of “truth” • Transparent (full and clear) presentation of methods and analyses • Enables assessment of methods and results • Allows duplication of study, re-analysis of results

5. ENHANCING REPRODUCIBILITY AND RIGOR IN CLINICAL RESEARCH • Study design of high methodologic quality • Minimizes bias: better estimate of “truth” • Transparent (full and clear) presentation of methods and analyses • Enables assessment of methods and results • Allows duplication of study, re-analysis of results • Registration before trial begins • Prevents changing design or pre-specified outcomes/analyses without explanation

6. ENHANCING REPRODUCIBILITY AND RIGOR IN CLINICAL RESEARCH • Study design of high methodologic quality • Minimizes bias: better estimate of “truth” • Transparent (full and clear) presentation of methods and analyses • Enables assessment of methods and results • Allows duplication of study, re-analysis of results • Registration before trial begins • Prevents changing design or pre-specified outcomes/analyses without explanation • Trials reported per international standards • All appropriate elements included

7. REPRODUCIBILITY ISSUESPreclinical Research Most Susceptible • Clinical trials seem to be less at risk because already governed by regulations that stipulate rigorous design and independent oversight (randomization, blinding, power estimates, pre-registration in standardized, public databases, oversight by IRBs and DSMBs) and adoption of standard reporting elements Collins FS, Tabak LA. Nature 2014;505:612

8. CLINICAL TRIALS

9. LEVELS OF EVIDENCE FOR CLINICAL RESEARCH STUDY DESIGNS Schillaci et al. Hypertension. 2013;62:470

10. OBSERVATIONAL VS. RANDOMIZED TRIALS

11. OBSERVATIONAL VS. RANDOMIZED TRIALS • “There are known knowns; there are things we know we know. We also know there are known unknowns; that is to say, we know there are some things we do not know. But there are also unknown unknowns--the ones we don’t know we don’t know.” - Donald Rumsfeld

12. OBSERVATIONAL VS. RANDOMIZED TRIALS • Observational Studies • Distribution of baseline factors that may impact outcome (e.g., age, meds, comorbidities) vary in study groups • Known knowns: known to impact outcome, collected • Statistical adjustment, matching • Known unknowns: known to impact outcome, can’t be collected • Unknown unknowns: don’t know impact outcome, not collected • Randomized Trials • All factors (known and unknown) that may impact outcome equally distributed among study groups

13. RANDOMIZED CONTROLLED TRIAL • Only randomized trials of sufficient size can adequately control for known and unknown confounding variables to minimize bias • No substantive differences between groups except study intervention (randomly assigned) • Difference between groups in predefined outcome can be attributed to the intervention being studied Hennekens & Buring Epidemiology in Medicine. 1987

14. DO WE ALWAYS NEED AN RCT TO DOCUMENT BENEFIT OF AN INTERVENTION?

15. DO WE ALWAYS NEED AN RCT TO DOCUMENT BENEFIT OF AN INTERVENTION? • “Perception that parachutes are a successful intervention based largely on anecdotal evidence” • No RCTs identified in systematic review • Under exceptional circumstances apply common sense BMJ 2003;327:1459

16. RCT MAY NOT BE POSSIBLE OR PRACTICAL • Not ethical/possible to assign intervention • Cigarette smoking and lung cancer • H. pylori infection and ulcers • Impractically large sample size • Very low-incidence outcome • e.g., rare side effect of medication • Impractically long duration • Outcome requires many years to develop • e.g., development of cancer

17. RANDOMIZED CONTROLLED TRIALSFirst Steps

18. RANDOMIZED CONTROLLED TRIALSFirst Steps • Clinically relevant question • Greatest impact if limited information or high variability in care or outcomes • Can be answered by properly designed RCT • Feasible to perform at your center(s)

19. RANDOMIZED CONTROLLED TRIALSFirst Steps • Clinically relevant question • Greatest impact if limited information or high variability in care or outcomes • Can be answered by properly designed RCT • Feasible to perform at your center(s) • Systematic review • Identify available information • Justify importance of question • Help design study

20. RANDOMIZED CONTROLLED TRIALSFirst Steps • Define key elements of study • Population • Intervention • Comparator • Outcome • State primary hypothesis • Expected result for primary outcome in population • e.g., in patients with cirrhosis fewer deaths with new intervention vs. control

21. STUDY DESIGN

22. RANDOMIZATION

23. RANDOMIZATION • Generate sequence of allocation • Computer generated, random numbers table • Randomize in blocks • Other features of randomization include • Concealed allocation • Non-manipulable allocation schedule • Off-site randomization schedule ideal • Stratification • Most important factor(s) that may impact endpoint

24. CONCEALED ALLOCATION • Concealed allocation is an extension of randomization • When obtaining informed consent to enroll a patient into a trial, the investigator does not know if the next patient will get new treatment or control

25. CONCEALED ALLOCATION • RCT comparing new therapy vs. placebo for abdominal pain in irritable bowel syndrome • Investigator interviews the next eligible patient, who complains of long-term severe, unrelenting symptoms that have never responded to previous medical therapy • Next patient to enter trial will get placebo

26. CONCEALED ALLOCATION • Investigator thinks that placebo is unlikely to relieve abdominal pain in this patient • Investigator may subconsciously try to convince patient not to enroll in the trial • Consequence: patients with severe abdominal pain will NOT be evenly divided between new therapy and placebo groups

27. STRATIFICATION • To assure baseline factor(s) that impact study outcome equally distributed in study groups • Especially useful in smaller trials • Choose factor(s) that have greatest impact on primary outcome • Aspirin use in MI study • Separate randomization schedules for patients with and without factor

28. RANDOMIZATIONBlock Size (e.g., 4, 10, 20; Random) • Assures equal number in each study arm for every successive block of patients enrolled • Prevents unequal numbers in study arms • Prevents differences in distribution over time • e.g., study intervention mostly early, comparator mostly later • Disadvantage: if block size figured out, next allocation may be predictable (unconcealed)—selection bias • Larger block sizes; random sequences of block sizes

29. BLINDING

30. BLINDING • Not known if subject getting new therapy or control • Subjects • Healthcare providers making management decisions • Investigators collecting/analyzing data • Prevents bias in management decisions and in assessment of outcomes by subject or investigator • Knowledge receiving placebo or active drug may influence • Administration of another therapy that my impact outcome • Assessment of symptoms, signs (endpoints)

31. BLINDING • Identical appearing therapies • Real vs. sham surgery/procedure • Surgical team uninvolved in further care/assessment • Double-dummy • Subjects receive identical active and control therapy together • Side effect of a therapy may unblind subjects • Assess whether unblinded

32. Treatment Treatment Treatment Placebo Placebo Placebo TREATMENT EFFECT OVERESTIMATED WITHOUT RANDOMIZATION AND BLINDING 35 30 25 20 15 Case Fatality Rates 10 5 0 ConcealedAllocation; Blinded Not Randomized Randomized Chalmers, et al. N Engl J Med 1983; 309: 1358

33. PATIENT POPULATION • Inclusion and exclusion criteria • Broad: exclude few, more generalizable • Restricted: exclude many, less generalizable • Prospectively screen consecutive patients with condition of interest • Skipping patients may introduce bias • Screening log • Subjects screened, but not enrolled • Brief characteristics, reason not enrolled • ?Differences from those enrolled • Is study generalizable?

34. STUDY INTERVENTIONS • Define all aspects of study interventions so uniform in trial, able to be reproduced • Control • Placebo control • Best to define efficacy of study therapy • May not be ethical, practical • Can’t withhold standard care if documented effective • Active control (a current standard) • Hypothesis: new therapy superior, non-inferior, or equivalent to active control

35. ENDPOINTS • What do you want to achieve with the new intervention • Primary endpoint • Additional endpoints • Surrogate vs. clinical endpoints • Surrogate endpoint: measure of treatment effect felt likely to correlate with clinical endpoint • e.g., gastric acid inhibition for ulcer prevention

36. CLINICALLY MEANINGFUL ENDPOINTS PREFERRED • Which study endpoint would alter practice? • Lab test (CRP) or clinical outcome (death) • Studies of intermediate/surrogate endpoints may indicate areas for further research, but generally don’t alter patient management • Some surrogate endpoints are accepted as “true” indicators of clinical outcomes • e.g., blood pressure, cholesterol, colon polyps

37. SAMPLE SIZE DETERMINATION

38. SAMPLE SIZE DETERMINATIONAssumptions for Superiority Study • Primary endpoint result for the intervention • Primary endpoint result for the comparator • Assumptions based on available data, clinical judgment • Hypothesized difference should be clinically meaningful, realistic • α • p = 0.05 • probability of finding difference when doesn’t exist (type I) • Power (1 – β) • Probability of finding difference when does exist • e.g., 80%, 90% • β: probability of not finding a difference when does exist (type II)

39. SAMPLE SIZE DETERMINATIONWhy Did They Stop the Study When They Did? • RCT: Wonderdrug vs. placebo in pancreatic cancer • Primary endpoint: 5-yr survival • Wonderdrug: 50% • Placebo: 10% • P-value (α = 0.05) • 90% power to detect 40% difference between Wonderdrug and placebo • 52 patients required (if 1:1 randomization)

40. SAMPLE SIZE DETERMINATIONAssumptions for Non-Inferiority Study • Determine non-inferiority margin • Clinical: maximal difference that would be considered clinically non-inferior • Not unacceptably worse than the control • Statistical: maintain benefit above placebo • Control is 20% more efficacious than placebo • Margin of 10% retains half control treatment effect • Margin (e.g., control – test drug = 3%) less than upper bound of CI of difference observed in study • Difference = 0% (95% CI -5% to 5%): Not non-inferior • Difference = 0% (95% CI -1% to 1%): Non-inferior

41. SAMPLE SIZE DETERMINATIONNon-Inferiority Study • Determine non-inferiority margin • Clinical: maximal difference that would be considered clinically non-inferior • Not unacceptably worse than the control • Statistical: maintain benefit above placebo • Control is 20% more efficacious than placebo • Margin of 10% retains half control treatment effect • Margin (e.g., control – test drug = 3%) less than upper bound of CI of difference observed in study • Difference = 0% (95% CI -5% to 5%): Not non-inferior • Difference = 0% (95% CI -1% to 1%): Non-inferior • Potential reasons to do non-inferiority study • New intervention has some other advantage that would recommend it if efficacy similar (non-inferior) to current standard therapy • e.g., cheaper, safer, easier to use (pill vs. enema), more readily available (oral rehydration vs. IV fluids); commercial

42. NON-INFERIORITY STUDY: FDA EXAMPLE • New thrombolytic (R) vs. approved therapy (S) • Outcome: Mortality • New thrombolytic must retain ≥50% benefit of approved therapy to be acceptable alternative • Mortality difference S vs. placebo • 2.6% (lower bound 95% CI = 2.1%) • Study has to rule out 1.05% increase in mortality with R compared to S • 95% CI of difference in mortality for R vs. S < 1.05% • Accept 1.05% increase as not unacceptably worse http://www.fda.gov/downloads/Drugs/GuidanceComplianceRegulatoryInformation/Guidances/UCM202140.pdf

43. IS THE SAMPLE SIZE FEASIBLE • Review medical records at study center(s) to • Determine number who meet enrollment criteria • Confirm assumptions about outcomes • “Preparatory to research” review doesn’t require IRB approval of the protocol • “This type of access is limited to a review of data to assist in formulating a hypothesis, determining the feasibility of conducting the study . . . or other similar uses that precede the development of an actual protocol.” • Submit Request for Access form

44. POPULATIONS FOR ANALYSIS

45. Intention to Treat Analysis • All randomized patients are included in final data analysis • Per Protocol Analysis • Only patients who complete the trial according • to protocol are analyzed POPULATIONS FOR ANALYSIS

46. POPULATIONS FOR ANALYSIS • Intention-to-treat population • All patients randomized regardless of follow-up or receipt of study intervention • Per-protocol population excludes those who • Did not receive sufficient study intervention • Did not return for adequate follow-up • Had major violations of inclusion criteria • e.g., did not have the disease being studied • Had major violations during the study • Took non-study PPI during PPI vs. placebo study

47. INTENTION-TO-TREAT ANALYSISExample • Comparison of radiology procedure (TIPS) vs. drug (β-blocker) for prevention of recurrent variceal bleeding with death as the primary endpoint • If a patient is randomized to get TIPS and dies from bleeding before the procedure can be done, should the patient be included in the final data analysis?

48. POPULATIONS FOR ANALYSIS • Choose the most conservative analysis • Less likely to favor intervention, be overly optimistic • Superiority study • Per-protocol assesses intervention under optimal circumstances (not real world, ignores study quality) • e.g., excluded if non-adherence, protocol violations, drop-out • ITT avoids bias to treatment difference and superiority • Non-inferiority study • ITT can bias to no treatment difference (non-inferiority) • e.g., non-adherence, drop-outs, misclassified subjects/endpoints • Per protocol analysis should be included

49. COMPLETE FOLLOW-UP OF PATIENTSAnother Requirement for High Methodologic Quality • If numerous patients are lost to follow-up, results of the trial may not be accurate • Predefine method to deal with such patients • Last observation carried forward • Imputation methods • Re-calculate results assuming that patients lost to follow-up in new treatment group had bad outcome and patients lost to follow-up in control group had good outcome

50. RECRUITMENT AND RETENTION • Engagement, communication with participants before and throughout trial • Brochures, ads, social media, phone/text/email/websites • Reminders for study personnel and participants • Benefits of participation for subjects • Societal, personal, financial reimbursement for time • Benefits of participation for research personnel • Academic (e.g., authorship), financial • Identify and minimize barriers to participation • Easy access to study personnel and activities • Participation as non-onerous as possible