key issues in analysis

1. Key Issues in Analysis Who gets analyzed? How are they grouped for analysis?

3. Intention-to-Treat An analysis which includes all randomized patients in the groups to which they were randomly assigned, regardless of their compliance with the entry criteria, regardless of the treatment they actually received, and regardless of subsequent withdrawal from treatment or deviation from protocol.

4. Key Points Deviation from intent-to-treat can result in bias from not comparing like with like. This method of analysis needs to be firmly in mind when designing the study, e.g., realistic estimates of treatment effect that account for non-adherence. There are practical difficulties in carrying out a strict intention to treat analysis and a strict per protocol analysis.

5. Arguments for Intention to Treat Consistent with randomization � get the right significance probability for hypothesis testing. Addresses the question of practical interest � a comparison of treatment policies. If the objective is to understand the implication of using a specific intervention in practice, this is the right analysis (e.g., non-adherence is a consequence of using a strategy in practice).

6. Arguments for Per Protocol Analysis Better estimate of pure pharmaceutical effect of treatment (i.e., including non-compliers dilutes the treatment difference). The relevant question is whether the treatment can work when used as intended, e.g., is it effective among patients who can tolerate it?

7. Obstacles to Intention to Treat Losses to follow-up Missing data

8. Obstacles to Per Protocol Analysis Defining adherence to treatment What is an acceptable level of adherence and how do you measure it?

9. Treatment Received Advantage: Undiluted treatment effect Disadvantage: Biased comparison of groups - can�t correct

10. Intention-to-Treat Advantage: Comparability of treatment groups; no bias resulting from exclusions. Disadvantage: Possible dilution of treatment effect; loss of power unless sample size was increased to account for it.

11. Intent-to-Treat May be More Powerful Not only larger sample size, but� If the treatment under study has an effect even after discontinuation (e.g., disease progression slowed, lingering pharmacologic effect)

12. ICH Guidelines � Full Analysis Set Exclusions may occur for failure to meet major entry criteria, failure to take at least one dose of medication, and for lack of any data after randomization Exclusion of ineligibles may only occur if: Criterion measured prior to randomization Eligibility can be objectively assessed There equal scrutiny for all patients All violations of specific type are excluded

13. ICH Guidelines � Per Protocol SetTypical Criteria Completion of pre-specified minimum exposure to treatment Availability of measurements or primary outcomes Absence of major protocol violations

15. Examples of Eligibility Errorsin AIDS Trials Liver enzyme tests are mixed up for Patient X and Patient Y; 2 weeks after randomization it is determined study drugs are contraindicated for Patient X Patients have CD4+ cell counts mixed up and the wrong patient is randomized. Patient X is randomized and is discovered 4 weeks later to be HIV negative Qualifying lab measurements made 45 days before randomization instead of within 30 days

17. What do you do about eligibility errors? Document them Determine whether it is safe for patients to continue treatment. If safe, assess whether patient should be allowed to continue treatment. Follow the patient like other randomized patients so that an intent to treat analysis can be carried out.

18. Examples of �Adherence� Problemsin AIDS Trials Patient X reports taking a study medication which is not allowed by the protocol Patient X dies after randomization but before study drug is picked up from pharmacy Patient X quits taking study treatment 2 weeks after randomization because she decides he does not want to participate in a placebo controlled study Patient X quits taking study drug 8 weeks after randomization due to side effects Patient X stops taking study drug before outcome assessment because their condition is worsening. Patient X is randomized twice because he did not like the first assignment

19. DEFINITE Study No. patients 229 229 No. implanted 29 225* No. lost None reported

20. COMPANION Study

21. Numbers of Survivors and Deaths 2 Years after Allocation to CABS or Medical Treatment

22. Two-year Mortality Rates as Calculated by 3 Methods

23. Anturane TrialRef. NEJM 1980;302:250-6 Total 813 74 816 89 0.28 Ineligible patients 38 10 33 4 Eligible patients 775 64 783 85 0.10 Nonanalyzable deaths 20 23 Analyzable deaths 44 62 0.08 Analyzable cardiac deaths 43 62 0.06 Analyzable sudden cardiac 22 37 0.04 Analyzable sudden cardiac deaths in 1st 6 months 6 24 0.003

24. Coronary Drug ProjectMortality Results No. patients 1103 2789 No. deaths in 5 years 221 583 Percent dead 20.0 20.9

25. Coronary Drug Project �Adherence to Clofibrate(3 capsules, 3 times per day)

26. Coronary Drug ProjectMortality According to Adherence to Clofibrate <80% 24.6 80%+ 15.0 Overall 20.0

27. The Obvious, But Na�ve, Solution Percent dead 15.0 20.9

29. Coronary Drug Project Mortality According to Adherenceto Clofibrate and Placebo <80% 24.6 28.2 80%+ 15.0 15.1 Overall 20.0 20.9

30. Deviations from Intent-to-Treat May Not be Obvious Patients who permanently stop study treatment (data collection should continue for ITT but often does not) Intent-to-treat for primary but not secondary outcomes Losses to follow-up (including competing events) Withdrawal of consent Missing data - minimize this with design, e.g., event-driven versus visit driven data collection, choice of investigators, and patient consent process

31. Some Protocols Define Situations When Patients Should No Longer Be Followed: Off Study Did not start treatment Ineligible Unacceptable toxicity Disease progression Incarceration Lost to follow-up Withdrawal of consent

32. Summary / Recommendations Primary analysis should be intent-to-treat (need to continue collecting data to do this right) It is appropriate to carry out secondary �per protocol� analyses but these have to be interpreted with caution. For analyses which are not intent-to-treat it is often difficult/impossible to quantify bias If exclusions after randomization are to be made as part of secondary �per protocol� analyses, they should be specified in the protocol