A Beginner's Guide to Methodological and Statistical Issues in HIV Research: Session 5 - Design of RCTs for HIV Treatmen

A beginners guide to some of the methodological and statistical issues in HIV research Session 5: The design of RCTs of treatments for HIV infection Caroline Sabin Reader in Medical Statistics and Epidemiology Department of Primary Care and Population Sciences, RF&UCMS

What is a clinical trial? Any form of planned experiment which involves patients and is designed to find the most appropriate treatment for a particular medical condition

Types of trials (clinical) Phase I studies Focus on safety rather than efficacy Dose-escalation studies, studies of drug metabolism and bioavailability Usually based on small numbers of subjects, often healthy volunteers Phase II studies Initial investigation for clinical effect. Small-scale studies into effectiveness and safety of drug. Phase III studies Full-scale treatment evaluation. Comparison to standard therapy (if one exists) or placebo Phase IV trials Post-marketing surveillance. Monitoring for adverse effects. Long-term studies of morbidity and mortality. Promotion exercises

Topics already covered in first session • Control groups • Randomisation • Blinding • Parallel vs. cross-over trials • The limitations of RCTs

Topics to be covered today • Why do we need a control group? • Why do we need randomisation? • The protocol • Defining endpoints (primary and secondary endpoints, clinical vs surrogate endpoints) • How to deal with ‘protocol violations’ (patients who drop out of the study and missing data) • Approaches to analysis (ITT, as treated) • Subgroup and interim analyses

Why do we need a control group • Early medical developments were usually so great that controls weren’t always needed (eg. trials of anaesthetics, first trials of antibiotics etc.) • However, most developments these days are more modest and some form of control group is now essential

Why do we need a control group Silverman, 1985 – Epidemic of retrolental fibroplasia in babies • Uncontrolled trials suggested that treatment with adrenocorticotrophic hormone had a 75% success rate • After controlled trials were finally carried out, it was found that 75% of infants return to normal without treatment • Identification of true cause of epidemic (oxygen to premature babies) was delayed

Why do we need a control group • Uncontrolled trials may give a distorted view of a new therapy • Patients may improve over time, even without treatment – thus, any improvement cannot necessarily be attributed to treatment • Patients selected for treatment may be less seriously ill than those not selected for treatment which may overestimate the benefits of new therapy • Patients in clinical trials generally do better than patients on same treatment who are not in trials

Which control group? - the use of historical or non-randomised controls Characteristics of patients • Controls less likely to have clearly defined criteria for inclusion/exclusion • May have been a change in the type of patient eligible for treatment, or prognosis may have changed over time • Investigator may have been more restrictive in choice of patients for the trial, then when treating patients in the past

Which control group? - the use of historical or non-randomised controls Experimental environment • Quality of recorded data may not be as good • Definitions of response may differ between groups (eg. viral load endpoints) • Ancillary care may improve in a trial (eg. adherence support, support for toxicities etc.) Thus, treatment and control groups may differ with respect to many features other than treatment, and so we cannot attribute any difference in outcome to the new treatment

What is randomisation? • Allocation of patients to treatments is determined by chance • Randomised trials provide most efficient trial design (ie. they are the most powerful) as they ensure that any factors that may affect outcome will be distributed equally between the treatment groups • This allows any difference in treatment response to be attributed to the treatment • Removes impact of known confounding factors as well as unknown ones

Why do trials need to be randomised • Non-randomised trials have the potential to be seriously biased • If there are systematic differences between the patients in the treatment groups at the outset of the trial, then any differences in treatment response cannot necessarily be attributed to the new treatment • Eg. treatment comparisons in cohort studies

When can a randomised trial be done New treatment better than standard New treatment worse than standard ‘Equipoise’ • Who should have equipoise? • The doctors recruiting patients • The patients entering the trial • (is this true in reality?)

Other benefits of randomisation • Helps with blinding of trial (see later) • Prevents any conscious or subconscious selection bias, whereby doctor tends to put more (or less) severely ill patients in a particular treatment group • Beware of any approach to randomisation whereby clinicians may be able to establish treatment allocation prior to entry to the trial (eg. systematic allocation by date of birth, alternate allocation)

Selection of patients for a trial Discuss trial with patient and assess eligibility Obtain informed consent Formally enter patient into trial Randomise

Other benefits of randomisation (cont.) Example: Trial of anticoagulant therapy (Wright 1948) Patients admitted on odd days – anticoagulants Patients admitted on even days – placebo Anticoagulant therapy – n=589 Placebo – n=442

The protocol The ‘workshop manual’ for the trial. Will contain many or all of the following: • Background, aims and objectives • Trial design • Patient selection – inclusion/exclusion criteria • Treatment schedules • Monitoring • Registration, randomisation and blinding • Methods of patient evaluation • Patient consent • Size of study • Plans for dealing with protocol deviations • Plans for statistical analysis • Ethical approval and administrative matters

Selection of patients for a trial • A trial should have explicit inclusion criteria and exclusion criteria – precise definitions of who can be included in the study • Patients should be broadly representative of some future group of patients to whom the trial may be applied • BUT – patients in trials are not necessarily a random selection of all HIV+ve individuals (unlikely to be the case)

Evaluation of response – the primary endpoint • In any trial we need to define (preferably) a single primary endpoint that captures the key effects of treatment on the patient • Primary endpoint is usually related to efficacy • If results from different endpoints are inconsistent, the primary endpoint will be the one on which any decisions about the value of the drug will be mainly based

Evaluation of response – secondary endpoints • In addition to the primary endpoint, we may also define any number of secondary endpoints • These are often related to toxicity or quality of life, or may be other measures of efficacy not captured by the primary endpoint

Definitions of endpoints – example Abacavir substitution for nucleoside analogs in patients with HIV lipoatrophy.Carr A et al. JAMA (2002); 288: 207-215. Primary endpoint: Mean change in limb fat mass measured by DXA at week 24 Secondary endpoints: Adverse events Anthropometry Total and central fat mass Biochemical, lipid, and glycemic measurements Viral load CD4 count Quality of life

Defining an endpoint • In most trials patients are monitored very regularly (eg. every 4 weeks after randomisation • Tempting to compare treatments at each time point - however, this is not advisable because of problems with multiple testing and the fact that the tests are not independent • Thus, must select a single time point for assessment of the primary endpoint (eg. 24 weeks or 48 weeks) • Treatments should be formally compared at that timepoint only

Clinical vs. surrogate endpoints • We are usually most interested in the effect of a new treatment on a clinical outcome (eg. new AIDS events or death) • However, currently, trials of HAART that use clinical endpoints generally have to be extremely large and follow patients for very long periods of time in order to have sufficient power to detect a difference between treatment regimens • Thus, we often consider the effect of the treatment regimen on a surrogate endpoint (eg. change in CD4, HIV RNA etc.)

Surrogate endpoints “A laboratory measurement or a physical sign used as a substitute for a clinically meaningful endpoint that measures directly how a patient feels, functions or survives.” Temple RJ. A regulatory authority’s opinion about surrogate endpoints. In: Nimmo WS, Tucker GT, eds. Clinical measurement in drug evaluation. New York, NY: John Wiley & Sons Inc. 1995.

Surrogate endpoints (cont.) In order for a laboratory marker to be a good surrogate endpoint for a clinical outcome, it has to fulfill two criteria • Surrogate must be on the causal pathway of the disease process • Entire effect of the intervention on clinical outcome should be captured by changes in the surrogate Changes in surrogate Improved clinical outcome Treatment

Surrogate endpoints (cont.) • Pre-HAART, CD4 count was established as reliable surrogate endpoint for AIDS/death • Most trials now use HIV RNA as a surrogate endpoint (eg. viral load <50 copies/ml) • BUT – not all of the effect of the treatment (eg. toxicities) may act through changes in the CD4 count or HIV RNA level • Many combinations have similar virological efficacy – other outcomes may now be more important

‘Protocol violations’ For a number of reasons, patients included and randomised in the trial may not ‘behave’ as stated in the protocol • Ineligible patients – may be recruited by mistake • Non-adherent – may forget to take some or all of their drugs, may not attend for follow-up visits, may take alternative treatments • Patient withdrawals – not able to tolerate drugs, may switch treatments QUESTION: how should these be dealt with in any analysis?

Analysis by Intention-to-treat (ITT) All patients randomised to treatment should be included in the analysis in the groups to which they were randomised

Analysis by Intention-to-treat (ITT) • Provides a measure of the real-life effect of treatment • Is the only unbiased estimate of the treatment’s effect • Most major journals require analysis by ITT • All presentations should include analysis by ITT as the primary analysis unless there is a strong justification for not doing this

On-treatment analyses Only include those patients who complete a full course of treatment to which they were randomised

On-treatment analyses • Suggested that this shows the optimal effect of treatment when taken as recommended • However, has potential to provide extremely biased estimates of treatment effect as those with the worse responses to treatment are likely to be the ones who drop-out/switch treatments • Approach will give an overly positive estimate of effect of new treatment

On-treatment analyses - example RCT with primary endpoint of virological failure at week 48. Patients are allowed to switch therapy once failure has occurred. Viral load > 50 copies/ml CHANGED TREATMENT Viral load < 50 copies/ml CHANGED TREATMENT CHANGED TREATMENT CHANGED TREATMENT

On-treatment analyses - example RCT with primary endpoint of virological failure at week 48. Patients are allowed to switch therapy once failure has occurred. Viral load > 50 copies/ml CHANGED TREATMENT Viral load < 50 copies/ml CHANGED TREATMENT Primary endpoint at week 48 = 1/1 (100%) CHANGED TREATMENT CHANGED TREATMENT

On-treatment analyses • Those remaining on randomised treatment at 48 weeks will, by definition, be those who have not experienced virological failure • Anyone with virological failure prior to week 48 will change treatment and will be excluded from the denominator • Primary event rate will always be close to 100% (depending on how quickly treatments are changed after virological failure) • FOR THIS REASON, ON-TREATMENT ANALYSES SHOULD NOT BE USED FOR THE PRIMARY ANALYSIS OF A TRIAL

Problems when analysing by ITT with surrogate endpoints • If patients are lost-to-follow-up or drop out of a trial, they are unlikely to attend for follow-up visits and blood tests • Whilst it may be possible to obtain information on clinical endpoints from other sources, information on CD4 counts or HIV RNA levels may be unavailable • Where data are missing, it is difficult to run a ITT analysis in which all patients are included in the analysis

Alternative methods of ITT analyses Where data on surrogate markers are missing, a number of alternative strategies have been proposed: • ITT Missing=Failure (ITT M=F) All missing values are treated as failures in the analysis irrespective of most recent value – ensures that all patients are included in the denominator. If anything, this gives the most pessimistic view of the new treatment.

Alternative methods of ITT analyses Where data on surrogate markers are missing, a number of alternative strategies have been proposed: • ITT last observation carried forward (LOCF) The last available measurement for each person is used in the analysis (irrespective of how long before the endpoint it was measured). This is an ITT analysis as all patients are included in the denominator but it is not favoured by regulatory bodies (eg. FDA)

Alternative methods of ITT analyses Where data on surrogate markers are missing, a number of alternative strategies have been proposed: • ITT missing=excluded All patients with missing surrogate values are excluded from the analyses – this is NOT an ITT analysis as the denominator does not include all patients recruited to the trial. Essentially this is an on-treatment analysis

Examples of different approaches Primary endpoint Viral load > 50 copies/ml Viral load < 50 copies/ml

Responder: On treatment/ ITT missing =excluded Examples of different approaches Primary endpoint 1 1 1 - - 0 - - 1 - 0 1 - - 1 - 1 - 1 0 Response rate = 8/11 = 73% Viral load > 50 copies/ml Viral load < 50 copies/ml

Responder: ITT missing =failure Examples of different approaches Primary endpoint 1 1 1 0 0 0 0 0 1 0 0 1 0 0 1 0 1 0 1 0 Response rate = 8/20 = 40% Viral load > 50 copies/ml Viral load < 50 copies/ml

Responder: ITT missing =LOCF Examples of different approaches Primary endpoint 1 1 1 0 1 0 0 1 1 1 0 1 0 0 1 0 1 0 1 0 Response rate = 11/20 = 55% Viral load > 50 copies/ml Viral load < 50 copies/ml

Examples of different approaches - summary Approach Response rate On treatment/ITT missing=excluded 73% ITT missing = failure 40% ITT missing = LOCF 55%

Subgroup analyses • It is often tempting to consider the effect of the treatment regimen in a number of subgroups of the analyses • For example, consider the effect of the regimen in the following groups: - Males/females - Low/high viral load at baseline - Low/high CD4 count at baseline - ARV-naïve/ARV-experienced at start of trial

Subgroup analyses • There are a number of dangers inherent in performing too many subgroup analyses • The increased number of tests being performed means that there are problems of multiple testing (ie. some of these comparisons are likely to be significant due to chance) • Although the study will have sufficient power to detect a difference, the subgroups will often be based on a much smaller sample size and so will not be sufficiently powered

Subgroup analyses – example 1 Although the difference between regimens A and B is similar in women as it is in men, it is not significant due to the small number of women in the study This does not provide evidence that there is no benefit of regimen B in women

Subgroup analyses – example 2 Although regimen B now looks better in females than males, a formal test of the interaction between sex and treatment group (p=0.11), suggests that these results are likely to have arisen by chance

Subgroup analyses • In any trial analysis, if subgroup analyses are thought to be important then they should be specified a priori in the protocol • The study should be sufficiently large that these subgroup analyses will be large enough to detect important differences • Evidence of a subgroup effect should never be based on a comparison of p-values in the individual subgroups, but should be based on formal tests of interaction between the factors of interest

A Beginner's Guide to Methodological and Statistical Issues in HIV Research: Session 5 - Design of RCTs for HIV Treatmen