The University of Manchester 17th September 2009 Presented by:

1. MethodologyResearch Group Methods of explanatory analysis for psychological treatment trials workshop The University of Manchester 17th September 2009 Presented by: Graham Dunn, Richard Emsley, Andrew Pickles & Ian White Funded by: MRC Methodology GrantG0600555 MHRN Methodology Research Group

2. Funding • MRC Methodology Research Programme • Design and methods of explanatory (causal) analysis for randomised trials of complex interventions in mental health (G0600555 ) • G Dunn (PI), R Emsley, L Davies, J Green, A Pickles, C Roberts, I White & F Windmeijer (with collaborators C Barrowclough, R Bentall, S Birch & P Garety).

3. Explanatory versus Pragmatic questions Pragmatic Effectiveness – what is the effect of offering treatment? Analyse as randomised: Intention-to-treat (ITT) Explanatory Efficacy – what is the effect of receiving treatment? Mechanisms – how do the treatments work? What are the potential mediators? How might process variables induce treatment- effect heterogeneity? We are dealing with an observational study embedded within a randomised trial.

4. The central problem Treatment receipt, mediators and other putative process variables are not under the direct control of the investigator. They are not randomised. The effects of these intermediate variables are likely to be subject to confounding (i.e. there are other variables – frequently not measured – that influence both the intermediate variable and the final outcome). If we cannot measure all of the confounders then we have hidden confounding or hidden selection effects (selection on unobservables).

5. Distinguishing treatment-free prognosis from treatment effects A typical example from the literature: An RCT demonstrates that CBT is effective. Investigators have measured the strength of alliance in the CBT arm. Outcome and alliance are correlated in the CBT arm. So what? This tells us nothing about treatment effects. Those capable of forming a strong alliance might have had the best outcomes in the absence of therapy. The correct question is “What is the moderating effect of the alliance on the effect of treatment?” Not a question that is easy to answer.

6. Missing data and other complications Missing data closely associated with non-compliance and other process variables. Participants who do not comply with their treatment allocation are less likely to provide outcome data. Participants with a poor working alliance with their therapist are less likely to attend the full course of therapy and may also be less likely to provide outcome data. Intermediate variables will be measured with error.

7. Potential mediators What are the participant’s beliefs? Does psychotherapy change attributions (beliefs), which, in turn, lead to better outcome? How much of the treatment effect is explained by changes in attributions? What is the concomitant medication? Does psychotherapy improve compliance with medication which, in turn, leads to better outcome? What is the direct effect of psychotherapy? What is the concomitant substance abuse? Does psychotherapy reduce cannabis use, which in turn leads to improvements in psychotic symptoms?

8. Process measures:Sources of treatment-effect heterogeneity Compliance with allocated treatment Does the participant turn up for any therapy? How many sessions does she attend? Fidelity of therapy How close is the therapy to that described in the treatment manual? Is it a cognitive-behavioural intervention, for example, or merely emotional support? Quality of the therapeutic relationship What is the strength of the therapeutic or working alliance?

9. The plan for the day A mixture of lectures and general discussion Ian will start by introducing causal inference and looking at the treatment effects in those who comply with their treatment allocation (Complier Average Causal Effects). Andrew will introduce concepts of mediation and moderation. Richard will describe the use of instrumental variable methods. Graham will describe the application of principal stratification.

10. Level of difficulty We realise that most of our audience are not statisticians. Our aim is to introduce key concepts and to describe the logic and philosophy behind the modern approaches to causal inference. There will some technical details that not everyone will easily follow, but we will try to keep mathematics to an absolute minimum. We will provide software scripts (or details of where to find them) for those who are interested in having a go themselves, but we will not require any prior knowledge of any particular software packages.