Robin Davidson Science and psychotherapy? London. Autumn 2010
The Journalists TaleA.A. Gill. Sun Times. May 24th 2009 There is an academic assumption that all research is a good thing and that all knowledge gleaned from research is of equal importance as every shard in a mosaic. This is of course empirical bollocks. Most research is forgotten and an awful lot is utterly pointless and has more to do with the search for funding, career building and hierarchies than it does with uncovering and thereby righting the world
The Statisticians TaleHome Office Data Analyst. 2007 : I could call up data that could prove anything I wanted to. If I want to make a case that is really good ....using proper data I can manipulate it to make it look pretty. Similarly, if I wanted to placepressure on the Home Office to support us with an initiative I could put my hands on data that would do just that. So numbers do whatever you want. For example with the ambulance service, we could put a slant on that which would suit us... Don’t all partnerships do that.?
UKDPC Definition Recovery is a process characterized by voluntarily maintained control over substance use, leading towards health and wellbeing and participating in the responsibilities and benefits of society.
THE PSYCHOTHERAPY OUTCOME DEBATE • Rosenzweig (1936) introduced the idea of psychotherapeutic equivalence • Luborsky et al (1975) the “dodo bird verdict”, ie all therapies were equivalent • Beutler (1991) “all have won and must have prizes”, ie all therapies were effective and equivalent • Chambless et al (2002) Selectively listed “empirically supported therapies (ESTs)”
Project MATCH: 3 interventions: Motivational Enhancement Therapy (4 sessions) 12 Step Facilitation (12 sessions) Cognitive Behavioural Coping Skills Therapy (12 sessions) Outpatient Arm: 952 clients in 5 centres Aftercare Arm: 774 clients in 5 centres UK Alcohol Treatment Trial (UKATT): 2 interventions: Motivational Enhancement Therapy (up to 3 sessions) Social Behaviour and Network Therapy (up to 8 sessions) 742 clients (MET - 422; SBNT 320) 7 treatment sites in 3 UK cities
PERCENTAGE DAYS ABSTINENT UKATT MATCH
DRINKS PER DRINKING DAY MATCH UKATT MATCH: % reduction 60% MATCH: 1 DRINK = 0.5oz (14.2 gms) UKATT: % reduction 28% UKATT: 1 DRINK = 8.5 gms
PERCENTAGE REDUCTION IN ALCOHOL RELATED PROBLEMS MATCH - Drinker Inventory of Consequences (DIC) UKATT - Alcohol Problems Questionnaire (APQ)
SITE x TREATMENT INTERACTION OUTCOME
The main MATCH reports did acknowledge site by treatment interactions, but these have never been detailed. If you regard MATCH as nine independent replications rather than a single trial there are sites at which there are main effects of treatment and not all in the same direction. Miller (2005) Personal Communication
MATCHTHERAPIST x INTERVENTION INTERACTION eg • Therapist’s level of education/years of experience had minimal impact on CBT and MET, but negatively related to TSF outcomes • Female clients had better outcomes with female therapists • Strength of working alliance predicts outcome HOWEVER • Therapists were nested within therapies, not randomly assigned • Correlation between Working Alliance Inventory (WAI) and motivational readiness to change
WHY DIFFERENT INTERVENTIONS PRODUCE THE SAME EFFECT • Treatments have elements in common • Treatments represent different paths to the same end • There are common processes of behaviour change that are facilitated equally well by different treatments • The treatments have no effect in producing outcomes
WHY THE UK v USA DIFFERENCE • Outcome Goal: MATCH was strictly abstinence, UKATT was control or abstinence • Health Culture: In USA general health treatment is paid for, therefore is compliance greater? • Baseline Severity: Had the UKATT sample higher alcohol dependence/problems? • Internal v External MATCH may have had stricter exclusion criteria Validity:
WHY THE UK v USA DIFFERENCE (2) • Motivation: Were there baseline motivational differences? MATCH clients tolerated 8 hours pre-treatment assessment! • Therapist Training: Did the MATCH therapists have longer/better training? • National Differences: Large USA psychotherapy outcome trials generally demonstrate greater improvement than UK trials • Assessment reactivity: MATCH clients had much more intensive research follow-up (every 3 months)
ALLITERATIVE ARGUEMENT Psychotherapy outcomes are not only about Evidence based processes but also Evidence based populations Evidence based places Evidence based practitioners
Is drug treatment research up to the challenge if it: • Focuses on technique not process • Ignores role of therapist • Ignores role of subjective experience • Ignores the therapeutic relationship • Measured out therapy in medicine-like doses • Ignores the agency
What is a fair Test, i.e. • BNT more effective than placebo control • BNT more effective than no treatment control • Clinically significant or statistically significant • What about non specific variance due to treatment infidelity, investigator allegiance, therapeutic alliance.
Are people in research trials real? Current eligibility criteria in alcohol treatment outcome research, Humphries et al (2008) % of 683 studies • Psychiatric/emotional problems 37.8% • Alcohol treatment 31.8% • Non-Compliance/low motivation 31.5% • Medical problems 31.6% • Neuro-cognitive problems 23.0% • Illicit drug use 22.7% • Social/Residential Instability 19.6%
Do eligibility criteria affect outcome? • Social-Residential Instability 15.6% • Non-Compliance/Low motivation 5.8% • Illicit drug use 7.2% • Medical Problems 10.1% • Psychiatric Problems 8.7%
Signs of the times The conservative NTA paper on Treatment Effectiveness concedes: • Therapist characteristics 10 to 50% of outcome variance • Therapeutic alliance important • Treatment accounts for only 1/3rd of improvement
Signs of the times The International Treatment Effectiveness Project (ITEP) is predicated on: “Evidence tells us that the way a drug treatment service is organized and managed can have as much if not more impact on client outcome as the interventions on offer” (2008)
Signs of the times The impact of Orford (2008) with regard to a potential root and branch reassessment of our psychotherapy outcome science
HOW TO INVENT A NEW PSYCHOTHERAPY - SIX SIMPLE STEPS! 1. Start with an established psychological process of change 2. Add functionally trivial bells and whistles 3. Demonstrate using one RCT that the “new” treatment is statistically (P< .05) better than no treatment 4. Give “new” treatment a name and acronym 5. Patent it 6. Set up accreditation process and retire to the South of France Davidson (2008) “The central goal of psychotherapy research should be to achieve an understanding of the psychological mechanisms or processes of change and not focus on brand name treatments” Davison (2003)
Key References Humphreys, K., Harris, A.S., & Weingardt, K. (2008). Subject eligibility criteria can substantially influence the results of alcohol treatment outcome research. Journal of Studies on Alcohol and Drugs, 69, 757-764 McLellan, A. R., McKay, J.R., Forman, R., Cacciola, J. & Kemp, J. (2005). Reconsidering the evaluation of addiction treatment: From retrospective follow-up to concurrent recovery monitoring. Addiction, 100, 447-458. Morgenstern, J., & McKay, J. (2007). Rethinking the paradigms that inform behavioral treatment research for substance use disorders. Addiction, 102, 1377-1389. Orford, J. (2008). Asking the right questions in the right way: The need for a shift in research on psychological treatments for addiction. Addiction, 103, 875-885. Davidson R. (2008) Accredited, brand named psychotherapies and the standard of evidence. Clinical Psychology Forum.191,p 48-52.