Service-related research: Therapy outcomes audit

Service-related research: Therapy outcomes audit Sarah Howley Trainee Clinical Psychologist UCL

Background (1) • Big changes in NHS – current emphasis on NICE guidance & evidence-based practice • NICE currently does not consider practice-based evidence as equivalent to RCT evidence • CBT is NICE-recommended therapy for depression and anxiety problems based on RCT evidence

Background (2) • BUT – does evidence of efficacy from RCTs prove the effectiveness of therapies in real-life clinical settings? Many say no! • AND – does absence of efficacy evidence mean therapies are not effective? • There has been very little research generally comparing CBT (“gold standard”) to other therapies

Rationale for study in this Service • No recent audit of therapy outcomes in PTS • Little explicit information about how clients are allocated to CBT or Exploratory waiting lists • Little info on comparison of therapies (PCP, Existential and Cognitive Behavioural therapies)

Aims of current study • To investigate: • Potential factors influencing allocation to different therapy modalities: demographic variables (age, gender, ethnicity), problem characteristics (type, severity, duration) • Are therapies equivalent at (a) reducing CORE-OM scores from pre- to post-therapy and (b) producing reliable and clinically significant change across therapy?

Methods • CORE-OM forms used routinely in service across all therapeutic modalities • Data taken from CORE database from 2002 to 2010 • CBT sample = 386, PCP = 15, ExT = 45, “Exploratory therapy” = 5 • Therefore: PCP, ExT + Exploratory amalgamated (“Exploratory Therapy - EPT” N = 65) and random sample taken from CBT dataset (N = 65; representative of full CBT dataset as determined by statistical analyses)

Methods • Variables: • Demographics: Age, Sex, Ethnicity • Referral source (GP, PCMHT etc) • Problem characteristics: type, severity & duration • CORE-OM mean scores at Assessment (IAS), Pre-therapy (1st treatment session) and Post-therapy (last treatment session)

Sample characteristics

Outcomes analyses • CORE-OM scores withintherapy groups (Effectiveness): • Assessment vs Pre-therapy (change on W/L) • Pre-therapy vs Post-therapy (therapy-related change) • Reliable Change Index – calculation to determine whether clients achieved reliable and clinically significant change (Jacobson and Truax, 1991)

Outcomes analyses • CORE-OM scores betweengroups (comparison of therapies) • Assessment scores – do clients vary in severity at assessment? • Pre-therapy scores (baseline) • Post-therapy scores – are therapies equivalent in terms of outcome i.e. reduction in CORE scores? • Reliable Change Index – are therapies equivalent in terms of achieving reliable and clinically significant change?

Results • Allocation to therapeutic modality: • Only difference between therapy groups was in primaryproblem type • Anxiety disorders significantly more likely to be found in CBT group as primary presenting problem • Chi Square test: χ2 = 6.65, p < 0.01

Results: Within groups • Within groups: • CBT & EPT: significant reductions in CORE domain scores from IAS to pre-therapy • Effectiveness? • CBT - sig reduction in all CORE domain scores pre-post • EPT – sig reduction in all CORE domain scores pre-post except Risk (p = .059)

Results: Between groups • Comparing CBT and EPT on CORE-OM: • IAS: No statistically significant differences between groups • Pre-therapy: No statistically significant differences between groups • HOWEVER: In CBT group Problems/Symptoms domain score was below clinical cut-off • Post-therapy: CBT group had significantly lower Problems/Symptoms and All Items scores compared to EPT

Comparing CORE-OM scores

Reliable Change • Jacobson & Truax (1991) Method used to identify whether individual clients achieve reliable and clinically significant change (improvement OR deterioration) • This indicator of clinical significance is distinct from statistical significance • There was no statistically significant difference between the groups in terms of no. of clients achieving reliable improvement

Recap of results • No differences observed in age, sex, ethnicity, referral source, problem severity or chronicity between therapy groups • Anxiety disorders more likely to be main problem in CBT group • Both therapies showed statistically significant improvement in CORE scores across domains (apart from EPT group in Risk but score below clinical cut off) • CBT group showed significantly more improvement on Problems/Symptoms domain (but were below clinical cut off at pre-therapy) • No statistically significant difference in number of clients achieving reliable clinical improvement between groups

Interpretation of results • Clients presenting with anxiety as main difficulty tend to be referred more often to CBT – Service is in line with NICE guidance on treating anxiety • Other factors in allocation to W/L – client preference, length of waiting lists, assessor’s preferred model? • No significant deterioration in Risk scores in CBT or EPT groups – effects of waiting list times on client risk? • CBT targets specific “symptoms” – therefore unsurprising that CBT clients show more reduction in this domain? (assuming result is valid – small effect size…) • Both CBT and EPT are effective in reducing CORE-OM scores from pre- to post-therapy in this Service and clients in both achieve comparable levels of reliable improvement

Limitations • Relatively small sample sizes (n = 65 and smaller for most analyses due to missing data) • Use of data from up to 10 years ago – does this reflect current service? • Amalgamation of PCP, ExT & “exploratory” – difficult to draw conclusions (but unavoidable!) • Use of non-parametric stats (increased risk of not finding significant differences i.e. type 1 error)

Recommendations • Encourage all clinicians to record CORE-OM scores at each stage of therapy – vital to practice-based evidence for therapies offered here • Carry out similar audits at regular intervals in order to establish a bedrock of practice-based evidence…

Service-related research: Therapy outcomes audit