Session 2 mediation and moderation of treatment effects andrew pickles
This presentation is the property of its rightful owner.
Sponsored Links
1 / 38

SESSION 2 Mediation and moderation of treatment effects Andrew Pickles PowerPoint PPT Presentation


  • 53 Views
  • Uploaded on
  • Presentation posted in: General

Methodology Research Group . Methods of explanatory analysis for psychological treatment trials workshop. SESSION 2 Mediation and moderation of treatment effects Andrew Pickles. Funded by: MRC Methodology Grant G0600555 MHRN Methodology Research Group. Moderators & Mediators.

Download Presentation

SESSION 2 Mediation and moderation of treatment effects Andrew Pickles

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Session 2 mediation and moderation of treatment effects andrew pickles

MethodologyResearch Group

Methods of explanatory analysis for psychological treatment trials workshop

SESSION 2

Mediation and moderation of treatment effectsAndrew Pickles

Funded by:

MRC Methodology GrantG0600555

MHRN Methodology Research Group


Moderators mediators

Moderators & Mediators

  • Moderator is a variable that modifies the form or strength of the relation between an independent and a dependent variable.

  • Mediator is a variable that is intermediate in the causal sequence relating an independent variable to a dependent variable.


Moderators in rcts

Moderators in RCTs

  • Moderators are baseline characteristics that influence the effect of treatment, or the effect of treatment allocation (on intermediate or final outcomes).

  • They are pre-randomisation effect-modifiers.

  • Examples: sex, age, previous history of mental illness, insight, treatment centre, therapist characteristics, genes etc.


Typical local example

Typical local example

Figure 2. SF36 scores by abuse categories at baseline and follow-up (treated patients only)

Creed et al., Psychosomatic Medicine 67:490–499 (2005)


Testing for moderation

Testing for Moderation

  • A moderator variable is typically a baseline variable (e.g. not-abused, abused)

  • Makes treatment effect greater in one group than another (moderator may or may not have an additional direct effect on outcome). It is a source of treatment effect heterogeneity

  • A classic error is to claim moderation when treatment effect is significant effect in one group and not significant in another. Is simply a recipe for increasing Type I (false positive) error rate


Interaction synergy

Interaction & Synergy

  • Need to show significant interaction with treatment on outcome

  • But on what scale?

    • Can find that interaction significant on one scale but is not significant if outcome variable is transformed. Choice of scale requires both statistical and clinical considerations.

    • If outcome binary then usual test is for interaction on the log-odds scale

      • Some argue that main effects on log-odds scale already suggests synergy

        • e.g. if the base outcome rate is low and the treatment and moderator each increase outcome by 100% then the two together increase the outcome rate not by 200% but by 300% even without an interaction


The socrates trial

The SoCRATES Trial

  • SoCRATES was a multi-centre RCT designed to evaluate the effects of cognitive behaviour therapy (CBT) and supportive counselling(SC) on the outcomes of an early episode of schizophrenia.

  • Participants were allocated to one of three conditions:

    Analysed as two conditions

    Control condition: Treatment as Usual (TAU),

    Treatment condition: TAU plus psychological, either CBT + TAU or SC + TAU.


Socrates contd

SoCRATES(contd.)

  • 3 treatment centres: Liverpool, Manchester and Nottinghamshire. Other baseline covariates include logarithm of untreated psychosis and years of education.

  • Outcome (a psychotic symptoms score) was obtained using the Positive and Negative Syndromes Schedule (PANSS).

  • From an ITT analyses of 18 month follow-up data, both psychological treatment groups had a superior outcome in terms of symptoms (as measured using the PANSS) compared to the control group.


Socrates contd1

SoCRATES (contd.)

  • Post-randomization variables that have a potential explanatory role in exploring the therapeutic effects include the total number of sessions of therapy actually attended and the quality or strength of the therapeutic alliance.

  • Therapeutic alliance was measured at the 4th session of therapy, early in the time-course of the intervention, but not too early to assess the development of the relationship between therapist and patient. We use a patient rating of alliance based on the CALPAS (California Therapeutic Alliance Scale).

  • Total CALPAS scores (ranging from 0, indicating low alliance, to 7, indicating high alliance) were used in some of the analyses reported below, but we also use a binary alliance variable (1 if CALPAS score ≥5, otherwise 0).

.


Socrates summary statistics

SoCRATES - Summary Statistics

Lewis et al, BJP (2002); Tarrier et al, BJP (2004); Dunn & Bentall, Stats in Medicine (2007).


Socrates positive symptoms basic analysis

Socrates positive symptoms:basic analysis

xi: regress enpstot psubtota rgrp

Source | SS df MS Number of obs = 225

-------------+------------------------------ F( 2, 222) = 14.80

Model | 792.779676 2 396.389838 Prob > F = 0.0000

Residual | 5945.22032 222 26.7802717 R-squared = 0.1177

-------------+------------------------------ Adj R-squared = 0.1097

Total | 6738 224 30.0803571 Root MSE = 5.175

------------------------------------------------------------------------------

enpstot | Coef. Std. Err. t P>|t| [95% Conf. Interval]

-------------+----------------------------------------------------------------

psubtota | .34999 .0785569 4.46 0.000 .1951774 .5048026

rgrp | -2.240193 .7425587 -3.02 0.003 -3.703559 -.7768275

_cons | 6.986856 1.954491 3.57 0.000 3.135127 10.83859

------------------------------------------------------------------------------


Socrates positive symptoms including main effects of centre

Socrates positive symptoms:including main effects of centre

xi: regress enpstot psubtota i.centre rgrp

------------------------------------------------------------------------------

enpstot | Coef. Std. Err. t P>|t| [95% Conf. Interval]

-------------+----------------------------------------------------------------

psubtota | .1710413 .0847491 2.02 0.045 .0040172 .3380653

_Icentre_2 | 1.679312 .8588526 1.96 0.052 -.0133193 3.371944

_Icentre_3 | -2.857869 .823287 -3.47 0.001 -4.480408 -1.235331

rgrp | -2.158757 .7039854 -3.07 0.002 -3.546176 -.7713389

_cons | 11.42025 2.038804 5.60 0.000 7.402161 15.43833

------------------------------------------------------------------------------

testparm _Icen*

( 1) _Icentre_2 = 0

( 2) _Icentre_3 = 0

F( 2, 220) = 13.56

Prob > F = 0.0000


Socrates positive symptoms treatment effect by centre

Socrates positive symptoms:treatment effect by centre

xi:xi: bysort centre : regress enpstot psubtota rgrp

-> centre = 1

------------------------------------------------------------------------------

enpstot | Coef. Std. Err. t P>|t| [95% Conf. Interval]

-------------+----------------------------------------------------------------

psubtota | .1686252 .1880974 0.90 0.373 -.2066189 .5438693

rgrp | -3.439661 1.348812 -2.55 0.013 -6.130467 -.7488547

_cons | 12.34583 4.501713 2.74 0.008 3.365161 21.3265

------------------------------------------------------------------------------

-> centre = 2

------------------------------------------------------------------------------

enpstot | Coef. Std. Err. t P>|t| [95% Conf. Interval]

-------------+----------------------------------------------------------------

psubtota | .0768268 .1548842 0.50 0.621 -.2314624 .385116

rgrp | -1.785964 1.448293 -1.23 0.221 -4.668719 1.096791

_cons | 15.31862 4.007697 3.82 0.000 7.341504 23.29575

------------------------------------------------------------------------------

-> centre = 3

------------------------------------------------------------------------------

enpstot | Coef. Std. Err. t P>|t| [95% Conf. Interval]

-------------+----------------------------------------------------------------

psubtota | .3170135 .086804 3.65 0.001 .1437987 .4902283

rgrp | -.7935641 .7000109 -1.13 0.261 -2.190414 .6032859

_cons | 4.4636 2.042415 2.19 0.032 .3880247 8.539176

------------------------------------------------------------------------------


Socrates positive symptoms moderation heterogeneity

Socrates positive symptoms:moderation/heterogeneity?

xi: regress enpstot psubtota i.centre*rgrp

------------------------------------------------------------------------------

enpstot | Coef. Std. Err. t P>|t| [95% Conf. Interval]

-------------+----------------------------------------------------------------

psubtota | .1685492 .0866722 1.94 0.053 -.0022736 .339372

_Icentre_2 | .6983508 1.398679 0.50 0.618 -2.058313 3.455015

_Icentre_3 | -4.532945 1.481842 -3.06 0.002 -7.453517 -1.612374

rgrp | -3.439764 1.245653 -2.76 0.006 -5.894829 -.9846987

_IcenXrgrp_2 | 1.458311 1.720016 0.85 0.397 -1.931679 4.848301

_IcenXrgrp_3 | 2.418837 1.779205 1.36 0.175 -1.087808 5.925483

_cons | 12.3476 2.257408 5.47 0.000 7.898459 16.79674

------------------------------------------------------------------------------

testparm _IcenX*

( 1) _IcenXrgrp_2 = 0

( 2) _IcenXrgrp_3 = 0

F( 2, 218) = 0.94

Prob > F = 0.3912


Mediators in randomised clinical trials rcts

Mediators in Randomised Clinical Trials (RCTs)

  • Mediators are intermediate outcomes on the causal pathway between allocation to or receipt of treatment and final outcome.

  • By definition, in an RCT, they are measured after randomisation.

  • Treatment effect may be fully or partially explained by a given mediator. Possible for a given mediator to serve the role of surrogate outcome.

  • Possibility of multiple mediators (multiple pathways) and interactions between mediators.


Post randomisation effect modifiers

Post-randomisation effect modifiers

  • Intermediate outcomes that influence either (a) the effects of treatment/treatment allocation on other intermediate outcomes (mediators) or (b) the effects of the other intermediate outcomes on the final outcome.

  • Candidates: amount of treatment (sessions attended), treatment fidelity, therapeuticalliance.

  • Distinction between these variables and mediators not obvious.


Examples

Examples

Compliance with allocated treatment

Does the participant turn up for any therapy?

How many sessions does she attend?

Fidelity of therapy

How close is the therapy to that described in the treatment manual? Is it a cognitive-behavioural intervention, for example, or merely emotional support?

Quality of the therapeutic relationship

What is the strength of the therapeutic alliance?


Examples cont

Examples (cont.)

What is the concomitant medication?

Does psychotherapy improve compliance with medication which, in turn, leads to better outcome? What is the direct effect of psychotherapy?

What is the concomitant substance abuse?

Does psychotherapy reduce cannabis use, which in turn leads to improvements in psychotic symptoms?

What are the participant’s beliefs?

Does psychotherapy change attributions (beliefs), which, in turn, lead to better outcome? How much of the treatment effect is explained by changes in attributions?


The mediation industry

The Mediation Industry

Baron RM & Kenny DA (1986). The moderator-mediator variable distinction in social psychological research: conceptual, strategic, and statistical considerations. Journal of Personality and Social Psychology51, 1173-1182.

As of 16th September 2009: 12,292 citations!

Assumptions are very rarely stated, let alone their validity discussed.

One suspects that the majority of investigators are oblivious of the implications.


Session 2 mediation and moderation of treatment effects andrew pickles

A Naïve Look at mediation: the B&K framework

Randomised to

Psych treatment

Independent X

c’

a

Psychotic

Symptoms

Dependent

Y

Number of

sessions

Mediator M

e3

e2

b

Regression eqns used to assess mediation

Y=d1+cX+e1

Y=d2+c’X+bM+e2

M=d3+aX+e3

total effect=c

mediated effect= ab or (c-c’) (in simple linear models these should be equal if

estimated on same sample)


Testing for mediation

Testing for Mediation

Estimate of mediated effect =

Confidence interval +/- 1.96*seab

Estimate of seab = sqrt( seb2 + sea2)

Bootstrap resampling better (allows for asymmetry)

Test of mediation (1) if 0 within CI

(2) z-test for /seab


Baron kenny steps na ve mediation

Baron & Kenny Steps: naïve mediation

  • Effect of X on Y (c) must be significant

  • Effect of X on M (a) must be significant

  • Effect of M on X (b) must be significant

  • When controlling for M, the direct effect of X on Y (c’) must be non-significant


Na ve mediation

naïve mediation

xi:regress nosess rgrp psubtota i.centre

------------------------------------------------------------------------------

nosess | Coef. Std. Err. t P>|t| [95% Conf. Interval]

-------------+----------------------------------------------------------------

rgrp | 13.82383 .5893788 23.45 0.000 12.66366 14.98401

psubtota | .1047549 .0649339 1.61 0.108 -.0230656 .2325754

_Icentre_2 | -1.387014 .7189374 -1.93 0.055 -2.802223 .0281941

_Icentre_3 | -2.87773 .7188629 -4.00 0.000 -4.292792 -1.462668

_cons | -1.210907 1.551379 -0.78 0.436 -4.264754 1.84294

------------------------------------------------------------------------------


Na ve mediation1

naïve mediation

xi:regress enpstot nosess rgrp psubtota i.centre

------------------------------------------------------------------------------

enpstot | Coef. Std. Err. t P>|t| [95% Conf. Interval]

-------------+----------------------------------------------------------------

nosess | .0417879 .0795974 0.52 0.600 -.1151377 .1987135

rgrp | -2.81782 1.345742 -2.09 0.037 -5.470936 -.1647028

psubtota | .1606631 .0871823 1.84 0.067 -.011216 .3325421

_Icentre_2 | 1.926335 .9083031 2.12 0.035 .1356243 3.717046

_Icentre_3 | -2.54384 .9285473 -2.74 0.007 -4.374462 -.7132184

_cons | 11.47856 2.103109 5.46 0.000 7.332299 15.62482

------------------------------------------------------------------------------

A=13.80 (0.59) , B=0.042 (0.08): A times B =0.58 (1.10)

Sobel estimate of standard error sqrt(13.82*0.082+0.0422*0.592)=1.10


Stata code for na ve mediation bootstrap 1

Stata code for naïve mediation: bootstrap 1

global model1 “nosess rgrp psubtota i.centre"global model2 “enpstota nosess rgrp psubtota i.centre"program mediate, rclassversion 8xi:regress $model1matrix a=e(b)xi:regress $model2matrix b=e(b)return scalar mediate=a[1,1]*b[1,1]endbootstrap mediate product=r(mediate), reps(100) dots


Stata code for na ve mediation bootstrap 2

Stata code for naïve mediation: bootstrap 2

bootstrap mediate product=r(mediate), reps(100) dots

command: mediate

statistic: product = r(mediate)

....................................................................................................

Bootstrap statistics Number of obs = 213

Replications = 100

------------------------------------------------------------------------------

Variable | Reps Observed Bias Std. Err. [95% Conf. Interval]

-------------+----------------------------------------------------------------

product | 100 .5776688 -.1432935 1.057901 -1.521436 2.676773 (N)

| -1.682636 2.333766 (P)

| -1.682636 2.333766 (BC)

------------------------------------------------------------------------------

Note: N = normal

P = percentile

BC = bias-corrected


Mediation and measurement error mediation or direct and indirect effects in sem mplus

Mediation and measurement error: mediation or direct and indirect effects in SEM (Mplus)

  • Testing for and estimating mediation can be susceptible to measurement error bias


Direct and indirect longitudinal and sleeper effects

a

b

c

y1

y2

y3

y4

Direct and Indirect: Longitudinal and sleeper effects

  • y1 directly influences y2 through path a

  • y1 only indirectly influences y3 through y2 on paths a and b

  • In a longitudinal study if y1 influences y3 directly (i.e. not through y2) this is a ‘sleeper effect’

  • This structure of restricting effects to those from the previous occasion is known as first order autorgression (AR1)


Longitudinal ability data correct at ages 6 7 9 and 11

Longitudinal Ability Data% correct at ages 6,7, 9 and 11

STANDARD DEVIATIONS

6.374 7.319 7.796 10.386

CORRELATION MATRIX

1

0.809 1

0.806 0.850 1

0.765 0.831 0.867 1


Ar1 model ability1 inp

AR1 Model: ability1.inp

TITLE: Ability autoregressive model

DATA: FILE IS D:\courses\mplus\ability.dat;

TYPE IS CORRELATION STDEVIATIONS;

NOBSERVATIONS=204;

VARIABLE: NAMES ARE y1-y4;

USEVARIABLES ARE y1-y4;

MODEL: y2 on y1;

y3 on y2;

y4 on y3;

OUTPUT: SAMPSTAT STANDARDIZED RESIDUAL;


Indirect effects ability1b inp

Indirect effects: Ability1b.inp

TITLE: Ability latent autoregressive model

DATA: FILE IS D:\courses\mplus\ability.dat;

TYPE IS CORRELATION STDEVIATIONS;

NOBSERVATIONS=204;

VARIABLE: NAMES ARE y1-y4;

USEVARIABLES ARE y1-y4;

MODEL: y2 on y1;

y3 on y2;

y4 on y3;

MODEL INDIRECT:

y4 IND y1;

y3 IND y1;

OUTPUT:STANDARDIZED;

CINTERVAL;


Ar1 model output 1

AR1 Model Output-1

Effects from Y1 to Y4

Total 0.971 0.076 12.814 0.971 0.596

Total indirect 0.971 0.076 12.814 0.971 0.596

Specific indirect

Y4

Y3

Y2

Y1 0.971 0.076 12.814 0.971 0.596

Effects from Y1 to Y3

Total 0.841 0.056 14.956 0.841 0.688

Total indirect 0.841 0.056 14.956 0.841 0.688

Specific indirect

Y3

Y2

Y1 0.841 0.056 14.956 0.841 0.688


Autoregressive output 1

Autoregressive Output-1

Chi-Square Test of Model Fit

Value 62.124! This fits

Degrees of Freedom 3 ! Very badly

P-Value 0.0000

ESTIMATED MODEL AND RESIDUALS (OBSERVED - ESTIMATED) Model Estimated Covariances/Correlations/Residual Correlations Y2 Y3 Y4 Y1 ________ ________ ________ ________ Y2 53.305 Y3 48.263 60.481 Y4 55.745 69.857 107.341 Y1 37.556 34.003 39.275 40.429 Residuals for Covariances/Correlations/Residual Correlations Y2 Y3 Y4 Y1 ________ ________ ________ ________ Y2 0.000 Y3 -0.001 -0.001 Y4 7.114 -0.001 -0.001 Y1 0.000 5.852 11.120 0.000


Autoregressive output 2

Autoregressive Output-2

TOTAL, TOTAL INDIRECT, SPECIFIC INDIRECT, AND DIRECT EFFECTS

Estimates S.E. Est./S.E. Std StdYX

Effects from Y1 to Y4

Total 0.971 0.076 12.814 0.971 0.596

Total indirect 0.971 0.076 12.814 0.971 0.596

Specific indirect

Y4

Y3

Y2

Y1 0.971 0.076 12.814 0.971 0.596

Effects from Y1 to Y3

Total 0.841 0.056 14.956 0.841 0.688

Total indirect 0.841 0.056 14.956 0.841 0.688

Specific indirect

Y3

Y2

Y1 0.841 0.056 14.956 0.841 0.688


Simplex model

Simplex Model

V’s measured with error

Autoregressive F’s

Age 7

Age 6

Age 11

Age 9

y1

y2

y3

y4

f1

f2

f3

f4

Curiously, middle part of model is identified without restrictions,

but the whole model is not identified without some restrictive

assumptions e.g. measurement error and reliability constant with age


Simplex model ability2 inp

Simplex Model: ability2.inp

TITLE: Ability latent autoregressive model

DATA: FILE IS D:\courses\mplus\ability.dat;

TYPE IS STDEVIATIONS CORRELATION;

NOBSERVATIONS=204;

VARIABLE: NAMES ARE y1-y4;

USEVARIABLES ARE y1-y4;

MODEL: f1 by y1 (1);

f2 by y2 (1);

f3 by y3 (1);

f4 by y4 (1);

y1 y2 y3 y4 (2);

f2 on f1;

f3 on f2;

f4 on f3;

MODEL INDIRECT: f3 IND f1;

f4 IND f1;

OUTPUT: STANDARDIZED;


Simplex model ability2 out

Simplex Model: ability2.out

TESTS OF MODEL FIT

Chi-Square Test of Model Fit

Value 1.440

Degrees of Freedom 2

P-Value 0.4835

TOTAL, TOTAL INDIRECT, SPECIFIC INDIRECT, AND DIRECT EFFECTS

Estimates S.E. Est./S.E. Std StdYX

Effects from F1 to F3

Total 1.170 0.074 15.901 0.925 0.925

Total indirect 1.170 0.074 15.901 0.925 0.925

Specific indirect

F3

F2

F1 1.170 0.074 15.901 0.925 0.925

Effects from F1 to F4

Total 1.516 0.105 14.396 0.877 0.877

Total indirect 1.516 0.105 14.396 0.877 0.877


Simplex model conclusion

Simplex Model: conclusion

  • Conclusion.

    In the presence of measurement error in the mediator the mediated effect is underestimated (attenuated) and the residual “direct” effect over-estimated.

    With multiple predictors (mediators) measurement error can result in decreased, increased and quite spurious effects being estimated.

  • But still ignores possible confounding – to be addressed this afternoon


  • Login