Experimental design and efficiency

1. Experimental design and efficiency Edoardo Zamuner Nicholas Wright Methods for Dummies 17th December 2008

2. Part I: Experimental design

3. Asking the Right Question

4. The Paradigm Design When you design an experiment ask yourself: What do you hope to find about the brain? What would the finding tell you about the cognitive/perceptual process/es involved? Has the experiment already been done? Would the experiment add anything to what is already known from other techniques? Could the same question be asked more easily and cheaply with other techniques? What are the possible confounds? Can you control for those confounds?

5. What is Experimental Design? The basic principle of experimental design in fMRI is to manipulate the subject’s experience and behaviour in some way that is likely to produce a functionally specific neurovascular response. What can we manipulate? Stimulus kind and properties; Stimulus timing; Subject instructions. What are the goals of experimental design? To test specific hypotheses

6. Goals of the Experimental Design What is active? How does its activity change over time?

7. Taxonomy of Experimental Designs Categorical Subtraction - Additive factors and pure insertion Conjunction - Testing multiple hypotheses Factorial Categorical - Interactions and pure insertion - Adaptation, modulation and dual-task inference Parametric - Linear and nonlinear interactions - Psychophysiological Interactions Parametric Linear - Cognitive components and dimensions Nonlinear - Polynomial expansions

8. Categorical Design goat bucket Categorical design:it compares the brain activation in one task to that in another task. Example: Stimulus: 30s epoch; visual presentation of 12 common nouns. Tasks: decide for each noun whether it refers to an animate or inanimate object.

9. Factorial Design goat bucket GOAT bucket Factorial design: it combines two or more factors within a task and looks at the effect of one factor upon the other/s. Example: Tasks: Condition A: decide for each noun whether it refers to an animate or inanimate object. Condition B: decide whether the nouns are written in CAPITAL or lower case letters.

10. Parametric Design Parametric design: the same task is presented throughout the experiment but some continuously variable parameter of the task is manipulated. Example: Semantic task repeated for 5 minutes. The interval between consecutive stimuli (words) vary from 10s at the start to 1s at the end.  Assumption: as the task becomes more difficult blood flow to the regions specialised for semantic analysis will increase.

11. Stimulus Presentation Strategies

12. fMRI Design Types • Blocked Designs • Event-Related Designs • Periodic Single Trial • Jittered Single Trial • Staggered Single Trial • Mixed Designs • Combination blocked/event-related • Variable stimulus probability

13. Blocked Design Task A Task B Task A Task B Task A Task B Task A Task B Task A REST Task B REST Task A REST Task B REST • It involves presenting two conditions – an activation (A) condition and a baseline (B) condition. Each condition is presented for an identical epoch of time.

14. Choosing Length of Blocks • Longer blocks allow for stability of extended patterns of brain activation. • Shorter blocks allow for more transitions between tasks. • Task-related variability increases with increasing numbers of transitions

15. Pros and Cons of Blocked Design Pros: • Avoid rapid task-switching (e.g. patients); • Fast and easy to run; • Good signal to noise ratio. Cons: • Expectation; • Habituation; • Signal drift; • Poor choice of baseline may preclude meaningful conclusions; • Many tasks cannot be conducted repeatedly.

16. Event-Related Design • time It allows different trials or stimuli to be presented in arbitrary sequences.

17. Pros and Cons of Event-Related Design Pros: • Eliminate predictability of block designs (e.g. expectation); • Can look at novelty and priming; • Can look at temporal dynamics of response. Cons: • More complex design and analysis (esp. timing and baseline issues).

18. Acknowledgements Chi-Hua Chen References E.T. Bullmore & J. Suckling (2001), “Functional Magnetic Resonance Imaging”, International Review of Psychiatry 13: 24-33. P. Jezzard, P.M. Matthews, and S.M. Smith (eds), (2001), Functional MRI. An Introduction to Methods, Oxford: OUP. S.A. Huettel, A.W. Song, G. McCarthy, (2003), Magnetic Resonance Imaging, Sunderland MA: Sinauer Associate, Inc. R.S.J. Frackowiak, K.J. Friston, C.D. Frith, R.J. Dolan, C.J. Price, S. Zeki, J. Ashburner, and W. Penny, (2004), Human Brain Function, Elsevier Academic Press.

19. Load task Part II: Example factorial design in SPM MOTION NO MOTION Rees, Frith & Lavie (1997) A B C D LOW LOAD HIGH • A – Low attentional load, motion • B – Low attentional load, no motion • C – High attentional load, motion • D – High attentional load, no motion

20. MOTION NO MOTION Terminology A B C D LOW LOAD HIGH • Simple main effects • Main effects • Interaction terms

21. MOTION NO MOTION SIMPLE MAIN EFFECTS A B C D LOW LOAD HIGH • A – B: Simple main effect of motion (vs. no motion) in the context of low load • B – D: Simple main effect of low load (vs. high load) in the context of no motion • D – C: ? • Simple main effect of no motion (vs. motion) in the context of high load The inverse simple main effect of motion (vs. no motion) in the Context of high load OR

22. MOTION NO MOTION MAIN EFFECTS A B C D LOW LOAD HIGH • (A + B) – (C + D): • the main effect of low load (vs. high load) irrelevant of motion • Main effect of load • (A + C) – (B + D): ? • The main effect of motion (vs. no motion) irrelevant of load •  Main effect of motion

23. MOTION NO MOTION INTERACTION TERMS A B C D LOW LOAD HIGH • (A - B) – (C - D): • the interaction effect of motion (vs. no motion) greater under low (vs. high) load • (B - A) – (D - C): ? • the interaction effect of no motion (vs. motion) greater under low (vs. high) load

24. MOTION NO MOTION Factorial design in SPM5 A B C D LOW LOAD HIGH • How do we enter these effects in SPM5? • Simple main effect of motion in the context of low load: • A vs. B or (A – B) A B C D [1 -1 0 0]

25. Factorial design in SPM5 A B C D • Main effect of low load: • (A + B) – (C + D) • Interaction term of motion greater under low load: • (A – B) – (C – D) [1 1 -1 -1] A B C D [1 -1 -1 1]

26. Factorial design in SPM5 • Interaction term of motion greater under low load: • (A – B) – (C – D) A B C D

27. Part III: Design efficiency • A key question: • How to design fMRI experiments that are sensitive to (efficient for) a specific hypothesis (i.e. a planned statistical comparison or “contrast”)?

28. Background: general advice from Rik Henson • Scan for as long as possible (incr # vol incr df). Usually for gp results stat power incr more by # subj than # vol per subj. • Try not to unnecessarily break scanning into runs. • Do not contrast trials far apart in time (low freq noise). • Randomise either the order or SOA of trials close together in time (efficiency).

29. What is efficiency? Ways to think about it • How to design fMRI experiments that are sensitive to (efficient for) a specific hypothesis (i.e. a planned statistical comparison or “contrast”) • Ways to think about efficiency: • Mathematical (statistical) • Signal processing • Correlations between regressors – not covered today, see Rik Henson’s website.

30. What is “efficiency”? A mathematical perspective • GLM: Y = X  + • T-statistic for a given contrast: T = cT  / var(cT  ) • Want to optimise experimental design • minimise the standard error of a contrast (i.e. denominator above) • If we assume that noise variance (2) is unaffected by changes in X, this is broadly equivalent to maximising the efficiency, e, of a contrast given by e (c, X) = (2cT XTX)-1c)-1 • So, • We can influence e (a priori) by the spacing and sequencing of epochs/events in our design matrix • e is specific for a given contrast!

31. Different designs • If there is only a single event type, we can parameterise any design using: • SOAmin (min time between events) • p(t) (prob of event occurring at each t • Deterministic design: p(t)=1, low efficiency • Stationary stochastic: p(t)<1 but constant over time • Dynamic stochastic: p(t) is a function of time. Extreme is boxcar.

32. Multiple event types • With randomised designs, optimal SOA for differential effect (A-B) is minimal SOA (>2 seconds, and assuming no saturation), whereas optimal SOA for main effect (A+B) is 16-20s.

33. Background: terminology • Trial: Replications of a condition. • ITI (Inter-trial Interval) • Components: A trial consists of one or more components, that may be: • “events” or “impulses” (brief bursts of neural activity) • “epochs” (periods of sustained neural activity) • SOA (Stimulus Onset Asynchrony): The time between the onsets of components. • ISI (Inter-stimulus Interval): Time between offset of one component and onset of next.

34. Background: the BOLD impulse response (IR) • Typical BOLD response to an “impulse” (brief burst) of neural activity. • Generally in efMRI not necessary to use long SOAs to allow return to baseline (however, note issues of linearity).

35. Signal processing perspective • Signal processing is the analysis, interpretation, and manipulation of signals. Processing of such signals includes: filtering, storage and reconstruction, separation of information from noise, compression and feature extraction (Wikipedia, Dec 2008). • Consider ability to detect the BOLD response to a single event-type versus baseline.

36. Maximising efficiency from the signal processing perspective • Basic idea behind maximising efficiency is to maximise the "energy" of the predicted fMRI timeseries. • This is simply the sum of squared signal values at each scan. It is also proportional to the variance of the signal. • Therefore, to be best able to detect the signal in the presence of background noise, we want to maximise the variability of that signal. A signal that varies little will be difficult to detect.

37. An inefficient design v. more efficient design • Inefficient design (SOA 4s): Overall signal is high, but variance is low. The majority of the stimulus energy will be lost after high pass filtering. • More efficient design: A “stochastic design” in which min SOA 4s but only 50% probability of event each 4s. Only half as many stimuli, but much larger variability in the signal (which we know!).

38. Boxcar - even more efficiency! • Systematic SOA variation (e.g. runs of trials & runs of “null events”). • Why is this even more efficient? Consider Fourier transforms • Regard the IR as a “filter” that will “pass” low freq but reduce high freq. • In current example the some higher-freq harmonics reduced but fundamental freq remains!

39. Design efficiency: Conclusions • Optimal design for one contrast may not be optimal for another • Blocked designs generally most efficient (with short SOAs, given optimal block length is not exceeded) • However, psychological efficiency often dictates intermixed designs, and often also sets limits on SOAs • With randomised designs, optimal SOA for differential effect (A-B) is minimal SOA (>2 seconds, and assuming no saturation), whereas optimal SOA for main effect (A+B) is 16-20s • Inclusion of null events improves efficiency for main effect at short SOAs (at cost of efficiency for differential effects) • If order constrained, intermediate SOAs (5-20s) can be optimal • If SOA constrained, pseudorandomised designs can be optimal (but may introduce context-sensitivity)

40. Useful links and thanks • http://imaging.mrc-cbu.cam.ac.uk/imaging/DesignEfficiency • Chapter 40, Human Brain Mapping, 2004 • SPM course 2008 (Christian Ruff’s slides) • Tali Sharot’s slides from MfD 2007