Data Analysis for fMRI

1. Data Analysis for fMRI Computational Analyses of Brain Imaging CALD 10-731 and Psychology 85-735 Tom M. Mitchell and Marcel Just January 15, 2003

2. Ten Minutes of Activity for One Voxel Indicates experimental condition

4. …

5. fMRI Data Visualization[from W. Schneider] Slice View 3 D view Time Series Rendered View Inflated View

6. Many Types of Analysis • Transformation from fourier space into spatial images, adjusting for head motion, noise, drift,... (FIASCO, SVM) • Warping individual brains to canonical structure (Talairach, AIR, SPM) • Identifying voxels activated during task (t-test, F-test,…) • Finding temporally correlated voxels (clustering) • Factoring signal into few components (PCA, ICA) • Modeling temporal evolution of activity (diffeqs, HMMs) • Learning classifiers to detect cognitive states (Bayes, SVM) • Modeling higher cognitive processes (4CAPS, ACT-R) • Combining fMRI with ERP, behavioral data, …

7. Identifying Voxels Activated During Task For each voxel, vi, calculate t statistic comparing activity of vi during task versus rest condition. Retain voxels with t-statistic above some threshold

8. Mental Rotation of Imagined Objects Clock rotation Shephard-Metz rotation both [Just, et al., 2001]

9. Study of Men and Women Listening “Men listen with only one side of their brains, while women use both”  Men listening Women listening (IU School of Medicine Department of Radiology)

10. Identifying Voxels with Similar Time Courses(functional connectivity)

11. Increase in functional connectivity between parietal and inferior temporal areas with workload (from Diwadkar, Carpenter, & Just, 2001) Easier Harder The activation in two cortical areas (parietal/dorsaland inferior temporal/ventral) becomes more synchronized as the object recognition task becomes more difficult. Figure 7

12. Factoring fMRI Signals into Fewer Components PCA, ICA, SVD, Hidden Units

13. Independent Component Analysis of fMRI time-series • ICA discovers statistically independent components that combine to form the observed fMRI signal • ICA is a data-driven approach, complementary to hypothesis-driven methods (e.g. GLM) for analyzing fMRI data • Finds reduced dimensionality descriptions of poorly understood, high dimensional spaces • Requires no a-priori knowledge about hemodynamics, noise models, time-courses of subject stimuli,…

14. Independent Component Analysis of fMRI time-series: data-model (McKeown et al., 1998)

15. Independent Component Analysis of fMRI Time-series [from W. Schneider] IC #1 ICA algorithm IC #2 . . . . . . . . . . fMRI time-series IC #T

16. Independent Component Analysis of fMRI time-series ICA Solution GLM Solution IC1 IC2 Images Elia Formisano & Rainer Goebel 2001

17. Advantages of ICA • Interpretation of non-explicit condition manipulation • Not just AB type designs • Applications driving, reading, problem solving • Identify dimensions of poorly understood spaces • Reduce high dimension data to few components • Applications: structure of semantic memory, processes underlying visual scene analysis in visual cortex

18. Learning Classifiers to Decode Cognitive States from fMRI Bayes classifiers, SVM’s, kNN, …

19. Family members Occupations Tools Kitchen items Dwellings Building parts Study 1: Word Categories [Francisco Pereira et al.] • 4 legged animals • Fish • Trees • Flowers • Fruits • Vegetables

20. Training Classifier for Word Categories Learn fMRI(t)  word-category(t) • fMRI(t) = 8470 to 11,136 voxels, depending on subject Feature selection: Select n voxels • Best single-voxel classifiers • Strongest contrast between fixation and some word category • Strongest contrast, spread equally over ROI’s • Randomly Training method: • train ten single-subect classifiers • Gaussian Naïve Bayes  P(fMRI(t) | word-category)

21. Results Classifier outputs ranked list of classes Evaluate by the fraction of classes ranked ahead of true class 0=perfect, 0.5=random, 1.0 unbelievably poor

22. Impact of Feature Selection

24. Summary • Able to classify instantaneous cognitive state • in contrast to describing average activity over time • Significance • Virtual sensors for mental states • Step toward modeling sequential cognitive processes? • Potential clinical applications: diagnosis = classification

25. Modeling temporal evolution of activity HMMs, Diffeqs, …

26. start recall correct answer recall error Challenge: learn process model -- HMM’s? a=6,… 3x+a=2 transform correct read problem transform error … time 

27. y y y y y DCM [Friston 2002] Aim: Functional integration and the modulation of specific pathways Contextual inputs Stimulus-free - u2(t) {e.g. cognitive set/time} BA39 Perturbing inputs Stimuli-bound u1(t) {e.g. visual words} STG V4 V1 BA37

28. y y y The DCM and its bilinear approximation [Friston 02] neuronal changes intrinsic connectivity induced connectivity induced response Input u(t) The bilinear model activity x2(t) activity x3(t) activity x1(t) Hemodynamic model

29. Overview [Friston, 2002] • Models of • Hemodynamics in a single region • Neuronal interactions • Constraints on • Connections • Hemodynamic parameters Bayesian estimation • Applications • Simulations • Plasticity in single word processing • Attentional modulation of coupling

30. Cognitive Models Grounded in fMRI Data

31. 4CAPS Model of Language Processing [Just, et al., 2002]

32. The player was followed by the parent. [Just, et al., 2002]

33. 4CAPS Prediction of fMRI Activity CU in 4CAPS comprehension model components Model prediction Model CU transform fMRI data Figure 10 4

34. [Anderson, Qin, & Sohn, 2002]

35. Answer question Recognize word See word What We’d Like Understand question Cognitive model: Understand statement Hypothesized intermediate states, representations, processes: Observed image sequence: time 

36. Machine Learning Problems • Learn f: image(t)  cognitiveState(t) • Discover useful intermediate abstractions • Learn process models