Spatio-Temporal Models for Mental Processes from fMRI

1. Spatio-Temporal Models for Mental Processes from fMRI Raghu Machiraju FirdausJanoos, Fellow, Harvard Medical Istavan (Pisti) Morocz, Instuctor, Harvard Medical

2. Premise Understanding the mind not only requires a comprehension of the workings of low–level neural networks but also demands a detailed map of the brain’s functional architecture and a description of the large–scale connections between populations of neurons and insights into how relations between these simpler networks give rise to higher–level thought

3. Goals • Understanding the representation of mental processes in functional neuroimaging • Distributed interactions • Space and time ! • Comparing processes across subjects • Neurophysiologic interpretability

4. Outline • fMRI Analysis • Representations • Spatio-temporal Models • Conclusion

5. What Is fMRI ? • fMRI is a non-invasive tool for studying brain activity • Spatio-temporal data (4D) • Spatial resolution – mm • Temporal resolutions – secs • Functional specialization • Classical neuroscience • Functional integration • Functional and effective

6. The fMRI Signal • The BOLD Effect • Measure of cerebral metabolism • Task related • Default-state networks • Confounds/Nuisance • Random – thermal + quantum mechanical • Structured component • Distortions, physiological, motion, reconstruction

7. The BOLD Effect Measure of “oxygenated blood” in the brain • Volume of deoxyhemoglobin • T2* weighted EPI sequences • The exact coupling between neuronal activity and the BOLD signal unknown • Linked primarily to metabolic activity at synapses • Depends on rCBF, rBVO2, rCMRO2 • The hemodynamic response function is highly variable

8. fMRI Noise • Acquisition • Reconstruction • Magnetic field • Inhomogeneities • Instability • Physiologic functions • Aliased onto signal • Head motion • Correlated with the task • Registration / Correction

9. Classical Pipeline

10. fMRI Analysis • Functional Localization • Static Activity Maps • GLM, PCA, ICA, PLS, • Functional Integration • Functional Connectivity • CCA, ICA, PCA, DBN • Effective Connectivity • SEM, DCM, DBN

11. Typical DCM

12. Benefits • fMRI provides information about the activity of large neural assemblies • Static pictures of the foci of activity and the interconnections • Mental processes arise from dynamic relationships between the neural substrates • Spatially distributed, temporally transient and occur at multiple scales of space and time. • Time resolved analysis • Ordering of information processing

13. Cascadic Recruitment

14. State-of-the-Art Janoos et al., EuroVis2009

15. Need Decoding ! • VOXEL-wise Representations Limited • Dynamic Processes • Distributed Representations Needed • Beyond functional localization • Where vs. how • Distributed activity and functional interactions • Pattern Classifiers • Atoms of Thought for Cracking Neural Code 

16. Haxby, 2001

17. Mitchell, 2008

18. Challenges • Very controlled experiments with copious training • General results have not always been positive • Applications to arbitrary settings ? • Temporal nature of mental processes • Neurophysiologic interpretability • Multi-subject analysis

19. Inspiration Lehmann, 1994

20. Preliminary Results visuo–spatial working memory

21. 2 Patients

22. Functional Networks

23. Functional Connectivity Estimation

24. Functional Distance Zt – activation patterns f - transportation

25. Cost Metric

26. Functional Distance t1 t2 t3

27. Algorithm

28. Mental Arithmetic • Involves basic manipulation of number and quantities • Magnitude based system – bilateral IPS • Verbal based system – left AG • Attentional system – ps Parietal Lobule • Other systems – SMA, primary visual cortex, liPFC, insula, etc

29. Paradigm

30. Clustering in Functional Space

31. Spatial Maps 10 10 same as 8 9 8 8 auditory cortices 7 6 , judgment 6 5 5 Frontal, parietal lobes 4 3 3 visual size estimation 2 1 1 Visual Cortex 0 0s 4s 8s +5.0 -5.0 0

32. Critique • No neurophysiologic model • Point estimates • Hemodynamic uncertainty • Temporal structure • Functional distance - an optimization problem • No metric structure • Expensive !

33. Functional Distance

34. Cost Metric

35. Cost Metric Distortion minimizing

36. Feature Space Φ Orthogonal Bases Graph Partitioning Normalized graph Laplacian of F

37. Feature Space Φ

38. Feature Selection

39. State-Space Model Janoos et al., MICCAI 2010

40. (Reduced) State-Space Model

41. Model Size Selection • Typically strike a balance between model complexity and model fit • Information theoretic or Bayesian criteria • Notion of model complexity • Cross-validation • IID Assumption

42. Maximally Predictive Criteria • Multiple spatio-temporal patterns in fMRI • Neurophysiological • task related vs. default networks • Extraneous • Breathing, pulsatile, scanner drift • Select a model that is maximally predictive with respect totask • Predictability of optimal state-sequence from stimulus, s

43. Dyscalculia Difficulty in learning arithmetic that cannot be explained by mental retardation, inappropriate schooling, or poor social environment • Core conceptual deficit dealing with numbers • Very common : 3-6% of school-age children • Heterogeneous

44. Paradigm

45. Results Self – same subject Cross – train on one subject and predict on another

46. Comparing Models Φ1 Subject 1 Φ2 Subject 2 . . . fMRI Data Φ42 Subject 42

47. MDS Plot

48. MDS Plot

49. Drawbacks • Approximations in the model • Elimination of the activity pattern layer • Spatially unvarying hemodynamics • Unsupervised approach • No explicit link to the experiment • May not necessarily learn relevant patterns

50. Semi-supervised Approach • Loose dependency between stimulus and signal • Not preclude discovery of un-modeled effects • Stabilize estimation • Generalizable to unconstrained designs • Functionally well-defined representation