Yi Chen, Radoslaw M. Cichy and John-Dylan Haynes

1. Yi Chen, Radoslaw M. Cichy and John-Dylan Haynes Bernstein Center for Computational Neuroscience Berlin &Charité – UniversitätsmedizinMax-Planck-Institute for Human Cognitive and Brain Sciences, Leipzig 25/11/2011 Berlin Multi-Scale Mapping Of fMRI Information On The Cortical Surface: A Graph Wavelet Based Approach

2. Multivariate Pattern Analysis of fMRI Signal ? Pattern Recognition Haxby et al. Science, 2001

3. Spatial Range of MVPA Methods Global Local Whole brain ROI-based Searchlight technique Searchlight technique affords unbiased, spatially localizedinformation detection. • Haxby et al., Science, 2001 Haynes & Rees, Nature Rev. Neurosci, 2006Kriegeskorte et al., PNAS, 2007

4. BOLD changes The fMRI Signal in Space Information carried by the fMRI signals resides in the convolved cortical sheets. The brain has a complex structure: 3D searchlight methods do not take this structural complexity of the brain into account. 3D searchlight • Jin & Kim, Neuroimage, 2008

5. Cortical Surface-based Searchlight Searchlight on cortical surface mesh 3D searchlight neglects local geometry Surface-based searchlight respects local geometry • Chen et al., NeuroImage, 2011

6. Application: Decoding Object Category Object categories: Trumpetsvs Chairs vs Boats Chen et al., NeuroImage, 2011

7. Surface-based vs 3D Method Collateral sulcus Fusiform gyrus Surface-based method observes local structure Collateral sulcus Fusiform gyrus 3D method deteriorates spatial specificity Surface-based method localizes fMRI informationmore precisely • Chen et al., NeuroImage, 2011

8. V2 V3 V1 V2 L R V3 Multiscale Organization of Brain Function Hierarchical organization with increasing spatial scale Ocular dominanceand orientation preference columns Retinotopic maps Object selective regions Knowing the spatial scale of patterns is crucial for understanding the brain’s functional organization Yacoub et al., PNAS, 2008Wandell, Encyclopedia Neurosci., 2007

9. Fine scale detail Large scale detail Multiscale Analysis – Wavelet Transform Wavelets, or “little waves”, are families of spatially local, band-passing filters: Fine scale wavelet Fine scale information Transform Output: Large scale information Scale up Large scale wavelet Information specific to different scales can be extracted with wavelets Hackmack and Haynes, in prep.

10. Wavelets on Irregular Mesh Wavelets on regular grid Wavelets on irregular mesh Translation invariant Varies on translation On an irregularmesh, wavelet transform cannot be directly implemented

11. Another Way to Look at Discrete Fourier Transform For a signal x defined on a one-dimensional, regular and circular field, we have: Discrete Laplacian: where K is a symmetric matrix, its eigenvectors, when sorted non-decreasingly w.r.t. eigenvalues: Projecting a signal onto the space spanned by these eigenvectors is thus computationally equivalent to its Discrete Fourier Transform (DFT): Manipulating the transform coefficients and exploiting the unitary property of U, we can implement filters on the frequency domain. The filtered signal is given by: The diagonal matrix contains the Impulse Response function of the designed filter. Taubin SIGGRAPH '95

12. characterizes the geometricproperties of the graph. The eigenvectors of graph Laplacian have a quasi-frequency property: Implementing Wavelets via Graph Laplacian • Generalized graph Laplacian H: • Wavelets on irregular mesh can then be defined on the eigenspectral domain: Freq. Response Biyikoglu et al., Laplacian Eigenvectors of Graphs, 2007Hammond et al., Applied & Comp. Harmonic Analysis, 2009 Eigenspectrum

13. Fine scale detail Large scale detail Multiscale Analysis on Irregular Mesh Fine scale wavelet Fine scale information Transform Outputs Large scale information Large scale wavelet Spectral graph wavelets can be used to achieve multiscale analysis on irregular meshes

14. Anisotropic Filters on Cortical Surface Vertical Horizontal Fine scale Large scale Anisotropic filters are possible by using different geometric schemes for the graph Laplacian

15. Categories: Objects vs Scenes vs Body parts vs Faces Exemplars: Child vs Female vs Male Multiscale Analysis of Object Categories/Exemplars 2-step procedure: • BOLD estimates were sampled onto the cortical surface & transformed with spectral graph wavelets • At each scale,the outputs from the filter bankswere taken as feature vectors for classification • Cichy et al., Cerebral Cortex, 2011

16. Scale Differentiated Analysis ofExemplar and Category Encoding Categories Exemplars Fine Scale Large Scale z-score Categories are preferentially encoded in large scale andexemplars in fine scale

17. Summary Cortical surface-based method • respects natural geometry of the brain • improves spatial specificity of MVPA Multi-scale analysis on the cortical surface • can extract information from fMRI signals at different scales using spectral graph wavelets • shows that object categories and exemplars are encoded in different spatial scales in the ventral visual stream The combination of surface-based technique and multi-scale information mapping promises a better understanding of human brain function

18. John-Dylan Haynes Radoslaw M. Cichy Jakob Heinzle Acknowledgements Kerstin Hackmack Fernando Ramirez NEUROCURE

19. Appendix

20. Spectral Graph Wavelets & Fast Algorithm • For filter with compact spatial support, its impulse response function defined on eigenspectral domain needs to be continuously differentiable. • Wavelet functions are defined by a family of dilated versions of a single function (mother wavelet). • Mother wavelet needs to meet the admissibilitycondition. • Fast algorithm is possible by approximating the wavelet function on eigenvalue domain with truncated orthogonal polynomials (e.g. Chebyshev polynomial), and calculating the eigenspace projection with recursive sparse matrix vector multiplications (Sect.6, Hammond et al., 2009). • Note, however, by adopting above fast algorithm, the dilation of mother wavelet is now carried on the eigenvalue domain, rather than the eigenvalue’s rank/index domain. Hammond et al., Applied & Comp. Harmonic Analysis, 2009

21. Fine scale detail Fine scale detail Large scale detail Large scale detail Multiscale Analysis on Regular Grid Fine scale wavelet Transform Outputs Large scale wavelet