Generative Models of M/EEG:
This presentation is the property of its rightful owner.
Sponsored Links
1 / 48

Generative Models of M/EEG: Group inversion and MEG+EEG+fMRI multimodal integration Rik Henson PowerPoint PPT Presentation


  • 90 Views
  • Uploaded on
  • Presentation posted in: General

Generative Models of M/EEG: Group inversion and MEG+EEG+fMRI multimodal integration Rik Henson (with much input from Karl Friston). A Generative Model of M/EEG Group inversion (optimising priors across subjects) Multimodal integration: 3.1 Symmetric integration (fusion) of MEG + EEG

Download Presentation

Generative Models of M/EEG: Group inversion and MEG+EEG+fMRI multimodal integration Rik Henson

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Generative models of m eeg group inversion and meg eeg fmri multimodal integration rik henson

Generative Models of M/EEG:

Group inversion and MEG+EEG+fMRI multimodal integration

Rik Henson

(with much input from Karl Friston)


Overview

  • A Generative Model of M/EEG

  • Group inversion (optimising priors across subjects)

  • Multimodal integration:

    • 3.1 Symmetric integration (fusion) of MEG + EEG

  • 3.2 Asymmetric integration of MEG + fMRI

  • 3.3 Full fusion of MEG/EEG + fMRI?

Overview


1 a peb framework for meg eeg generative model

(Linear) Forward Model for MEG/EEG (for one timepoint):

Y = Data n sensors

J = Sources p>>n sources

L = Leadfieldsn sensors x p sources

E= Error n sensors

1. A PEB Framework for MEG/EEG(Generative Model)

(Gaussian) Likelihood:

C(e)= n x n Sensor (error) covariance

Prior:

C(j)= p x p Source (prior) covariance

Posterior:

Phillips et al (2005), Neuroimage


Generative models of m eeg group inversion and meg eeg fmri multimodal integration rik henson

# sensors

# sensors

# sensors

# sensors

# sources

# sources

# sources

# sources

1. A PEB Framework for MEG/EEG(Generative Model)

Specifying (co)variance components (priors/regularisation):

C = Sensor/Source covariance

Q= Covariance components

λ= Hyper-parameters

1. Sensor components, (error):

“IID” (white noise):

Empty-room:

2. Source components, (priors/regularisation):

Multiple Sparse

Priors (MSP):

“IID” (min norm):

Friston et al (2008) Neuroimage


Generative models of m eeg group inversion and meg eeg fmri multimodal integration rik henson

1. A PEB Framework for MEG/EEG(Generative Model)

Fixed

Variable

Data

Friston et al (2008) Neuroimage


Generative models of m eeg group inversion and meg eeg fmri multimodal integration rik henson

1. A PEB Framework for MEG/EEG(Inversion)

1. Obtain Restricted Maximum Likelihood (ReML) estimates of the hyperparameters (λ) by maximising the variational “free energy” (F):

2. Obtain Maximum A Posteriori (MAP) estimates of parameters (sources, J):

cf. Tikhonov

…and an estimate of their posterior covariance (inverse precision):

(relevant to MEG+EEG integration)

3. Maximal F approximates Bayesian (log) “model evidence” for a model, m:

(relevant to MEG+fMRI integration)

Friston et al (2002) Neuroimage


Generative models of m eeg group inversion and meg eeg fmri multimodal integration rik henson

1. A PEB Framework for MEG/EEG

Summary:

  • Automatically “regularises” in principled fashion…

  • …allows for multiple constraints (priors)…

  • …to the extent that multiple (100’s) of sparse priors possible…

  • …(or multiple error components or multiple fMRI priors)…

  • …furnishes estimates of source precisions and model evidence


2 group inversion

# sensors

# sensors

# sensors

# sensors

# sources

# sources

# sources

# sources

Specifying (co)variance components (priors/regularisation):

C = Sensor/Source covariance

Q = Covariance components

λ= Hyper-parameters

2. Group Inversion

1. Sensor components, (error):

“IID” (white noise):

Empty-room:

2. Source components, (priors/regularisation):

Multiple Sparse

Priors (MSP):

“IID” (min norm):

Friston et al (2008) Neuroimage


Generative models of m eeg group inversion and meg eeg fmri multimodal integration rik henson

# sensors

# sensors

# sensors

# sensors

# sources

# sources

2. Group Inversion

Specifying (co)variance components (priors/regularisation):

C = Sensor/Source covariance

Q = Covariance components

λ= Hyper-parameters

1. Sensor components, (error):

“IID” (white noise):

Empty-room:

2. Optimise Multiple Sparse Priors by pooling across participants

Litvak & Friston (2008) Neuroimage


Generative models of m eeg group inversion and meg eeg fmri multimodal integration rik henson

2. Group Inversion (single subject)(Generative Model)

Litvak & Friston (2008) Neuroimage


Generative models of m eeg group inversion and meg eeg fmri multimodal integration rik henson

2. Group Inversion (multiple subjects)(Generative Model)

Litvak & Friston (2008) Neuroimage


Generative models of m eeg group inversion and meg eeg fmri multimodal integration rik henson

2. Group Inversion(Generative Model)

…projecting data and leadfields to a reference subject (0):

Common source-level priors:

Subject-specific sensor-level priors:

Litvak & Friston (2008) Neuroimage


Generative models of m eeg group inversion and meg eeg fmri multimodal integration rik henson

2. Group Inversion(Generative Model)

MSP

MMN

MSP (Group)

Litvak & Friston (2008) Neuroimage


3 types of multimodal integration

“Neural”

Activity

Causes (hidden):

3. Types of Multimodal Integration

(inversion)

Balloon

Model

Head

Model

Head

Model

Generative (Forward)

Models:

?

Data:

fMRI

MEG

EEG

? (future)


3 types of multimodal integration1

“Neural”

Activity

Causes (hidden):

3. Types of Multimodal Integration

Symmetric

Integration

(Fusion)

Balloon

Model

Head

Model

Head

Model

Generative (Forward)

Models:

?

Data:

fMRI

MEG

EEG

? (future)

Asymmetric

Integration

Daunizeau et al (2007), Neuroimage


Generative models of m eeg group inversion and meg eeg fmri multimodal integration rik henson

# sensors

# sensors

# sensors

# sensors

# sources

# sources

# sources

# sources

3.1 Fusion of MEG+EEG(Theory)

Specifying (co)variance components (priors/regularisation):

C = Sensor/Source covariance

Q= Covariance components

λ= Hyper-parameters

1. Sensor components, (error):

“IID” (white noise):

Empty-room:

2. Source components, (priors/regularisation):

Multiple Sparse

Priors (MSP):

“IID” (min norm):

Friston et al (2008) Neuroimage


Generative models of m eeg group inversion and meg eeg fmri multimodal integration rik henson

# sensors

# sources

# sources

# sensors

# sensors

# sensors

# sources

# sources

3.1 Fusion of MEG+EEG(Theory)

Specifying (co)variance components (priors/regularisation):

Ci(e)= Sensor error covariance for ith modality

Qij= jth component for ith modality

λij= Hyper-parameters

1. Sensor components, (error):

E.g, white noise for 2 modalities:

2. Source components, (priors/regularisation):

Multiple Sparse

Priors (MSP):

“IID” (min norm):

Henson et al (2009) Neuroimage


Generative models of m eeg group inversion and meg eeg fmri multimodal integration rik henson

3.1 Fusion of MEG+EEG(Generative Model)

Henson et al (2009) Neuroimage


Generative models of m eeg group inversion and meg eeg fmri multimodal integration rik henson

3.1 Fusion of MEG+EEG(Generative Model)

Henson et al (2009) Neuroimage


3 1 fusion of meg eeg theory

  • Stack data and leadfields for d modalities:

3.1 Fusion of MEG+EEG(Theory)

(note: common sources and source priors, but separate error components)

  • Where data / leadfields scaled to have same average / predicted variance:

mi = Number of spatial modes

(e.g, channels)

Henson et al (2009) Neuroimage


3 1 fusion of meg eeg application

ERs from 12 subjects for 3 simultaneously-acquired Neuromag sensor-types:

Magnetometers

(MEG, 102)

(Planar) Gradiometers

(MEG, 204)

Electrodes

(EEG, 70)

3.1 Fusion of MEG+EEG(Application)

fT

mV

RMS fT/m

Faces

Scrambled

ms

ms

ms

Faces - Scrambled

150-190ms

Henson et al (2009) Neuroimage


3 1 fusion of meg eeg

+19 -48 -6

+31 -51 -15

MEG mags

MEG grads

Faces

Scrambled

3.1 Fusion of MEG+EEG

Faces – Scrambled,

150-190ms

+43 -67 -11

+44 -64 -4

FUSED

EEG

IID noise for each modality; common MSP for sources

Henson et al (2009) Neuroimage

(fixed number of spatial+temporal modes)


3 1 fusion of meg eeg conclusions

  • Fusing magnetometers, gradiometers and EEG increased the conditional precision of the source estimates relative to inverting any one modality alone

  • (when equating number of spatial+temporal modes)

  • The maximal sources recovered from fusion were a plausible combination of the ventral temporal sources recovered by MEG and the lateral temporal sources recovered by EEG

  • (Simulations show the relative scaling of mags and grads agrees with empty-room data)

3.1 Fusion of MEG+EEG(Conclusions)

Henson et al (2009) Neuroimage


3 2 integration of m eeg fmri

# sensors

# sensors

# sensors

# sensors

# sources

# sources

# sources

# sources

Specifying (co)variance components (priors/regularisation):

C = Sensor/Source covariance

Q = Covariance components

λ= Hyper-parameters

3.2 Integration of M/EEG+fMRI

1. Sensor components, (error):

“IID” (white noise):

Empty-room:

2. Source components, (priors/regularisation):

Multiple Sparse

Priors (MSP):

“IID” (min norm):

Friston et al (2008) Neuroimage


Generative models of m eeg group inversion and meg eeg fmri multimodal integration rik henson

# sensors

# sensors

# sensors

# sensors

# sources

# sources

3.2 Integration of M/EEG+fMRI

Specifying (co)variance components (priors/regularisation):

C = Sensor/Source covariance

Q = Covariance components

λ= Hyper-parameters

1. Sensor components, (error):

“IID” (white noise):

Empty-room:

2. Each suprathreshold fMRI cluster becomes a separate prior

# sources

fMRI Priors:

“IID” (min norm):

# sources

Henson et al (in press) Human Brain Mapping


Generative models of m eeg group inversion and meg eeg fmri multimodal integration rik henson

3.2 Integration of M/EEG+fMRI(Generative Model)


Generative models of m eeg group inversion and meg eeg fmri multimodal integration rik henson

3.2 Integration of M/EEG+fMRI(Generative Model)


Generative models of m eeg group inversion and meg eeg fmri multimodal integration rik henson

3.2 Integration of M/EEG+fMRI (Priors)

T1-weighted MRI

{T,F,Z}-SPM

Anatomical data

Functional data

1. Thresholding and connected component labelling

Cortical surfaceextraction

Gray matter segmentation

2. Projection onto the cortical surface using the Voronoï diagram

3D geodesicVoronoï diagram

3. Prior covariance components

Henson et al (in press) Human Brain Mapping


3 2 integration of m eeg fmri application

1

2

3.2 Integration of M/EEG+fMRI (Application)

SPM{F} for faces versus scrambled faces,

15 voxels, p<.05 FWE

3

4

5

5 clusters from SPM of fMRI data from separate group of (18) subjects in MNI space

Henson et al (in press) Human Brain Mapping

Prior 4.

Prior 5.


3 2 fusion of meg fmri application

Magnetometers (MEG)

*

*

3.2 Fusion of MEG+fMRI (Application)

*

*

Gradiometers (MEG)

*

*

Negative Free Energy (a.u.)

(model evidence)

*

*

Electrodes (EEG)

*

*

*

None Global Local (Valid) Local (Invalid) Valid+Invalid

(binarised, variance priors)

Henson et al (in press) Human Brain Mapping

Prior 4.

Prior 5.


3 2 fusion of meg fmri application1

Magnetometers (MEG)

*

*

3.2 Fusion of MEG+fMRI (Application)

*

*

Gradiometers (MEG)

*

*

Negative Free Energy (a.u.)

(model evidence)

*

*

Electrodes (EEG)

*

*

*

None Global Local (Valid) Local (Invalid) Valid+Invalid

(binarised, variance priors)

Henson et al (in press) Human Brain Mapping

Prior 4.

Prior 5.


3 2 fusion of meg fmri application2

Magnetometers (MEG)

*

*

3.2 Fusion of MEG+fMRI (Application)

*

*

Gradiometers (MEG)

*

*

Negative Free Energy (a.u.)

(model evidence)

*

*

Electrodes (EEG)

*

*

*

None Global Local (Valid) Local (Invalid) Valid+Invalid

(binarised, variance priors)

Henson et al (in press) Human Brain Mapping

Prior 4.

Prior 5.


3 2 fusion of meg fmri application3

Magnetometers (MEG)

*

*

3.2 Fusion of MEG+fMRI (Application)

*

*

Gradiometers (MEG)

*

*

Negative Free Energy (a.u.)

(model evidence)

*

*

Electrodes (EEG)

*

*

*

None Global Local (Valid) Local (Invalid) Valid+Invalid

(binarised, variance priors)

Henson et al (in press) Human Brain Mapping

Prior 4.

Prior 5.


3 2 fusion of meg fmri application4

Magnetometers (MEG)

*

*

3.2 Fusion of MEG+fMRI (Application)

*

*

Gradiometers (MEG)

*

*

Negative Free Energy (a.u.)

(model evidence)

*

*

Electrodes (EEG)

*

*

*

None Global Local (Valid) Local (Invalid) Valid+Invalid

(binarised, variance priors)

Henson et al (in press) Human Brain Mapping

Prior 4.

Prior 5.


3 2 fusion of meg fmri application5

IID sources and IID noise (L2 MNM)

Magnetometers (MEG)

3.2 Fusion of MEG+fMRI (Application)

Gradiometers (MEG)

Electrodes (EEG)

None Global Local (Valid) Local (Invalid)

Henson et al (in press) Human Brain Mapping


3 2 fusion of meg fmri application6

IID sources and IID noise (L2 MNM)

Magnetometers (MEG)

3.2 Fusion of MEG+fMRI (Application)

Gradiometers (MEG)

Electrodes (EEG)

None Global Local (Valid) Local (Invalid)

Henson et al (in press) Human Brain Mapping


3 2 fusion of meg fmri application7

IID sources and IID noise (L2 MNM)

Magnetometers (MEG)

3.2 Fusion of MEG+fMRI (Application)

Gradiometers (MEG)

Electrodes (EEG)

None Global Local (Valid) Local (Invalid)

fMRI priors counteract superficial bias of L2-norm

Henson et al (in press) Human Brain Mapping


3 2 fusion of meg fmri application8

IID sources and IID noise (L2 MNM)

Magnetometers (MEG)

3.2 Fusion of MEG+fMRI (Application)

Gradiometers (MEG)

Electrodes (EEG)

None Global Local (Valid) Local (Invalid)

fMRI priors counteract superficial bias of L2-norm

Henson et al (in press) Human Brain Mapping


3 2 fusion of meg fmri application9

Right Posterior Fusiform (rPF) Right Medial Fusiform (rMF) Right Lateral Fusiform (rLF)

+41 -43 -24

+32 -45 -12

+26 -76 -11

Differential Response

(Faces vs Scrambled)

3.2 Fusion of MEG+fMRI (Application)

Left occipital pole (lOP)

R

-27 -93 0

Differential Response

(Faces vs Scrambled)

Gradiometers (MEG)

(5 Local Valid Priors)

Left Lateral Fusiform (lLF)

-43 -47 -21

L

Differential Response

(Faces vs Scrambled)

NB: Priors affect variance, not precise timecourse…

Henson et al (in press) Human Brain Mapping

Prior 4.

Prior 5.


Generative models of m eeg group inversion and meg eeg fmri multimodal integration rik henson

3.2 Fusion of MEG+fMRI (Conclusions)

  • Adding a single, global fMRI prior increases model evidence

  • Adding multiple valid priors increases model evidence further

  • Helpful if some fMRI regions produce no MEG/EEG signal (or arise from neural activity at different times)

  • Adding invalid priors rarely increases model evidence, particularly in conjunction with valid priors

  • Can counteract superficial bias of, e.g, minimum-norm

  • Affects variance but not not precise timecourse

  • (Adding fMRI priors to MSP has less effect)

Henson et al (in press) Human Brain Mapping


3 3 fusion of fmri and meg eeg

“Neural”

Activity

Causes (hidden):

3.3 Fusion of fMRI and MEG/EEG?

Fusion of fMRI + MEG/EEG?

Balloon

Model

Head

Model

Head

Model

?

Data:

fMRI

MEG

EEG

? (future)

Henson (2010) Biomag


Generative models of m eeg group inversion and meg eeg fmri multimodal integration rik henson

3.3 Fusion of fMRI and MEG/EEG?

Henson (2010) Biomag


Generative models of m eeg group inversion and meg eeg fmri multimodal integration rik henson

3.3 Fusion of fMRI and MEG/EEG?

space (s)

time (t)?

Henson (2010) Biomag


Generative models of m eeg group inversion and meg eeg fmri multimodal integration rik henson

Overall Conclusions

  • The PEB (in SPM8) framework is advantageous

  • Group optimisation of MSPs can be advantageous

  • Full fusion of MEG and EEG is advantageous

  • Using fMRI as (spatial) priors on MEG is advantageous

  • Unclear that fusion of fMRI and M/EEG is advantageous


Generative models of m eeg group inversion and meg eeg fmri multimodal integration rik henson

The End


3 fusion of meg eeg

3. Fusion of MEG+EEG

Henson et al (2009) Neuroimage


Generative models of m eeg group inversion and meg eeg fmri multimodal integration rik henson

3. Fusion of MEG+EEG

log(λx106)

log(λx106)

Hyperparameters

Participant

Participant

EEG

Grads

Mags

Grads

Mags

Henson et al (2009) Neuroimage


4 fusion of meg fmri

Magnetometers (MEG)

Gradiometers (MEG)

Electrodes (EEG)

ln(λ)+32

4. Fusion of MEG+fMRI

Participant

fMRI hyperparameters

Local

Valid

ln(λ)+32

Participant

Local

Invalid

Henson et al (in press) Human Brain Mapping

Prior 4.

Prior 5.


  • Login