Multimodal Brain Imaging

1. Multimodal Brain Imaging Will Penny Wellcome Trust Centre for Neuroimaging, University College, London Guillaume Flandin, CEA, Paris Nelson Trujillo-Barreto, CNC, Havana

2. Neuronal Activity Experimental Manipulation Optical Imaging MEG,EEG PET fMRI FORWARD MODELS Single/multi-unit recordings Spatial convolution via Maxwell’s equations Sensorimotor Memory Language Emotion Social cognition Temporal convolution via Hemodynamic/Balloon models

3. Neuronal Activity Experimental Manipulation MEG,EEG fMRI INVERSION Spatial deconvolution via beamformers

4. MEG/EEG: Source reconstruction Source model ‘Imaging’ ‘ECD’ Forward model Registration Inverse method Data Anatomy

5. MEG/EEG: Source reconstruction Sources MEG data Source Reconstruction ‘Equivalent Current Dipoles’ (ECD) ‘Imaging’ EEG data

6. Neuronal Activity Experimental Manipulation MEG,EEG fMRI INVERSION Spatial deconvolution via beamformers Temporal deconvolution via model fitting/inversion

7. fMRI Forward Model Seconds

8. Effect Map Parameter Estimates fMRI model fitting Design matrix fMRI time-series Motion Correction Smoothing General Linear Model Spatial Normalisation Anatomical Reference

9. Neuronal Activity Experimental Manipulation MEG,EEG fMRI INVERSION 1. Spatio-temporal deconvolution Spatial deconvolution via beamformers 2. Probabilistic treatment Temporal deconvolution via model fitting/inversion

10. Overview • Spatio-temporal deconvolution for M/EEG • Spatio-temporal deconvolution for fMRI • Towards models for multimodal imaging

11. Add temporal constraints in the form of a General Linear Model to describe the temporal evolution of the signal. Puts M/EEG analysis into same framework as PET/fMRI analysis. Work with Nelson. Described in chapter of new SPM book: Friston et al. Statistical Parametric Mapping, Elsevier, London, 2006 Spatio-temporal deconvolution for M/EEG

13. Repeat L • Update source estimates, q(j) • Update regression coefficients, q(w) • Update spatial precisions, q(a) • Update temporal precisions, q(l) • Update sensor precisions, q() KL F Until change in F is small Approximate Bayesian Inference

15. Corr(R3,R4)=0.47

18. 700ms 500ms 2456ms 600ms Fa o + + + Time Sa + o Sb + + High Symmetry Low Symmetry Low Asymmetry High Asymmetry Phase 1 Ub Henson R. et al., Cerebral Cortex, 2005

19. A1 Faces minus Scrambled Faces B8 170ms post-stimulus

20. B8 A1 Faces Scrambled Faces

21. Daubechies Cubic Splines Wavelets

22. Daubechies-4 28 Basis Functions 30 Basis Functions

23. ERP Faces ERP Scrambled

24. t = 170 ms

25. Faces – Scrambled faces: Difference of absolute values t = 170 ms

26. Temporal evolution is described by GLM in the usual way. Add spatial constraints on regression coefficients in the form of a spatial basis set eg. spatial wavelets. Automatically select the appropriate basis subset using a mixture prior which switches off irrelevant bases. Embed this in a probabilistic model. Work with Guillaume. In Neuroimage. Spatio-temporal deconvolution for fMRI

27. Spatial Model eg. Wavelets

28. Mixture prior on wavelet coefficients • Wavelet switches: d=1 if coefficient is ON. Occurs with probability p • If switch is on, draw z from the fat Gaussian.

29. Probabilistic Model for fMRI Switch priors Wavelet switches Wavelet coefficients Spatial Model General Linear Model Temporal Model fMRI data

31. Compare to (i) GMRF prior used in M/EEG and (ii) no prior

32. Inversion using wavelet priors is faster than using standard EEG priors

33. Results on face fMRI data

34. Use simultaneous EEG- fMRI to identify relationship Between EEG and BOLD (MMN and Flicker paradigms) EEG is compromised -> artifact removal Testing the `heuristic’ Start work on specifying generative models Ongoing work with Felix Blankenburg and James Kilner Towards multimodal imaging

35. fMRI results

36. fMRI results

37. MRI Gradient artefact removal from EEG We have “synchronized sEEG-fMRI” – MR clock triggers both fMRI and EEG acquisition; after each trigger we get 1 slice of fMRI and 65ms worth of EEG. Synchronisation makes removal of GA artefact easier

38. Ballistocardiogram removal Could identify QRS complex from ECG to set up a ‘BCG window’ for subsequent processing

39. Ballistocardiogram removal

40. Ballistocardiogram removal

41. Testing the heuristic • The EEG-BOLD heuristic (Kilner, Mattout, Henson & Friston) contends that • increases in average EEG frequency predict BOLD activation. g(w) = spectral density

42. Log of Bayes factor for Heuristic versus Null

43. Log of Bayes factor for Heuristic versus Alpha

44. Probabilistic model for EEG-fMRI

45. THANK-YOU FOR YOUR ATTENTION !