Incorporating higher dimensions in joint decomposition of EEG-fMRI

1. Incorporating higher dimensions in joint decomposition of EEG-fMRI Wout Swinnen, BIOMED KU Leuven

2. Introduction: EEG and fMRI EEG fMRI

3. Introduction: EEG and fMRI EEG fMRI • Problems: • Difficult to interpret • EEG bad spatial resolution, fMRI bad temporal resolution • Lots of data: • EEG: SubjectsChannelsTime (3rd order tensor) • fMRI: SubjectsXYZTime (5th order tensor) • Solution: • Combine modalities and extract joint components • In a data-driven fashion (BSS)  JointICA

4. Introduction: EEG and fMRI EEG fMRI • Problems: • Difficult to interpret • EEG bad spatial resolution, fMRI bad temporal resolution • Lots of data: • EEG: SubjectsChannelsTime (3rd order tensor) • fMRI: SubjectsXYZTime (5th order tensor) • Solution: • Combine modalities and extract joint components • In a data-driven fashion (BSS)  JointICA

5. JointICA • Assume: ERP activity (= average EEG over trials) and fMRI response are generated by same neuronal activity (Stronger ERP peaks lead to stronger fMRI response) • If correct: Allows ICA formulation with common mixing matrix for EEG and fMRI, is EEG channel readings in matrix, is fMRI readings in matrix. JointICA will decomposeproblem into components where fMRI sources show regions that participated in ERP source activity

6. JointICA • Assume: ERP activity (= average EEG over trials) and fMRI response are generated by same neuronal activity (Stronger ERP peaks lead to stronger fMRI response) • If correct: Allows ICA formulation with common mixing matrix for EEG and fMRI, is EEG channel readings in matrix, is fMRI readings in matrix. JointICA will decomposeproblem into components where fMRI sources show regions that participated in ERP source activity

7. JointICA (B. Mijovic, 2013)  Proved that assumption is correct But one EEG channel only 

8. Incorporate multiple channels in JointICA • Concatenate EEG channels in subject dimension: sJointICA Regards all channels as one common virtual channel,with higher number of subjects • Concatenate EEG channels in time dimension: tJointICA fMRI activity is linked to a pattern of ERP peaks in multiple electrodes, all generated by the same neuronal activity

9. Incorporate multiple channels in JointICA • Concatenate EEG channels in subject dimension: sJointICA Regards all channels as one common virtual channel,with higher number of subjects • Concatenate EEG channels in time dimension: tJointICA fMRI activity is linked to a pattern of ERP peaks in multiple electrodes, all generated by the same neuronal activity

10. Incorporate multiple channels in JointICA • Concatenate EEG channels in subject dimension: sJointICA Regards all channels as one common virtual channel,with higher number of subjects • Concatenate EEG channels in time dimension: tJointICA fMRI activity is linked to a pattern of ERP peaks over multiple electrodes, all generated by the same neuronal activity

11. Materials • Visual detection task(Mijovic, 2013) • Down-left visual stimulusPress of button • 18 subjects, fMRI and EEGread non-simultaneously • Preprocessing: • EEG  ERP’sAveraged and interpolated • fMRI  PSC mapsUsing SPM software and contrastingfMRI signal after stimulus vs background

12. Results: JointICA • For electrode PO8, 18 ICs extracted (ICASSO)First 5 ICs shown • Results: fMRI regions corresponding to ERP peaks coincide with sourceregions described in literature Example: IC with Late N1 ERP (d,e)(d) Activations in somatosensory and motor areas (BA 1,2,3,4,6) (e) And in visual areas (BA 19)

13. Results: JointICA • For electrode PO8, 18 ICs extracted (ICASSO)First 5 shown • Conclusions:Meaningful decomposition revealing underlyingphysiological mechanismsBut one channel only! 

14. Results: sJointICA Analysis for electrode sets:(a) [O2, PO8] (b)[Oz, O2, PO8] (c) [PO7, Oz, PO8] (c) [PO7, O1, Oz, O2, PO8] 54 components extracted, first one shown Compare to single channel jointICA • Results: The IC’s produced describe:-fMRI areas that are more hard to interpret-less stable/natural ERP phenomenon

15. Results: sJointICA Analysis for electrode sets:(a) [O2, PO8] (b)[Oz, O2, PO8] (c)[PO7, Oz, PO8] Look at other IC’s (c) [PO7, O1, Oz, O2, PO8] 54 components extracted, first one shown Compare to single channel jointICA • Results: The IC’s produced describe:-fMRI areas that are more hard to interpret-less stable/natural ERP phenomenon

16. Results: sJointICA First 18 IC’s of 54 for electrode set [PO7,Oz,PO8] Results: ERP parts of the IC’s make up smaller timeresolutionBut ERP phenomenondescribed is less natural (narrow peak)and more hard tointerpret

17. Results: tJointICA Analysis for electrode sets: (a) [Oz, PO8] (b)[PO7, Oz, PO8] (c) [PO7, O1, Oz, O2, PO8] 18 IC’s extracted, First IC shown Compare to jointICA • Results: The IC’s produced show increasingly pronounced/robust the areas thatfunction as sources for the patternof N1 activity in the different electrodes

18. Results: tJointICA Analysis for electrode sets: (a) [Oz, PO8] (b)[PO7, Oz, PO8] Look at other IC’s (c) [PO7, O1, Oz, O2, PO8] 18 IC’s extracted, First IC shown Compare to jointICA • Results: The IC’s produced show increasingly pronounced/robust the areas thatfunction as sources for the patternof N1 activity in the different electrodes

19. Results: tJointICA All 18 IC’s forelectrode set[PO7,Oz,PO8] Results: Only strongestERP characteristicsdescribed(such as N1)No good IC’sfor weaker ERPcharacteristics(such as P1,...)

20. Conclusions • JointICA: Meaningful decomposition showing underlying physiological mechanisms, but only 1 channel • sJointICA: • Results are more difficult to interpret • tJointICA: • IC’s show more pronounced and robust fMRI sources for certain patterns of ERP activity • More information on strong ERP characteristics at expense of weak ERP characteristics

21. Questions?