650 likes | 725 Views
Spatio-Temporal Models for Mental Processes from fMRI. Raghu Machiraju Firdaus Janoos , Fellow, Harvard Medical Istavan ( Pisti ) Morocz , Instuctor , Harvard Medical. Premise.
E N D
Spatio-Temporal Models for Mental Processes from fMRI Raghu Machiraju FirdausJanoos, Fellow, Harvard Medical Istavan (Pisti) Morocz, Instuctor, Harvard Medical
Premise Understanding the mind not only requires a comprehension of the workings of low–level neural networks but also demands a detailed map of the brain’s functional architecture and a description of the large–scale connections between populations of neurons and insights into how relations between these simpler networks give rise to higher–level thought
Goals • Understanding the representation of mental processes in functional neuroimaging • Distributed interactions • Space and time ! • Comparing processes across subjects • Neurophysiologic interpretability
Outline • fMRI Analysis • Representations • Spatio-temporal Models • Conclusion
What Is fMRI ? • fMRI is a non-invasive tool for studying brain activity • Spatio-temporal data (4D) • Spatial resolution – mm • Temporal resolutions – secs • Functional specialization • Classical neuroscience • Functional integration • Functional and effective
The fMRI Signal • The BOLD Effect • Measure of cerebral metabolism • Task related • Default-state networks • Confounds/Nuisance • Random – thermal + quantum mechanical • Structured component • Distortions, physiological, motion, reconstruction
The BOLD Effect Measure of “oxygenated blood” in the brain • Volume of deoxyhemoglobin • T2* weighted EPI sequences • The exact coupling between neuronal activity and the BOLD signal unknown • Linked primarily to metabolic activity at synapses • Depends on rCBF, rBVO2, rCMRO2 • The hemodynamic response function is highly variable
fMRI Noise • Acquisition • Reconstruction • Magnetic field • Inhomogeneities • Instability • Physiologic functions • Aliased onto signal • Head motion • Correlated with the task • Registration / Correction
fMRI Analysis • Functional Localization • Static Activity Maps • GLM, PCA, ICA, PLS, • Functional Integration • Functional Connectivity • CCA, ICA, PCA, DBN • Effective Connectivity • SEM, DCM, DBN
Benefits • fMRI provides information about the activity of large neural assemblies • Static pictures of the foci of activity and the interconnections • Mental processes arise from dynamic relationships between the neural substrates • Spatially distributed, temporally transient and occur at multiple scales of space and time. • Time resolved analysis • Ordering of information processing
State-of-the-Art Janoos et al., EuroVis2009
Need Decoding ! • VOXEL-wise Representations Limited • Dynamic Processes • Distributed Representations Needed • Beyond functional localization • Where vs. how • Distributed activity and functional interactions • Pattern Classifiers • Atoms of Thought for Cracking Neural Code
Challenges • Very controlled experiments with copious training • General results have not always been positive • Applications to arbitrary settings ? • Temporal nature of mental processes • Neurophysiologic interpretability • Multi-subject analysis
Inspiration Lehmann, 1994
Preliminary Results visuo–spatial working memory
Functional Distance Zt – activation patterns f - transportation
Functional Distance t1 t2 t3
Mental Arithmetic • Involves basic manipulation of number and quantities • Magnitude based system – bilateral IPS • Verbal based system – left AG • Attentional system – ps Parietal Lobule • Other systems – SMA, primary visual cortex, liPFC, insula, etc
Spatial Maps 10 10 same as 8 9 8 8 auditory cortices 7 6 , judgment 6 5 5 Frontal, parietal lobes 4 3 3 visual size estimation 2 1 1 Visual Cortex 0 0s 4s 8s +5.0 -5.0 0
Critique • No neurophysiologic model • Point estimates • Hemodynamic uncertainty • Temporal structure • Functional distance - an optimization problem • No metric structure • Expensive !
Cost Metric Distortion minimizing
Feature Space Φ Orthogonal Bases Graph Partitioning Normalized graph Laplacian of F
State-Space Model Janoos et al., MICCAI 2010
Model Size Selection • Typically strike a balance between model complexity and model fit • Information theoretic or Bayesian criteria • Notion of model complexity • Cross-validation • IID Assumption
Maximally Predictive Criteria • Multiple spatio-temporal patterns in fMRI • Neurophysiological • task related vs. default networks • Extraneous • Breathing, pulsatile, scanner drift • Select a model that is maximally predictive with respect totask • Predictability of optimal state-sequence from stimulus, s
Dyscalculia Difficulty in learning arithmetic that cannot be explained by mental retardation, inappropriate schooling, or poor social environment • Core conceptual deficit dealing with numbers • Very common : 3-6% of school-age children • Heterogeneous
Results Self – same subject Cross – train on one subject and predict on another
Comparing Models Φ1 Subject 1 Φ2 Subject 2 . . . fMRI Data Φ42 Subject 42
Drawbacks • Approximations in the model • Elimination of the activity pattern layer • Spatially unvarying hemodynamics • Unsupervised approach • No explicit link to the experiment • May not necessarily learn relevant patterns
Semi-supervised Approach • Loose dependency between stimulus and signal • Not preclude discovery of un-modeled effects • Stabilize estimation • Generalizable to unconstrained designs • Functionally well-defined representation