1 / 33

Sponsored by US Army Research Office STIR: Advanced Estimation of Cognitive Status

PLS Tools in Electroencephalography Leonard J. Trejo PDT Institute Palo Alto, CA 94303, USA The 6 th International Conference on Partial Least Squares and Related Methods Sept. 4 th – 7 th , 2009 Beijing, China. Sponsored by

lonato
Download Presentation

Sponsored by US Army Research Office STIR: Advanced Estimation of Cognitive Status

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. PLS Tools in ElectroencephalographyLeonard J. TrejoPDT InstitutePalo Alto, CA 94303, USAThe 6th International Conference on Partial Least Squares and Related MethodsSept. 4th – 7th, 2009Beijing, China • Sponsored by • US Army Research Office STIR: Advanced Estimation of Cognitive Status • Contract No. W911NF-08-C-0121 15-SEP-2008 TO14-MAR-2009

  2. PDT and PDT Institute • PDT • Neuroergonomics Models and Applications • Human-System Integration • Human Performance Optimization • Robust Biomedical Signal Processing • Embedded and Real-time Systems for Bio-Sensing • PDT Institute • PhD/Masters/Undergraduate Training • University Partners (UC Santa Cruz, Tsinghua University, UC San Diego, Univ. of West Florida)

  3. When I am not working…

  4. Outline • Problem: stress, workload, fatigue and performance • Response: Neuroergonomic models and control systems • Create useful definitions of cognitive states • Model, estimate and control cognitive states • Background • Multimodal sensor-state models using PLS and KPLS Algorithms • Successes and failures: fatigue / BCI / engagement / workload • New directions • Truly multidimensional sensor-process models • PARAFAC, N-PLS • Summary

  5. Estimation of Cognitive States Biosignals Models and Algorithms

  6. Useful Definitions • Engagement: selection of a task as the focus of attention and effort • Workload: significant commitment of attention and effort to an engaged task • Visual, Auditory, Haptic • Psychomotor • Cognitive (memory, executive) • Overload: task demands outstrip performance capacity • Mental Fatigue: desire to withdraw attention and effort from an engaged task associated with extended performance (~45 min)

  7. Electroencaphalogram • Cerebral Cortex • the outermost layers of brain • 2-4 mm thick (human)

  8. EEG Sources A pyramidal neuron with a soma, apical & basal dendrites and a single axon

  9. EEG Sources A pyramidal neuron with a soma, apical & basal dendrites and a single axon

  10. Other Elements of Sensor-State Models

  11. Successful Application 1: Mental Fatigue Black = Alert Red = Mentally Fatigued Parietal Fz Pz Alpha Frontal Theta

  12. Robust EEG-Based Classification of Mental Fatigue 2300 (Day 1) vs. 1900 Hrs (Day 2) 100 90 80 Test Proportion Correct 70 60 50 40 -18 -15 -12 -9 -6 -3 0 50 56 79 94 99 100 100 21 Channels 50 51 65 88 96 98 98 12 Channels 50 54 76 88 90 88 87 4 Channels Signal-to-noise Ratio (dB)

  13. Successful Application 2: BCI

  14. Successful Application 2: BCI

  15. Stress, Workload, Fatigue and Performance Trejo, et al. ACI 2007

  16. Cognitive Overload (Trejo, et al. ACI 2007) Trejo, et al. ACI 2007 Trejo, et al. ACI 2007

  17. Stabilizing Classifiers

  18. Multimodal Overload Patterns Value Time (s)

  19. Workload-related EEG Sources

  20. Application Summary • Individual models of engagement and fatigue: • 90-100% accurate • Stable within a day • Stable from day to day • Individual models of workload or effort: • 60-90% accurate • Moderately stable within a day • Unstable from day to day • Normative models (limited data): • 50-70% accurate • Moderately stable

  21. Recommended Directions • Deployable multimodal sensors (EEG, fNIR, EOG, gaze, HRV, EMG, SCR, SpO2,BP, core body temperature, gesture, posture facial expression, ...) • Multimodalexperimental designsandoperational tests • Advanced neurocognitive process models • Multimodal sensor-processmapping algorithms APECS

  22. “Atomic” Decomposition • “In the parlance of modern harmonic analysis (Chen and Donoho, 2001), we performed a space/ frequency/time ‘‘atomic decomposition’’ of multidimensional data. In other words, we assume that each neural mass contributes a distinctive atom to the topographic frequency/time description of the EEG, so that the estimation of these atoms is possible by means of signal-processing techniques. Each atom will be defined by its topography, spectral content, and time profile; in other words, by its spatial, spectral, and temporal signatures.” • Fumikazu Miwakeichi, et al, Decomposing EEG data into space–time–frequency components using Parallel Factor AnalysisNeuroImage 22 (2004) 1035–1045. • Chen, S., Donoho, D., 2001. Atomic decomposition by basis pursuit. SIAM Rev. 43, 129– 159.

  23. “Atomic” EEG Elements Basic Sources“atoms” CoherentSystems “molecules” Atoms Molecule Covalent Bonds Coherence Bonds

  24. “Molecular” EEG Processes Coherence Bonds Atoms

  25. bf af Familiar (bilinear) Mapping Algorithms Factor Analysis Principal Component Analysis (PCA)

  26. Multimodal Mapping • How to generalize bilinear models to systems with more dimensions? • Unfolding a bilinear model • Represent all experimental factors in one dimension • Observations (trials) is second dimension • Contrast each dimension vs. pairs of the other two • Multidimensional model • Assume orthogonal factors: PARAFAC • Allow interacting factors: Tucker 3 • Modeling approach • Unsupervised extraction: PARAFAC, CANDECOMP, Tucker 3 • Supervised extraction: N-PLS

  27. X X1 X2 X3 Dim 2 & 3 Dim 1 & 3 Dim 1 & 2 Dim 2 Dim 3 Dim 1 Unfolding a Bilinear Model X Unfolding

  28. Multidimensional Modeling(Tucker 3 Model, unsupervised) • xijkis an element of (l x m x n)multidimensional array • F1, F2, F3are the number of components extracted on the 1st, 2nd and 3rd mode • a, b, c are elements of the A, B, C loadings matrices for the 1st, 2nd and 3rd mode • gare the elements of the core matrix G which defines how individualloading vectors in different modes interact • eijkis an error element (unexplained variance)

  29. PARAFAC (Parallel Factor Analysis, unsupervised) PARAFAC is a special case of the Tucker 3 model where F1= F2 = F3=F and G = I For a 3-way array: bf cf af

  30. electrodes af – spectral atom bf – spatial atom cf – temporal atom EEG time frequency max. covariance vf – workload atom uf – temporal atom X time Labels workload condition N-way PLS(supervised) X cf uf bf af vf

  31. Demo: Workload / PARAFAC EEG Workload condtions (e.g., trials, time) Electrodes EEG Frequency

  32. Summary • Successes and Failures • Individual models of engagement and fatigue: accurate, stable • Individual models of workload or effort: variably accurate, unstable • Normative models (limited data): inaccurate, unstable • Useful models of state-related EEG sources • “Atomic” EEG sources • “Molecular” EEG systems • Approaches to multidimensional models and algorithms • Tradtional bilinear methods (PCA, factor analysis, ICA) • Truly multidimensional methods • Correlated factors (Tucker 3) • Uncorrleated factors (PARAFAC, CANDECOMP, N-PLS) • Supervised algorithms (N-PLS)

  33. Within-day Results

More Related