Distributed Computational Architectures for Integrated Time-Dynamic Neuroimaging

1. Distributed Computational Architectures forIntegrated Time-Dynamic Neuroimaging Dr. Allen D. Malony malony@cs.uoregon.edu Computer & Information Science Department Computational Science Institute CIBER University of Oregon

2. Who Am I • Associate Professor, CIS Department, UO • Computer Science specialties / interests • parallel performance analysis (primary) • environments computational science (secondary) • software development environments • distributed and parallel computing environment • Cognitive Neuroscience interests • two-year association with Don Tucker (Psychology, UO) • Carmel Neuroinformatics workshop (2000, presentation) • HBP Neuroinformatics Review Panel (2000, 2001) • HBP Annual Meeting (2000, presentation) Hill Center

3. Talk Outline • Computational science and cognitive neuroscience • Brain dynamics analysis problem (my view) • integrated electromagnetic analysis system • Motivating case studies • observations: computation and informatics • Computational architectures • models and technology • key ideas • Opportunities and the Neural Informatics Center • Final Thoughts Hill Center

4. Computational Science & Cognitive Neuroscience • Computational methods applied to scientific research • high-performance simulation of complex phenomena • large-scale data analysis and visualization • Understand functional activity of the human cortex • multiple cognitive domains • multiple experimental paradigms and methods • Need for coupled/integrated modeling and analysis • electrical and magnetic, cortical and theoretical • Need for robust tools: computational & informatic • Problem solving environment for brain analysis Hill Center

5. Brain Dynamics Analysis Problem (My View) • Identifyfunctional components in cognitive contexts • Interpret with respect to cognitive theoretical models • Requirements: spatial (structure), temporal (activity) • Imaging techniques for analyzing brain dynamics • blood flow neuroimaging (PET, fMRI) • good spatial resolution  functional brain mapping • temporal limitations to tracking of dynamic activities • electromagnetic measures (EEG/ERP, MEG) • msec temporal resolution to distinguish components • spatial resolution sub-optimal (source localization) • potential to map electrical activity to cortex surface Hill Center

6. Electromagnetic Analysis Methodology • Multi-trial analysis • signal analysis and response analysis • averaging across subjects and trials (S/N ratio) • distortion (smearing) of estimated source response • noise artifacts, signal variation (individuals, trials) • improvements: artifact removal, selective averaging • create component response models • ERP identification • factor analysis: PCA, ICA, … • error in source factors: variability, statistics • Multi-subject and single-subject analysis • quantify differences of individual from population Hill Center

7. Single-Trial Analysis Capability • Improve fidelity of single-subject response model • higher information content than multi-trial/subject • reduce analysis error from trial/subject variability • knowledge of subject population, stimulus deviations • Diagnosis (identification) of cognitive state • known stimulus • blind stimulus • match response to known component response model • Problems • greater noise • greater complexity Hill Center

8. Single-Trial Analysis Methodology • Integrate methods for analyzing brain dynamics • Improve resolution and robustness of techniques • increase measurement density (128 to 256 channels) • Coupled modeling: constraints and cross-validation • component response model  cortical activity model • tuned models for single individual • Build models in experimental paradigm context • Match single-trial measurements to models • known stimulus  multiple trial models • blind stimulus  multiple stimulus/trial models • Training and learning Hill Center

9. Integrated Electromagnetic Brain Analysis Cortical Activity Knowledge Base Head Analysis Source Analysis Structural / Functional MRI spatial pattern recognition temporal dynamics Cortical Activity Model Experiment subject Constraint Analysis Single-trial Analysis EEG MEG Component Response Model neural constraints Dense Array EEG / MEG temporal pattern recognition Signal Analysis Response Analysis Component Response Knowledge Base Hill Center

10. Integrated Electromagnetic Analysis System Carmel Workshop

11. Case Study: Readiness Potential • Self-paced button pressing task • slow negative shifts in potential contralateral to hand • Single subject examination • multi-trial (150 trials) averaged ERP analysis • Dense-array scalp electrical measurement • 129 electrode array (EGI Geodesic Sensor Net) • Modeling of brain electrical activity • MRI and CT data analysis with tissue segmentation • realistic boundary element meshes (2K ’s for brain) • source localization with dipole modeling • Can ERP analysis accurately localize cortical activity? Hill Center

12. Experimental Methodology 16x256bits permicrosec (30MB/m) CT / MRI EEG segmentedtissues NetStation processed EEG BrainVoyager mesh generation, source localization constrained to cortical surface EMSE Interpolator 3D Hill Center

13. Electrical Activity of Scalp and Brain Lateralize Readiness Potential (LRP) • Expected brain activity • Correlated with fMRI experimental studies • Topographic and cortex mapped spatial analysis -404 ms -56 ms 0 ms 160 ms Hill Center

14. Optimizing Spatial Resolution for ERP • Adequate spatial sampling • Accurate head surface mapping • Accurate sensor registration • Measured skull conductivity • Convergence with MEG  MEG-compatible EEG • Convergence with fMRI  fMRI-compatible EEG • Test spatial resolution with know pathological sources • EEG as link for converging analysis? • What problems exist? Hill Center

15. Electrical Impedance Tomography • Small (10µA) currents are injectedbetween electrode pair • Resulting potential is measuredfrom all remaining electrodes • Measures used to estimateconductivity of each tissue compartment • Boundary element forward solution • 4-shell polyhedron model (1280 faces) • direct (31244 sec) and iterative approaches (933 sec) • Finite element forward solution • greater computational requirements Hill Center

16. Case Study: Self-Monitored Motivated Action • Learning task with feedback (Gehring et al. (1993)) • left- or right-hand button press response • "incorrect" feedback on error • "OK" or “late” feedback if correct • timed expectancy and motivated response • Error-Related Negativity (ERN) • large medial negative response on error • self-monitoring when motivated action goes wrong • What is the nature and complexity of the ERN with respect to dynamic components of brain activity? Hill Center

17. Visualize the dynamic operations of brain Example: fMRI blood flow response to reading a word Dense-array EEG / MEG frontal lobe activity (ERN) significant changes in milliseconds frontal oscillations and separate time courses BrainVoyager Cognitive Experiments and Brain Dynamics Hill Center

18. ERN Analysis using ICA (Makeig, Salk Institute) • Average analysis smears temporal/spatial dynamics • Single-trial analysis may expose greater detail • Independent Components Analysis (ICA) • find independent EEG component contributors • temporal and spatial • components accounting for artifacts • components accounting for functional sources (ERN) • analysis over single trials • Two components account for averaged ERN • response-locked ERN difference wave dominated • show temporal and functional independence Hill Center

19. ERP and Component Envelopes (Left/Correct) Component #2 Component #7 • Complementary • behavior • Both active at strongest ERN channels Hill Center

20. ERPs averaged across response hand Neither C2 nor C7 explain the waveforms Component sum does explain the waveforms and shows ERN response Hill Center

21. Topographic Imaging and Dipole Modeling Component #2 Component #7 Averaged ERN Brain Electrical Source Analysis (BESA) Hill Center

22. ICA Component #2 Dynamics • Stimulus locked • Memory of deadline Hill Center

23. ICA Component #7 Dynamics • Phase reset byresponse, largestafter incorrect Hill Center

24. Optimize Temporal Information • Inherent problem – both electrical and magnetic • Trial averaging methodologies can mask dynamics • Techniques to boost signal to noise ratio • Selective averaging • Stimulus and response locking • Techniques to estimate time function • fMRI timing models • EEG/MEG time function for fMRI signal extraction • Single trial analysis with individual modeling • What problems exist? Hill Center

25. Case Study Observations • Diverse set of tools • function and implementation • separate tools (monolithic) and not integrated • incompatibilities and limitations for interoperation • Complex analysis processes • multiple processes applied (process pipeline) • high-level, hierarchical process methodology • scientific discovery through integrated techniques • heterogeneous, flexible, extensible capabilities • increasingly high computational demands • Multiple, interdisciplinary scientific domains Hill Center

26. High-Performance Computational Environments • Integrated database, analysis, and visualization • Distributed tool infrastructure • diverse tools across multiple platforms • interoperation requirements • user interaction requirements • support portability, flexibility, extensibility • Scalable, high-performance parallel computing • increase data resolution • minimize solution time • High-level access to tools • web-based access Hill Center

27. Domain-specific, problem-specific environments (PSE) TIERRA Scientific “workbench” SCIRun Programming environments numerical frameworks POOMA application coupling PVM / MPI CUMULVS PAWS SILOON / PDT Metacomputing / GRID Legion Globus Heterogeneous distributed computing / coupling NetSolve INTERLACE HARNESS Web-based environments ViNE PUNCH VNC Computational Systems: Models and Technology Hill Center

28. TIERRA (Computational Science Institute, UO) • Tomographic Imaging Environment for Ridge Research and Analysis • High-performance, domain-specific environment for seismis tomography • parallelized tomography code • runtime distributed array access • computational steering via MatLab frontend • full problem solving process for seismic tomography • Led to new discoveries for three-dimensional melt migration beneath the East Pacific Rise Hill Center

29. TIERRA Architecture • KEY IDEAS • Domain specific • Support for the entire process Hill Center

30. SCIRun (Johnson, University of Utah) • Scientific programming environment • large-scale simulations • “computational workbench” • visual programming interface • dataflow model of computing • modules: operation or algorithm with I/O ports • network: set of modules and their interconnections • widgets: 3D user interaction • data types: Mesh, Surface, Matrix, Field, Geometry • extensible module library • computational steering Hill Center

31. Visual programming lets users select, arrange, and connect modules into a desired network Interactive steering of design, computation, and visualization allows more rapid convergence SCIRun User Interface Hill Center

32. ICA for EEG Source Localization with SCIRun • PCA decomposition forEEG signal/noise subspaces • ICA activity map separationon signal subspace • Solution to a single dipolesource forward problem • underlying model is shownin the MRI planes • dipole source is indicated by red and blue spheres • electric field visualized by cropped scalp potential map and wire-frame equipotential isosurface • KEY IDEAS • Integrated application development environment • “Component-based” application programming • High-level data objects Hill Center

33. POOMA (Advanced Computing Lab, LANL) • Parallel Object-Oriented Methods and Applications • Goals • use object-oriented programming to help manage complexity of modern scientific simulation codes • extract physics content of simulations from details of parallel, high-performance computing • framework approach: allows flexible code structure, object reuse across problem domains • build upon standards to maintain code portability • An object-oriented framework for scientific computing applications on parallel computers Hill Center

34. POOMA Approach • C++ class library • high-level, generally data-parallel API • Generic programming • classes modeled after STL style • heavy use of C++ templates • Parallelism encapsulated • message-passing for distributed memory machines • multi-threaded shared memory (POOMA II) • Cross platform code development and scalable parallelism Hill Center

35. Application MC++ NTTP LINAC Algorithm Differential Operators Interpolators FFT Physics Global Fields Meshes Particles Computer Science Parallel Load Balancing Domain Decomposition Message Passing Compile-time Polymorphism Local Expression Templates STL POOMA Framework • KEY IDEAS • Numerical programming framework • Encapsulated parallelism • High-level API’s / data support Hill Center

36. PDT (Malony, University of Oregon) • Program Database Toolkit • Program analysis • multi-language(Fortran, C,C++, Java) • commercial-grade parsers • IL to programdatabase (PDB) • API for PDBaccess / query • Tools: instrumentation, code wrapping, documentation Hill Center

37. SILOON (Advanced Computing Lab, LANL; UO) • Scripting Interface Language for OONumerics • Toolkit and run-time support for building easy-to-use external interfaces to existing numerical codes • Scripting language to “glue” components together • KEY IDEAS • Support for application interaction control • Support for application code wrapping • Application / tool coupling • Data exchange support Hill Center

38. Metasystems and Metacomputing • Many resources accessible on the internet • computers, data, devices, people • Extend single system model to internet domain • wide-area (department, campus, region, country) • scalable, transparent access to resources • hides network complexity (“as if on your machine”) • Extend computing model to internet domain • shared persistent space of objects (data, execution) • heterogeneous distributed and parallel processing • meta-applications (multi-component, hierarchical) • Deal with complex environment / primitive tools Hill Center

39. “The GRID” • New applications based on high-speed coupling of people, computers, databases, instruments, ... • computer-enhanced instruments • collaborative engineering • browsing of remote datasets • use of remote software • data-intensive computing • very large-scale simulation • large-scale parameter studies Hill Center

40. GRID Architectural Picture • KEY IDEAS • Metasystems infrastructure / services • Metacomputing applications programming • GRID resources Hill Center

41. NetSolve (Dongarra, University of Tennessee) • Client-server systemto access distributedcomputational / DBHW/SW resources • Distributed computing:resources, processes,data, users • Load-balancing policy for efficiency / performance • Integration with arbitrary software components • C, Fortran, Java, MatLab, Mathematica, Excel • BLAS, (Sca)LAPACK, MINPACK, FFTPACK Hill Center

42. NetSolve Usage • “Blue collar” GRID-based computing • users can set things up (without “su” privileges) • no deep network programming knowledge required • Scenarios • clients, servers, and agents anywhere on Internet • clients, servers, and agents on an Intranet • clients, servers, and agent on the same machine • Focus on MATLAB users • OO-style language (objects are matrices) • one of most popular desktop systems for numerical computing (> 400K users) Hill Center

43. NetSolve – The Client • NetSolve API hides complexity of numerical software • Computation is location transparent • Provides access to virtual libraries: • Component GRID-based framework • Central management of library resources • User not concerned with most up-to-date versions • Automatic tie to Netlib repository • Synchronous or asynchronous calls • User-level parallelism Hill Center

44. NetSolve – The Agent and Server • Agent • gateway to computational services • performs load balancing and resource management • Server • various software installed on various hardware • configurable and extendable • framework to easily add software • many numerical libraries being integrated • supports parallel computing Hill Center

45. MCell (Bartol, Salk Institute; Salpeter, Cornell) • Monte Carlo simulator of cellular microphysiology • Study how neurotransmitters diffuse and activate receptors in synapses between different cells • NetSolve distributesprocessing workloadand allows access tocomputational resources • Simultaneous evaluationof large number ofdifferent parametercombinations Hill Center

46. INTERLACE (Malony, University of Oregon) • INTERoperation and Linking Architecture for Computational Engines • Goals • framework for building high-performance computing environments from existing tools • reusable components in heterogeneous environment • abstract connection mechanisms for control/data flow • resource management for dynamic operation • use standard software technologies • parallel and distributed computational environments • http://www.cs.uoregon.edu/research/paracomp/proj/interlace/ Hill Center

47. INTERLACE Components • Computational engines: libraries or programs providing specific functions • Computational server: program interfacing multiple engines with middleware • Wrappers: server-engine interface for data/control • Middleware: server-to-server interoperation software • KEY IDEAS • High-level numeric computational services • Access to metasystem resources • Wrapping/linking of computational engines • Dynamic, adaptable, extensible • High-level metasystems programming support Hill Center

48. ViNE (Malony, University of Oregon) • Virtual Notebook Environment • High-level, sharednotebooks, data, andtools in distributed,heterogenous system • Architecture • leaves: notebookfunctions and data • stems: notebookcommunication • Web-based access Hill Center

49. ViNE Experiment Builder • List of available, named data, tools, and experiments • Visual dataflow model of experiment process • Wrapped tools and databases wrapped MATLAB “tool” Hill Center

50. Brain Electrophysiology Lab Notebook • Dense array EEG datasets • Commercial of the shelf statistical and numerical packages • Multiple machines types • Notebook content automatically generated from experiment results Hill Center