Synergistic Use of Data-Driven & Mechanistic Modeling Approaches

1. BMSRNIBIB Synergistic Use of Data-Driven & Mechanistic Modeling Approaches Vasilis Z. Marmarelis Biomedical Simulations Resource Department of Biomedical Engineering University of Southern California

2. BMSRNIBIB Biomedical System Modeling(Class of Problems: Dynamic interrelationships among multiple time-series variables) Data-based & Hypothesis-based Parametric & Nonparametric Input-Output & Compartmental/Mechanistic Static & Dynamic Linear & Nonlinear Stationary & Nonstationary Deterministic & Stochastic Key Issues: Model Specification, Estimation, Validation, Interpretation and Uniqueness

3. BMSRNIBIB Computational/Mathematical Modeling of Time-Series Data Modeling of time-series data is essential because: it organizes quantitatively our knowledge regarding the functional relations among observed variables; it summarizes the observed data in a manner that allows mathematical/statistical inference; it allows rigorous testing of scientific hypotheses; it enables the design of effective control strategies.

4. BMSRNIBIB Data-based Input-Output Modeling A general and canonical representation for all finite-memory nonlinear systems (Volterra functional series): where y(n) is the output, x(n) is the input and { ki} are theVolterra kernels describing the system nonlinearities as a hierarchy of dynamic interactions.

5. BMSRNIBIB Multi-Input Multi-Output (MIMO) Model • The Volterra modeling approach has been extended to systems with multiple inputs and outputs (MIMO), whereby each output y(t) can be expressed in terms of all inputs {xn(t)} using self-kernels and cross-kernels:

6. BMSRNIBIB PDM-based MIMO Models To mitigate the computational problem arising from multi-dimensional high-order kernels, we have employed kernel expansions on the Laguerre basis – with excellent results to date. We also introduced the concept of Principal Dynamic Modes (PDMs) which represent the most efficient basis for kernel expansion. Ifpj is the j-th PDM, xn is the n-th input and uj,n (t) = Σ pj (m) xn (t-m) isthe j-th PDM output, then each output can be expressed as:

7. BMSRNIBIB Structure of PDM Model for Single Input without Cross-Terms Associated NonlinearFunction: f1 y1=f1(u1) u1 y0 Principal Dynamic Mode #1 y2 Principal Dynamic Mode #2 Associated NonlinearFunction: f2 u2 y(t) x(t) ⁞ Principal Dynamic Mode #P Associated NonlinearFunction: fP yP uP

8. Recent Monograph on Methodologies for PDM Modeling of Nonlinear Physiological Systems This monograph provides a comprehensive treatment of the subject, starting with the mathematical foundations of this methodology and presenting the detailed methods that have been developed and used over the past 25 years. The text discusses the key practical issues for the actual application to physiological systems and presents illustrative examples using synthetic and experimental data. The relations among the various modeling approaches – nonparametric, parametric, modular, and connectionist – are discussed in detail, along with their comparative advantages and disadvantages. The key criteria that must guide the successful application of this methodology are identified. The presented selected applications include neural, sensory, cardiovascular, renal, endocrine and metabolic systems. This lucid and comprehensive text is a valuable reference and guide for scientists and engineers who wish to apply nonlinear modeling methods to physiological systems. Please visit www.wiley.com/ieee for ordering information

9. BMSRNIBIB Advocated Approach • Balance model completeness and parsimony. • Data-based model estimation and validation. • Hypothesis-based (mechanistic) model interpretation. • Synergistic useof data-based and mechanistic approaches for model development.

10. BMSRNIBIB Synergistic Use of Data-based & Mechanistic Modeling Key observation: Data-based models can be used to validate postulated mechanistic models and suggest enhancements in model structure. Two case studies: (1) Insulin-glucose interactions (2) Nonlinear synaptic dynamics.

11. Parametric Models of Insulin-Glucose Interactions Minimal insulin-glucose model with 3 parameters for IVGTT: BMSRNIBIB • Extended Minimal insulin-glucose model with insulin secretion r(t): * Model extensions may include the effects of meals, pancreatic glucagon secretion and hepatic gluconeogenesis (among others).

12. Nonparametric PDM Models Equivalent to the Parametric “Minimal” and “Extended Minimal” Insulin-Glucose Models BMSRNIBIB Extended Minimal Model (EMM) Minimal Model (MM) The PDM model of MM resembles one PDM-branch of the EMM

13. The data-based model has a glucoleptic (upper branch) and a glucogenic (lower branch) component, suggesting specific modifications of the previous parametric models: BMSRNIBIB Data-based Insulin-to-Glucose Model of Human Subject z1 =f1(v1) P1(τ) 0.2 0.2 10 10 Nonlinearity f1 v1 v1 0.1 0.1 τ Glucoleptic 0 0 0 0 - - 10 10 z1 PDM 1 v1 - - 0.1 0.1 v1 - - 20 20 - - 0.2 0.2 - - 0.3 0.3 - - 30 30 0 0 100 100 200 200 300 300 400 400 500 500 - - 8 8 - - 6 6 - - 4 4 - - 2 2 0 0 2 2 4 4 Glucose Insulin P2(τ) Time [ Time [ mins mins ] ] τ z2=f2(v2) z2 0 0 60 60 Nonlinearity f2 z2 - - 0.05 0.05 40 40 v2 PDM 2 - - 0.1 0.1 20 20 Glucogenic - - 0.15 0.15 0 0 v2 v2 - - 0.2 0.2 - - 20 20 0 0 100 100 200 200 300 300 400 400 500 500 - - 6 6 - - 4 4 - - 2 2 0 0 Time [ Time [ mins mins ] ]

14. BMSRNIBIB An equivalent linear ODE model can be constructed for each PDM that may represent a system mechanism (state variable): Ak (D) uk(t) = Bk (D) x(t) (k=1,2) where D=d/dt is the differential operator and { Ak, Bk } are polynomials in D obtained via “Realization” methods. The nonlinear output equation is: y(t) = f1 (u1) + f2 (u2) where f1 and f2 are polynomial nonlinearities estimated from data. Transitioning from Data-based to State-Space (Mechanistic) Models

15. BMSRNIBIB Short-Term Synaptic Plasticity Models Mechanism-based Parametric Facilitation-Depression (FD) model: Release Probability: EPSC(t) = F(t) D(t) Facilitation Factor: F(t) = R0 + (1-R0) / [1+KF / CaXF(t)] Depression Factor: dD(t) =[1-D(t)]Kr(CaXD)-F(t0)D(t0) δ(t-t0) Calcium-bound F-mol: dCaXF(t)/dt = - CaXF(t)/TF + δ(t-t0) Calcium-bound D-mol: dCaXD(t)/dt = - CaXD(t)/TD + δ(t-t0) Ca-dependent Recovery: Kr(CaXD) = k0 + (km-k0)/[1+KD/CaXD(t)] where KF and KD are the affinities for CaXF and CaXD respectively, and k0 , km denote the baseline and max rate of refractory recovery.

16. BMSRNIBIB Data-based Model Kernel Estimates for SC Synapse onto CA1 Pyramidal Cell From Experimental data using Poisson input Discrepancies are evident From Simulated data of Parametric Model using the same input

17. BMSRNIBIB Kernel Estimates of Modified Parametric Model with Two Facilitation-Depression Processes Kernels of Fast Process Kernels of Slow Process Kernels Closer to Experimental Kernels of Dual Process Note: The kernel-based model yields better predictions

18. BMSRNIBIB INPUT s1(t) I1 OUTPUTrK(t) …. INPUT sM(t) OK IM pK(t) Scaling up to MIMO Neural Models • Signal transformations from hillock to hillock: • Axon propagation and synaptic transformations • Somato-dendritic integration & generation of action potentials

19. BMSRNIBIB CA1 CA3 Example of Multi-Input / Multi-Output Modeling of CA3-to-CA1 Ensemble Transformations in the Rat Hippocampus LS • Multi-electrode array data-records from 7 neurons at CA3 (inputs) and 5 neurons at CA1 (outputs) are analyzed to obtain the multi-input/multi-output (MIMO) model. • MIMO model is validated via its predictive performance using ROC curves. • The significant inputs and model terms are selected using statistical criteria (the Mann-Whitney test in this case).

20. BMSRNIBIB Functional CA3-to-CA1 Ensemble Connectivity in Rat Hippocampus Right Sample Left Sample Right Nonmatch Left Nonmatch

21. BMSRNIBIB “In medicine, one must pay attention not to plausible theorizing but to experience and reason together. I agree that theorizing is to be approved, provided that it is based on facts and systematically makes its inductions from what is observed.” Hippocrates, “Precepts” Athens, 5th Century B.C.