Dynamic Systems Identification with Gaussian Process Models

1. Dynamic Systems Identification with Gaussian Process Models Juš Kocijan Jožef Stefan Institute, Ljubljana, Slovenia & University of Nova Gorica, Nova Gorica, Slovenia Seminar České Společnostipro Kybernetiku a Informatiku , October 2009, Prague, Czech republic

2. Motivation: • Topic: on overview of dynamic systems identification with Gaussian process models (GP models) • Problem: application of machine learning approach for dynamic systems modelling and its applications • Theoretical solution: conventional approach: delayed input and output values as regressors • Validation of theory: applications in various domains since 1999

3. Identification – why and how • Theoretical modelling – first principles modelling vs. identification • Dynamic system identification  model  e.g. prediction, automatic control, ... • Nonlinear dynamic system identification • problems  ANN, fuzzy models, ... • difficult to use (structure determination, large number of parameters, lots of training data) • the issue of confidence in model • GP model – recent complementary approach

4. Historical overview • GP literature in the field of statistics, where this approach originates • `Kriging´ in the geophysics literature • GP in curve fitting and regression: O’Hagan, 1978 (triggered no special attention) • Relation with ANN: Neal, 1996 • Further developments of GP regression: Williams and Rasmussen, 1996 (modelling of static non-linearities) • use of GP for dynamic systems: EU 5th framework RTN - MAC project (2000-2004) • GP models, GP priors, GP regression, GP Dynamical Models

5. http://dsc.ijs.si/jus.kocijan/GPdyn/

6. GP model • Probabilistic (Bayes) model. • Nonparametric model – no predetermined structure (basis functions) depending on system • Determined by: • Input/output data (data points, not signals) (learning data – identification data): • Covariance matrix:

7. Gaussian process model Bayes based modelling y x GP model

8. GP model • Prediction of the output based on similarity test input – training inputs • Output: normal distribution • Predicted mean • Prediction variance

9. Covariance function • Covariance function: • functional part and noise part • stationary/unstationary, periodic/nonperiodic, etc. • Expreses prior knowledge about system properties, • frequently: Gaussian covariance function • Smooth function • Stationary function

10. Hyperparameters • Identification of GP model = optimisation of covariance function parameters • Optimisation: • Cost function: maximum likelihood of data for learning

11. Nonlinear fuctionand GP model 10 8 Funkcija, ki jo želimo identificirati 8 y=f(x) 6 učne točke 6 4 y 2 4 y x GP model 0 y 2 -2 0 -4 Learning points m ± s 2 -2 -6 m -1.5 -1 -0.5 0 0.5 1 1.5 2 f(x) x -4 Prediciton error and double standard deviation of prediction -1.5 -1 -0.5 0 0.5 1 1.5 2 x s 2 6 |e| 4 e 2 0 -1.5 -1 -0.5 0 0.5 1 1.5 2 x Static illustrative example • Static example: • 9 learning points: • Grey band: • Rare data density  increased variance (higher uncertainty).

12. Dynamic systems • Static vs. Dynamic • Dynamic models: conventional approach (ANN, fuzzy models, etc.) is delayed inputs and outputs as regressors • Input/output training pairs xi/yi xi ... regresor values (GPARX model) [u(k-1),..,u(k-L), y(k-1),..,y(k-L)] yi ... system outputvalues y(k)

13. GP model Dynamic system identification and model simulation • Why does identification of dynamic systems seem more complex than modelling of static functions? • Simulation • “naive” ... m(k) • with propagation m(k),v(k) • Analytic app. • Taylor app. • “exact” • MC Monte Carlo with mixtures

14. GP model attributes (vs. e.g. ANN) • Smaller number of parameters • Measure of confidence in prediction, depending on data • Incorporation of prior knowledge * • Easy to use (practice) • Computational cost increases with amount of data  • Recent method, still in development • Nonparametrical model * (also possible in some other models)

15. Identification challenges • Methodology of experimental modelling for dynamic systems based on GP models • Procedure, suggestions, examples, etc. • Incorporation of prior knowledge • Nonparametric model  Utility of the method

16. Infl. rate u Concentration of microorganisms x1 x1 … concentration of microorganisms, x2 … substrate concentration, u … inflow rate, 0 ≤ u(k) ≤ 0.7, … white Gaussian noise, σ = 0.5% (ymax- ymin). Substrate concentration x2 Outfl rate u Identification case study – Bioreactor Identification procedure, properties of obtained GP model  Bioreactor – discrete nonlinear 2nd order dynamic system:

17. Bioreactor (2) • Defining the purpose of the model: response prediction • Model selection: • Gaussian covariance function (stationarity, smoothness) • Regressors selection • Design of the experiment: • Input/output signal  • ~ 600 data for identification • Realisation of the experiment, data processing Input and output signals used for generating data for identification [y(k-1)…y(k-L) u(k-1)…u(k-L)] [y(k-1)…y(k-L) u(k-1)…u(k-L)], y(k)

18. Bioreactor (3) • Model training: • Optimisation of hyperparameters • Model validation: • plausibility (“looks”, “behaves” logical) • falseness (I/O inspection) • purposiveness(satisfaction of the purpose)

19. Bioreactor (4) • Model validation: • plausibility: • qualitative (visualy from I/O response) • quantitative – cost functions: • Mean squared error (MSE), • Mean relative square error (MRSE), • Log predictive density error (LD), • Negative log-likelihood of the training data (LL). variance

20. Bioreactor (5) • validation  L=2 • ARD  the number of regressors is reduced [u(k-1) u(k-2) y(k-1) y(k-2)] [u(k-1) u(k-2) y(k-2)]

21. Bioreactor (5) Simulation in the trained region, but not with the identification signal Simulated response with 95% confidence band and error

22. Bioreactor (6) Simulation in the not modelled region: u(k) > 0.7. Simulation result in the not modelled region

23. Bioreactor (7) More noise: σ= 2%(ymax- ymin) Increased variance: • Increased noise • Insufficient data Simulation result with more noise in identification signal, s=0.002

24. Bioreactor (8) Unmodelled input z=0.05 for t>=30s  Prediction confidence is not changed in the modelled region Simulation result in the case of not modelled input, z=0.05 for t>=30 s

25. Applications and domains of use • dynamic systems modelling • time-series prediction • dynamic systems control • fault detection • smoothing • chemical engineering and process control • biomedicalengineering • biological systems • environmentalsystem • power systems and engineering • motion recognition • traffic

26. Incorporation of local linear models (LMGP model) • Derivative of function observed beside the values of function • Derivatives are coefficients of linear local model in an equilibrium point (prior knowledge) • Covariance function to be replaced; the procedure equals as with usual GP • Very suited to data distribution that can be found in practice J. Kocijan and A. Girard. Incorporating linear local models in Gaussian process model. In Proceedings of IFAC 16th World Congress, Praga, 2005.

28. Applications for control design General model based predictive control principle Cost function (PFC) constraints on input signal, input signal rate, state signals, state signals rate and constrained optimisation – SAFE CONTROL J. Kocijan and R. Murray-Smith. Nonlinear predictive control with Gaussian process model. In Switching and Learning in Feedback Systems, volume 3355 of Lecture Notes in Computer Science, Pages 185-200. Springer, Heidelberg, 2005. B. Likar and J. Kocijan. Predictive control of a gas-liquid separation plant based on a Gaussian process model. Computers and Chemical Engineering, Volume 31, Issue 3, Pages 142-152, 2007.

29. pH process: control results – constrained case (constraint on variance only)

30. pH process: control results – constrained case (constraint on variance only) Next step: Explicit Nonlinear Predictive Control Based on Gaussian Process Models A. Grancharova, J. Kocijan and T. A. Johansen. Explicit stochastic predictive control of combustion plants based on Gaussian process models. Automatica, Volume 44, Issue 6, Pages 1621-1631, 2008.

31. Application of GP models for fault diagnosis and detection • Is the fault diagnosed because of the fault occurance or because model is not OK?

33. Dj. Juričić and J. Kocijan. Fault detection based on Gaussian process model. In I. Troch and F. Breitenecker, editors, Proceedings of the 5th Vienna Symposium on Mathematical Modeling (MathMod), Vienna, 2006.

34. Conclusions • The Gaussian process model is an example of a flexible, probabilistic, nonparametric model with inherent uncertainty prediction • It is suitable for dynamic systems modelling • When to use GP model? • systems: nonlinearity, corrupted data (noise, uneven distribution), insufficient prior knowledge, uncertainty • biological, environmental systems, etc. • A case study for the illustration of identification procedure.

35. Conclusions • The research of GP modells for dynamic systems is growing. • Further work: • Software • Analytical tools • Application niches