Gaussian Process Model Identification : a Process Engineering Case Study

1. Gaussian Process ModelIdentification: a Process Engineering Case Study Juš Kocijan1,2, Kristjan Ažman1, 1Jožef Stefan Institute, Ljubljana, Slovenia 2University of Nova Gorica, Nova Gorica, Slovenia Systems Science XVI, September 2007, Wroclaw

2. Motivation: • Topic: nonlinear dynamic systems identification • Problem: unballance between number of measurements in equilibrium and out of equilibrium • Theoretical solution:Gaussian process model with incorporated linear local modelsproblem solution + measure of confidence in prediction • Validation of theory: application in a process engineering case study Systems Science XVI, September 2007, Wroclaw

3. Overview: • Modelling with Gaussian process (GP) priors • Incorporation of linear local models • Modelling case study of gas-liquid separator Systems Science XVI, September 2007, Wroclaw

4. Identification – why and how • Dynamic system identification  model  e.g. prediction, automatic control, ... • Nonlinear dynamic system identification • problems  ANN, fuzzy models, ... • difficult to use (structure determination, large number of parameters, lots of training data)  • GP model – reduces some of these problems Systems Science XVI, September 2007, Wroclaw

5. y | p(y) x=x0 * * * * * x x0 GP model • Probabilistic, non-parametric model, constituted of: • covariance function • input/output data pairs (points, not signals) Prediction of the output based on similarity test input – training inputs Normally distributed output: Systems Science XVI, September 2007, Wroclaw

6. Gaussian processes • Gaussian process – set of normally distributed random variables: • mean μ(X) • covariance matrix K(X) • Covariance function Gaussian • Optimisation: • cost function: log-density • method: maximum likelihood • optimisation: conjugate gradients Systems Science XVI, September 2007, Wroclaw

7. GP model attributes (vs. e.g. ANN) • Smaller number of parameters • Measure of confidence in prediction, depending on data • Incorporation of prior knowledge * • Easy to use (practice) • computational cost increases with amount of data  • Recent method, still in development • Nonparametrical model * (also possible in some other models) Systems Science XVI, September 2007, Wroclaw

8. y x GP model Static example y = f(x) = = cos (6x2) Systems Science XVI, September 2007, Wroclaw

9. GP model Dynamic system • Input/output training pairs xi/yi xi ... regressor values [ut-1,..,ut-k, yt-1,..,yt-k] yi ... system outputyt • Simulation • “naive” ... m(k) Systems Science XVI, September 2007, Wroclaw

10. Problem of nonlinear dynamic systems identification Engine example – longitudinal dynamics Systems Science XVI, September 2007, Wroclaw

11. Incorporation of local linear models (LMGP model) • Derivative of function observed beside the values of function • Derivatives are coefficients of linear local model in an equilibrium point (prior knowledge) • Covariance function to be replaced; the procedure equals as with usual GP • Very suited to data distribution that can be found in practice Systems Science XVI, September 2007, Wroclaw

12. Systems Science XVI, September 2007, Wroclaw

13. Case study: gas-liquid separator Systems Science XVI, September 2007, Wroclaw

14. Nonlinearity of the system Model structure: Systems Science XVI, September 2007, Wroclaw

15. Model identification • Seven equilibrium points • Seven linear LM (14 points) • 60 off equilibrium points Systems Science XVI, September 2007, Wroclaw

16. Model validation SE=0.00056 LD=-1.97 Systems Science XVI, September 2007, Wroclaw

17. Model validation Systems Science XVI, September 2007, Wroclaw

18. Conclusions • The Gaussian process model is an example of a flexible, probabilistic, nonparametric model with inherent uncertainty prediction. • The GP model with incorporated local linear models (LMGP) is a possible solution for the problem of measurement data distribution in equilibrium and out of equilibrium. • The application of LMGP modelling method on a gas-liquid separator demonstrated feasibility of this solution in practice. Systems Science XVI, September 2007, Wroclaw