Tom Musicka Centre for Process Analytics and Control Technology University Of Newcastle

1. The Incorporation of Prior Knowledge and Constraints through Bayesian Analysis into a Neuro-Fuzzy Inference System Framework Tom Musicka Centre for Process Analytics and Control Technology University Of Newcastle Contact : tom.musicka@ncl.ac.uk

2. Introduction • Problem Definitions • Data Analysis • PreScreening • Data Validation • Neuro-Fuzzy Modelling • Model Comparison • Results • Bayesian Framework • Results • Conclusions • Future Work

3. Data Mining • Data Capture and Treatment • ‘Analysis of observational data sets to find unsuspecting relationships and to summarise the data in a number of understandable and useful ways’ • Prediction • Classification • Detection of Relationships • Explicit Modelling • Clustering • Deviation Detection

4. Corus - Process and Scheduling Description • Hot Strip Steel Rolling - Corus • Multigrade Operation • Multistage Process • Reheating • Multi-Pass Reversing Rougher • Multi-Pass Finishing Rolling • Run Out Table (ROT) Cooling • Coiling • Testing

5. Corus - Problem Definition • Virtual Test House • Mechanical Property Prediction • Ultimate Tensile Strength • Yield Stress • Elongation • Project Description: • ‘To produce a modelling system capable of accurately and consistently predicting the the mechanical properties of Steel rolled at Hot Strip Mills within Corus UK.’

6. Shell - Process and Scheduling Description • Refinery High Viscosity Index (HVI) Operation - Shell • Multicrude Operation • Continuous and Batch Operations • Pressure to Minimise Working Capital • Statistical Visualisation • Masters Thesis • Four Unit Process • High Vacuum Unit (HVU) • Furfural Extraction Unit (FEU) • Propane De-Asphalting Unit (PDU) • MEK De-Waxing Unit (MDU)

7. Shell - Problem Definition • Slack Wax Oil Content (SWOC) Prediction • MDU Produces Lube Oil and Slack Wax • Slack Wax Contains Oil • Useful to know the % Oil Content • Project Description: • ‘To produce a model capable of accurately and consistently predicting the SWOC. This Model will be considered within the feasibility report for the purchase of an online physical measurement system’

8. Data Characteristics • Multiple Input, Single Output Models • Discrete and Continuous Variables • Missing Data • Process Noise • Outliers • Multiple Operating Regions • Strong Clustering • Overlapping Regions • Local Modelling Techniques • Non-Linear Relationships

9. Data Screening • Univariate Checks • Time Series Visualisation • Multivariate Statistics • Principal Component Analysis • Outlier Detection • Process Visualisation • Multi-Modal Data • Local PCA Models • Process Grades • Fuzzy Clustering • MixPCA

10. Data Analysis - Principal Component Analysis

11. Local Principal Component Analysis

12. Data Splitting • Data is Divided into 3 Data Sets • Training • Data to Build Model • Validation • Data used to Define Training Stopping Epoch • Improve Generalisation • Testing • Data Used to Quote Model Results • Unseen by Model in Training Phase • Ratio 70:15:15

13. Data Splitting • Outliers Removed • Fully Populated Data Set • Require Representation of Full Scope of Process Within Training Data • Random Selection • Sample from Data Distribution • With Respect to Operating Region • Kennard-Stone Selection • Sample from Flat Distribution • Mahalanobis Distance Metric for Selection • Analysis for Each Operating Region • Ensures Training Set Covers Full Scope

14. Advanced Modelling Studies • Projection to Latent Structures • Linear • Predictive Extension of PCA • Neural Networks • Non-linear • ‘Black-Box’ • Neuro-Fuzzy Inference Systems • Adaptive Neuro-Fuzzy Inference System - ANFIS • Fuzzy Rulebase - Model Switching / Merging • Local Linear Models • Compromise between Accuracy, Interpretation and Simplicity

15. Input Data S Input Fuzzification Normalised Rulebase Local Models Network Output ANFIS Output ANFIS Structure

16. Corus - Model Comparison Results

17. Shell - Model Comparison Results

18. Model Comparison Results - Corus and Shell

19. Bayesian Methods • Mathematical Foundation to Model Optimisation • Incorporates Use of Prior Knowledge • Advantage over Maximum Likelihood Models • Bayes Rule • Posterior = (Likelihood * Prior)/Evidence • Evidence is Constant for Given Model Structure

20. Bayesian Methods • Posterior - p(w|D,H) • Probability of Parameters, w, Representing Data, D, for Given Model, H • Likelihood - p(D|w,H) • Probability of the Model Fitting the Data • Accounts for Least Squares Modeling Power • Prior - p(w|H) • Incorporates Prior Knowledge into Training • Model Constraints • Independent of Data

21. Bayesian Methods • Bayesian Model Averaging • Accounts for Model Uncertainty • Avoids Use of Single Model • Prior Distribution • Incorporation of Process Knowledge • Favour Certain Characteristics • Model Interpretability • Model Accuracy

22. Bayesian ANFIS - BANFIS • Single ANFIS Structure • Process Knowledge • Cluster Analysis • Bayesian Model Averaging • Sample Rulebase Parameters • Markov Chain Monte Carlo • Rejection Sampling • Single Local Model Learning Stage • Cover Important Parameter Space

23. BANFIS - Results Comparison

24. BANFIS - Results Comparison - Corus

25. BANFIS - Results Comparison - Shell

26. BANFIS - Advantages and Disadvantages • Advantages • Improved results • Incorporates Parameter Uncertainty • Avoids Optimisation Training Problems • Avoids Single Network Solution • Incorporation of Prior Knowledge • Disadvantages • Complex Hierarchical Analysis • Network Interpretation Issues

27. Conclusions • Data Validation Methods • Screening • Local Models • Essential for Non-linear Modelling Methods • Compromise between: • Accuracy • Interpretation • Model Complexity • Results • Improvements in Accuracy and Data Analysis • Implementation of Bayesian Framework

28. Acknowledgements • CPACT • TCD • Paul Kitson - Corus • Phil Jonathan, Paul Blackhurst - Shell