1 / 78

Identification and Neural Networks

Identification and Neural Networks. G. Horv áth. I S R G. Department of Measurement and Information Systems. Identification and Neural Networks. Part III Industrial application. http ://www.mit.bme.hu/~horvath/nimia. Overview. Introduction Modeling approaches Building neural models

bowena
Download Presentation

Identification and Neural Networks

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Identification and Neural Networks G. Horváth I S R G Department of Measurement and Information Systems NIMIA Crema, Italy

  2. Identification and Neural Networks Part III Industrial application http://www.mit.bme.hu/~horvath/nimia NIMIA Crema, Italy

  3. Overview • Introduction • Modeling approaches • Building neural models • Data base construction • Model selection • Modular approach • Hybrid approach • Information system • Experiences with the advisory system • Conclusions NIMIA Crema, Italy

  4. Introduction to the problem • Task • to develop an advisory system for operation of a Linz-Donawitz steel converter • to propose component composition • to support the factory staff in supervising the steel-making process • A model of the process is required NIMIA Crema, Italy

  5. LD Converter modeling NIMIA Crema, Italy

  6. Linz-Donawitz converter Phases of steelmaking • 1. Filling of waste iron • 2. Filling of pig iron • 3. Blasting with pure oxygen • 4. Supplement additives • 5. Sampling for quality testing • 6. Tapping of steel and slag NIMIA Crema, Italy

  7. Linz-Donawitz converter Phases of steelmaking • 1. Filling of waste iron • 2. Filling of pig iron • 3. Blasting with pure oxygen • 4. Supplement additives • 5. Sampling for quality testing • 6. Tapping of steel and slag NIMIA Crema, Italy

  8. Linz-Donawitz converter Phases of steelmaking • 1. Filling of waste iron • 2. Filling of pig iron • 3. Blasting with pure oxygen • 4. Supplement additives • 5. Sampling for quality testing • 6. Tapping of steel and slag NIMIA Crema, Italy

  9. Linz-Donawitz converter Phases of steelmaking • 1. Filling of waste iron • 2. Filling of pig iron • 3. Blasting with pure oxygen • 4. Supplement additives • 5. Sampling for quality testing • 6. Tapping of steel and slag NIMIA Crema, Italy

  10. Linz-Donawitz converter Phases of steelmaking • 1. Filling of waste iron • 2. Filling of pig iron • 3. Blasting with pure oxygen • 4. Supplement additives • 5. Sampling for quality testing • 6. Tapping of steel and slag NIMIA Crema, Italy

  11. Linz-Donawitz converter Phases of steelmaking • 1. Filling of waste iron • 2. Filling of pig iron • 3. Blasting with pure oxygen • 4. Supplement additives • 5. Sampling for quality testing • 6. Tapping of steel and slag NIMIA Crema, Italy

  12. Main parameters of the process • Nonlinear input-output relation between many inputs and two outputs • input parameters (~50 different parameters) • certain features “measured” during the process • The main output parameters • temperature (1640-1700 CO -10 … +15 CO) • carbon content (0.03 - 0.70 % ) • More than 5000 records of data NIMIA Crema, Italy

  13. Modeling task • The difficulties of model building • High complexity nonlinear input-output relationship • No (or unsatisfactory) physical insight • Relatively few measurement data • There are unmeasurable parameters • Noisy, imprecise, unreliable data • Classical approach (heat balance, mass balance) gives no acceptable results NIMIA Crema, Italy

  14. Modeling approaches • Theoretical model - based on chemical, physical equations • Input - output behavioral model • Neural model - based on the measured process data • Rule based system - based on the experimental knowledge of the factory staff • Combined neural - rule based system NIMIA Crema, Italy

  15. oxygen temperature System + components (parameters) e S Neural - Model predicted temperature model output components (parameters) temperature Copy of Inverse Model Model measured + predicted temperature oxygen e S - The modeling task NIMIA Crema, Italy

  16. „Neural” solution • The steps of solving a practical problem NIMIA Crema, Italy

  17. Building neural models • Creating a reliable database • the problem of noisy data • the problem of missing data • the problem of uneven data distribution • Selecting a proper neural architecture • static network • dynamic network • regressor selection • Training and validating the model NIMIA Crema, Italy

  18. Creating a reliable database • Input components • measure of importance • physical insight • sensitivity analysis • principal components • Normalization • input normalization • output normalization • Missing data • artificially generated data • Noisy data • preprocessing, filtering NIMIA Crema, Italy

  19. Initial database New database Neural network training Sensitivity analysis Input parameter cancellation Input parameter of small effect on the output? yes no Building database • Selecting input components, dimension reduction NIMIA Crema, Italy

  20. Building database • Dimension reduction: mathematical methods • PCA • Non-linear PCA • ICA • Combined methods NIMIA Crema, Italy

  21. x 2 y 2 y 1 x 1 Data compression, PCA networks • Principal component analysis (Karhunen-Loeve transformation NIMIA Crema, Italy

  22. Oja network • Linear feed-forward network NIMIA Crema, Italy

  23. Oja network • Learning rule • Normalized Hebbian learning NIMIA Crema, Italy

  24. Oja subspace network • Multi-output extension NIMIA Crema, Italy

  25. GHA, Sanger network • Multi-output extension Oja rule + Gram-Schmidt orthogonalization NIMIA Crema, Italy

  26. x 2 x 1 y 1 Nonlinear data compression • Nonlinear principal components NIMIA Crema, Italy

  27. Independent component analysis • A method of finding a transformation where the transformed components are statistically independent • Applies higher order statistics • Based on the different definitions of statistical independence The typical task • Can be implemented using neural architecture NIMIA Crema, Italy

  28. Normalizing Data • Typical data distributions NIMIA Crema, Italy

  29. Normalization • Zero mean, unit standard deviation • Normalization into [0,1] • Decorrelation + normalization NIMIA Crema, Italy

  30. Whitened Original Normalization • Decorrelation + normalization = Whitening transformation NIMIA Crema, Italy

  31. Missing or few data • Filling in the missing values • Artificially generated data • using trends • using correlation • using realistic transformations NIMIA Crema, Italy

  32. Few data • Artificial data generation • using realistic transformations • using sensitivity values: data generation around various working points (a good example: ALVINN) NIMIA Crema, Italy

  33. Noisy data • EIV • input and output noise are taken into consideration • modified criterion function • SVM • e-insensitive criterion function • Inherent noise suppression • classical neural nets have noise suppression property (inherent regularization) • averaging (modular approach) NIMIA Crema, Italy

  34. [ i ] n p , k * * y x k k + System [ i ] [ i ] n n + m , y , k + m , x , k [ ] i [ i ] y x k k Errors in variables (EIV) • Handling of noisy data NIMIA Crema, Italy

  35. EIV • LS vs EIV criterion function • EIV training NIMIA Crema, Italy

  36. EIV • Example NIMIA Crema, Italy

  37. EIV • Example NIMIA Crema, Italy

  38. SVM • Why SVM? • „Classical” Neural Networks • (MLP) • -„Overfitting” Support Vector Machine (SVM) +Better generalization (upper bounds) +Selects the more important input samples +Handles noise +~Automatic structure and parameter selection • Model • Structure • Parameter Selection difficulties NIMIA Crema, Italy

  39. SVM • Special problem of SVM • selecting hyperparameters •  insensitive • RBF type SVM: , C • slow „training”, complex computations • SVM-Light • Smaller, reduced teaching set • difficulty of real-time adaptation NIMIA Crema, Italy

  40. Selecting the optimal parameters C=1, =0.05, σ=0.9 C=1, =0.05, σ=1.9 NIMIA Crema, Italy

  41. Selecting the optimal parameters Sigma NIMIA Crema, Italy

  42. Selecting the optimal parameters Mean square error Sigma NIMIA Crema, Italy

  43. 1.4 1.2 EIV-SVM comparison f(x)=sin(x)/x Training points 1 Support vectors Training result of the SVM 0.8 Training result with EIV Training result with MLP 0.6 0.4 0.2 0 -0.2 -0.4 -10 -8 -6 -4 -2 0 2 4 6 8 10 Comparison of SVM, EIV and NN NIMIA Crema, Italy

  44. Model selection • Static or dynamic • Dynamic model class • regressor selection • basis function selection • Size of the network • number of layers • number of hidden neurons • model order NIMIA Crema, Italy

  45. 12 18 11 16 10 14 9 8 12 7 10 6 8 5 4 6 1 2 3 4 5 6 7 8 9 0 5 10 15 20 Model selection • NARX model, NOE model • Lipschitz number, Lipschitz quotient NIMIA Crema, Italy

  46. Model selection • Lipschitz quotient general nonlinear input-output relation, f(.) continuous, smooth multivariable function bounded derivatives Lipschitz quotient Sensitivity analysis NIMIA Crema, Italy

  47. Model selection • Lipschitz number for optimal n NIMIA Crema, Italy

  48. Modular solution • Ensemble of networks • linear combination of networks • Mixture of experts • using the same paradigm (e.g. neural networks) • using different paradigms (e.g. neural networks + symbolic systems) • Hybrid solution • expert systems • neural networks • physical (mathematical) models NIMIA Crema, Italy

  49. Cooperative networks Ensemble of cooperating networks (classification/regression) • The motivation • Heuristic explanation • Different experts together can solve a problem better • Complementary knowledge • Mathematical justification • Accurate and diverse modules NIMIA Crema, Italy

  50. Ensemble of networks • Mathematical justification • Ensemble output • Ambiguity (diversity) • Individual error • Ensemble error • Constraint NIMIA Crema, Italy

More Related