Neuro-Fyzzy Methods for Modeling and Identification Part 2 : Examples

1. Neuro-Fyzzy Methods for Modeling and Identification Part 2 : Examples Presented by: Ali Maleki

2. Presentation Agenda • Introduction Tools for Fuzzy Identification and Modelling MATLAB – Fuzzy Logic Toolbox – ANFIS • Fuzzy Modeling and Identification Toolbox • Examples : • Example 1: Hair Dryer • Example 2: Static Function • Example 3: Neutralization Process

3. Introduction • Control Systems • Competion • Environment requirements • Energy and material costs • Demand for robust, fault-tolerant systems • Extra needs for Effective process modeling techniques • Conventional modeling? • Lack precise and formal knowledg about the system • Strongly nonlinear behavior, • High degree of uncertainty, • Time varying characteristics

4. Introduction (cont.) • Solution: Neuro-fuzzy modeling • A powerful tool which can facilitate the effective development of models by combining information from different source: • Empirical models • Heuristics Data • Neuro-fuzzy models • Describe systems by means of fuzzy if-then rules • Represented in a network structure • Apply algorithms from the area of Neural Networks

5. Introduction (cont.) Zero-order TS fuzzy model Typical membership function Input-output equation

6. Introduction (cont.) • System identification steps: • Structure identification • Parameter estimation • choice of the model’s structure determines the flexibility of the model in the approximation of (unknown) systems • model with a rich structure can approximate more complicated functions, but, will have worse generalization properties • Good generalization means that a model fitted to one data set will also perform well on another data set from the same process.

7. Tools for Fuzzy Modeling & Identification • Fuzzy Modelling and Identification Toolbox • Develped by R. Babuska • http://Lcewww.et.tudelft.nl/~Babuska • Installation • Version 3.03 , April 2001 • Fuzzy Logic Toolbox for MATLAB • www.Mathworks.com/products/fuzzylogic • Version 2.1.3 , June 2004

8. Fuzzy Modeling and Identification Toolbox

9. Fuzzy Modeling and Identification Toolbox (cont.)

10. Fuzzy Modeling and Identification Toolbox (cont.) FMSIM function: Simulate a MIMO input-output fuzzy model

11. Fuzzy Modeling and Identification Toolbox (cont.) Structure of FM (fmstruct function):

12. Fuzzy Modeling and Identification Toolbox (cont.) Structure of FM (fmstruct function):

13. Fuzzy Modeling and Identification Toolbox (cont.) plotmfs function: Plot membership functions rms function: Root mean square between two signals vaf function: Percentile variance accounted for (VAF) between two signals

14. MATLAB - Fuzzy Toolbox anfis function: Training routine for Sugeno-type FIS anfisedit function: Open the ANFIS Editor GUI genfis1 function: Generate an FIS structure from data without data clustering genfis2 function: Generate an FIS structure from data using subtractive clustering

15. MATLAB - Fuzzy Toolbox (cont.) ANFIS Editor GUI

16. Example 1 : Hair Dryer (Fuzzy Logic Toolbox) • Nonlinear dynamical system identification • With use of ANFIS function in the Fuzzy Logic Toolbox • Data set was obtained from a laboratory device called Feedback's Process Trainer PT 326, • L. Ljung, "System Identification, Theory for the User", Prentice-Hall, 1987 – Chapter 17 • The device's function is like a hair dryer: • Input u(k) : Voltage over the mesh of resistor wires • Output y(k) : Outlet air temperature

17. Example 1 : Hair Dryer(Fuzzy Logic Toolbox)(cont.) • Input u(k) : Binary random signal shifting between 3.41 and 6.41 V • Output y(k) : Outlet air temperature • Sampling Time : 0.08 sec

18. Example 1 : Hair Dryer(Fuzzy Logic Toolbox)(cont.) • Linear ARX model: • y(k)+a1*y(k-1)+...+am*y(k-m)=b1*u(k-d)+...+bn*u(k-d-n+1) • ai and bj are linear parameters to be determined by least-squares methods • This structure is exactly specified by three integers [m, n, d] • Remind: • System Identification : structure selection + parameter estimation

19. Example 1 : Hair Dryer(Fuzzy Logic Toolbox)(cont.) • Remove the means from the data • The data set was divided into a training set and a checking set • Training data set : (k = 1 to 300) Checking data set : (k = 301 to 600) • An exhaustive search was performed to find the best combination [m, n, d] • each of the integer is allowed to changed from 1 to 10 independently • Run through all different models: • V = arxstruc(ze, zv, struc(1:10, 1:10, 1:10)); • Find the best model: nn = selstruc(V, 0); • The best ARX model : [m, n, d] = [5, 10, 2]

20. Example 1 : Hair Dryer(Fuzzy Logic Toolbox)(cont.) • Training RMSE = 0.1122 Checking RMSE = 0.0749

21. Example 1 : Hair Dryer(Fuzzy Logic Toolbox)(cont.) • Advantage of ARX model: • Rapid model structure selection • Rapid parameter identification • The performance in the above plots appear to be satisfactory. • If a better performance level is desired, we might want to resort to a nonlinear model. • Neuro-fuzzy modeling approach, ANFIS

22. Example 1 : Hair Dryer(Fuzzy Logic Toolbox)(cont.) • Use ANFIS for system identification: • First step: input selection • To determine which variables should be the input arguments to an ANFIS model. • For simplicity, we suppose that there are 10 input candidates • y(k-1), y(k-2), y(k-3), y(k-4), u(k-1), u(k-2), u(k-3), u(k-4), u(k-5), u(k-6) • Two approaches for input selection: • Sequential search • Exhaustive search

23. Example 1 : Hair Dryer(Fuzzy Logic Toolbox)(cont.) • Sequential searchfor input selection:can be done by the function seqsrch • 10 + 9 + 8 • =27 • Selected inputs : y(k-1), u(k-3), and u(k-4) • Training RMSE = 0.0609 Checking RMSE = 0.0604.

24. Example 1 : Hair Dryer(Fuzzy Logic Toolbox)(cont.) • Exhaustive search on all possible combinations of the input candidates • Can be done by function exhsrch • We want to selects 3 inputs from 10 candidates, therefore, the total number of ANFIS models is • Fortunately, for dynamical system identification, we do know that the inputs should not come from either of the following two sets of input candidates exclusively: • Y = {y(k-1), y(k-2), y(k-3), y(k-4)} U = {u(k-1), u(k-2), u(k-3), u(k-4), u(k-5), u(k-6)} • A reasonable guess: two inputs from Y and one from U

25. Example 1 : Hair Dryer(Fuzzy Logic Toolbox)(cont.) • Exhaustive search • Selected inputs : y(k-1), y(k-2) , u(k-3) • Training RMSE = 0.0474 Checking RMSE = 0.0485

26. Example 1 : Hair Dryer(Fuzzy Logic Toolbox)(cont.) • ARX model

27. Example 1 : Hair Dryer(Fuzzy Logic Toolbox)(cont.) • ANFIS model

28. Example 1 : Hair Dryer(Fuzzy Logic Toolbox)(cont.) • Comparision • If fast modeling is the goal, then ARX is the right choice, • If precision is the utmost concern, then we can go for ANFIS that is designed for nonlinear modeling and higher precision

29. Example 2 - Static Function ANFIS model with linear consequent function Number of rules: five rules Construction of initial model: Gustafson-Kessel algorithm Fit of the function with initial model – local models - membership functions

30. Example 2 - Static Function (cont.) this initial model can easily be interpreted in terms of the local behavior It is reasonably accurate (RMS= 0.0258) ANFIS method, 100 learning epochs anfis function of the MATLABFuzzy Logic Toolbox Fit of the function with fine-tuned model, local models, membership functions

31. Example 2 - Static Function (cont.) RMS error is about 23 times better than the initial model Initial modelFine-tuned model RMS error = 0.0258 RMS error = 0.0011

32. Example 2 - Static Function (cont.) after learning, the local models are much further from the true local description of the function Initial modelFine-tuned model Fine-tuned model are thus less accurate in describing the system locally

33. Example 3 – pH Neutralization Process Influent streams Acid Buffer Base Neutralization tank Effluent stream Acid flowrate = cteBuffer flowrate = cteBase stream flowrate Neutralization tank pH in the tank

34. Example 3 - pH Neutralization Process (cont.) • Identification and validation data sets: • Simulating the model by Hall and Seborg for random change of the influent base stream flow rate • N = 499 samples with the sampling time of 15 s. • The process is approximated as a first–order discrete-time NARX model

35. Example 3 - pH Neutralization Process (cont.) Membership functions Befor training After training

36. Example 3 - pH Neutralization Process (cont.) Rules: Initial Rules: Fine Tuned Rules: after 1000 epochs of hybrid learning using the ANFIS function of the MATLAB Fuzzy Logic Toolbox

37. Example 3 - pH Neutralization Process (cont.) • Overtraining Problem: • Comparision of RMS ERROR befor and after training • Prediction befor and after training

38. References [1] Robert Babuska, “Neuro-Fuzzy Methods for Modeling and Identification”, Recent Advances in Intelligent Paradigms and Application, Springer-Verilag, 2002 [2] Robert Babuska, “Fuzzy Modeling and Identification Toolbox User’s Guide - For Use with MATLAB”, 1998. [3] MathWorks Inc., “Fuzzy Logic Toolbox – Users Guide – Version 2”, 2004. [4] L. Ljung, “System Identification, Theory for the User”, Prentice-Hall, 1987.

39. THANK YOU VERY MUCH For your Attention Presented by: Ali Maleki

40. Introduction - appendix • Types of fuzzy models: (depending on the structure of if-then rules) • Mamdani Model • IF D1 is low and D2 is highTHEN D is medium • Takagi-Sugeno Model • IF D1 is low and D2 is highTHEN D=k (zero-order) • IF D1 is low and D2 is highTHEN D=0.7D1+0.2D2+0.1 (first-order)