1 / 63

Lecture 11

Lecture 11. Hybrid intelligent systems:. Neural expert systems and neuro-fuzzy systems. Introduction Neural expert systems Neuro-fuzzy systems ANFIS: Adaptive Neuro-Fuzzy Inference System Summary. Introduction.

zuzana
Download Presentation

Lecture 11

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Lecture 11 Hybrid intelligent systems: Neural expert systems and neuro-fuzzy systems • Introduction • Neural expert systems • Neuro-fuzzy systems • ANFIS: Adaptive Neuro-Fuzzy InferenceSystem • Summary

  2. Introduction • A hybrid intelligent system is one that combinesat least two intelligent technologies. For example, combining a neural network with a fuzzy systemresults in a hybrid neuro-fuzzy system. • The combination of probabilistic reasoning, fuzzylogic, neural networks and evolutionarycomputation forms the core of soft computing, anemerging approach to building hybrid intelligentsystems capable of reasoning and learning in anuncertain and imprecise environment.

  3. Although words are less precise than numbers, precision carries a high cost. We use wordswhen there is a tolerance for imprecision. Softcomputing exploits the tolerance for uncertaintyand imprecision to achieve greater tractabilityand robustness, and lower the cost of solutions. • We also use words when the available data isnot precise enough to use numbers. This isoften the case with complex problems, andwhile “hard” computing fails to produce anysolution, soft computing is still capable offinding good solutions.

  4. Lotfi Zadehis reputed to have said that a goodhybrid would be “British Police, GermanMechanics, French Cuisine, Swiss Banking andItalian Love”. But “British Cuisine, GermanPolice, French Mechanics, Italian Banking andSwiss Love” would be a bad one. Likewise, ahybrid intelligent system can be good or bad – itdepends on which components constitute thehybrid. So our goal is to select the rightcomponents for building a good hybrid system.

  5. Comparison of Expert Systems, FuzzySystems, Neural Networks and Genetic Algorithms

  6. Neural expert systems • Expert systems rely on logical inferences anddecision trees and focus on modelling humanreasoning. Neural networks rely on parallel dataprocessing and focus on modelling a human brain. • Expert systems treat the brain as a black-box. Neural networks look at its structure and functions, particularly at its ability to learn. • Knowledge in a rule-based expert system isrepresented by IF-THEN production rules. Knowledge in neural networks is stored assynaptic weights between neurons.

  7. In expert systems, knowledge can be divided intoindividual rules and the user can see andunderstand the piece of knowledge applied by thesystem. • In neural networks, one cannot select a singlesynaptic weight as a discrete piece of knowledge. Here knowledge is embedded in the entirenetwork; it cannot be broken into individualpieces, and any change of a synaptic weight maylead to unpredictable results. A neural network is,in fact, a black-boxfor its user.

  8. Can we combine advantages of expert systemsand neural networks to create a more powerfuland effective expert system? A hybrid system that combines a neural networkand a rule-based expert system is called aneuralexpert system(or a connectionist expert system).

  9. Basic structure of a neural expert system

  10. The heart of a neural expert system is theinference engine. It controls the informationflow in the system and initiates inference overthe neural knowledge base. A neural inferenceengine also ensures approximate reasoning.

  11. Approximate reasoning • In a rule-based expert system, the inference enginecompares the condition part of each rule with datagiven in the database. When the IF part of the rulematches the data in the database, the rule is fired andits THEN part is executed. The precise matching isrequired (inference engine cannot cope with noisy orincomplete data). • Neural expert systems use a trained neural network inplace of the knowledge base. The input data does nothave to precisely match the data that was used innetwork training. This ability is called approximatereasoning.

  12. Rule extraction • Neurons in the network are connected by links,each of which has a numerical weight attached to it. • The weights in a trained neural network determinethe strength or importance of the associated neuroninputs.

  13. The neural knowledge base

  14. If we set each input of the input layer to either +1(true), -1 (false), or 0 (unknown), we can give asemantic interpretation for the activation ofany outputneuron. For example, if the object has Wings (+1), Beak (+1) and Feathers (+1), but does not haveEngine (-1), then we can conclude that this object isBird (+1):

  15. We can similarly conclude that this object is notPlane: and not Glider:

  16. By attaching a corresponding question to each inputneuron, we can enable the system to prompt the userfor initial values of the input variables: Neuron: WingsQuestion: Does the object have wings? Neuron: TailQuestion: Does the object have a tail? Neuron: BeakQuestion: Does the object have a beak? Neuron: FeathersQuestion: Does the object have feathers? Neuron: EngineQuestion: Does the object have an engine?

  17. An inference can be made if the known netweighted input to a neuron is greater than thesum of the absolute values of the weights ofthe unknown inputs. where i Î known, j Ï known and nis the numberof neuron inputs.

  18. Example: Enter initial value for the input Feathers:> +1 KNOWN = 1.2.8 = 2.8UNKNOWN = ½-0.8½ + ½-0.2½ + ½2.2½ + ½-1.1½ = 4.3KNOWN < UNKNOWN Enter initial value for the input Beak: >+1 KNOWN = 1.2.8 + 1.2.2 = 5.0UNKNOWN = ½-0.8½ + ½-0.2½ + ½-1.1½ = 2.1KNOWN > UNKNOWN CONCLUDE: Bird is TRUE

  19. An example of a multi-layer knowledge base

  20. Neuro-fuzzy systems • Fuzzy logic and neural networks are naturalcomplementary tools in building intelligentsystems. While neural networks are low-levelcomputational structures that perform well whendealing with raw data, fuzzy logic deals withreasoning on a higher level, using linguisticinformation acquired from domain experts.However, fuzzy systems lack the ability to learnand cannot adjust themselves to a newenvironment. On the other hand, although neuralnetworks can learn, they are opaque to the user.

  21. Integrated neuro-fuzzy systems can combine theparallel computation and learning abilities ofneural networks with the human-like knowledgerepresentation and explanation abilities of fuzzysystems. As a result, neural networks becomemore transparent, while fuzzy systems becomecapable of learning.

  22. A neuro-fuzzy system is a neural network whichis functionallyequivalent to a fuzzy inferencemodel. It can be trained to develop IF-THENfuzzy rules and determine membership functionsfor input and output variables of the system. Expert knowledge can be incorporated into thestructure of theneuro-fuzzy system. At the sametime, the connectionist structure avoids fuzzyinference, which entails a substantialcomputational burden.

  23. The structure of a neuro-fuzzy system is similarto a multi-layer neural network. In general, aneuro-fuzzy system has input and output layers, and three hidden layers that representmembership functions and fuzzy rules.

  24. Neuro-fuzzy system

  25. Each layer in the neuro-fuzzy system is associatedwith a particular step in the fuzzy inference process. Layer 1is the input layer. Each neuron in this layertransmits external crisp signals directly to the nextlayer. That is, Layer 2is the fuzzification layer. Neurons in thislayer represent fuzzy sets used in the antecedentsof fuzzy rules. A fuzzification neuron receives acrisp input and determines the degree to whichthis input belongs to the neuron’sfuzzy set.

  26. The activation function of a membershipneuronisset to the function that specifies the neuron’s fuzzyset. We use triangular sets, and therefore, theactivation functions for the neuronsinLayer 2 areset to the triangular membership functions. Atriangular membership function can be specified bytwo parameters {a, b} as follows:

  27. Triangular activation functions

  28. Layer 3is the fuzzy rule layer. Eachneuron in thislayer corresponds to a single fuzzy rule. A fuzzyruleneuron receives inputs from thefuzzificationneurons that represent fuzzy sets in the ruleantecedents. For instance, neuron R1, whichcorresponds toRule 1, receives inputs fromneurons A1 and B1. In a neuro-fuzzy system, intersection can beimplemented by the product operator. Thus, theoutput of neuron i inLayer 3 is obtained as:

  29. Layer 4is the output membership layer. Neuronsin this layer represent fuzzy sets used in theconsequent of fuzzy rules. An output membership neuron combines all itsinputs by using the fuzzy operation union. This operation can be implemented by theprobabilistic OR. That is, The value of mC1 represents the integrated firingstrength of fuzzy rule neurons R3 and R6

  30. Layer 5is the defuzzification layer. Each neuronin this layer represents a single output of theneuro-fuzzy system. It takes the output fuzzy setsclipped by the respective integrated firingstrengths and combines them into a single fuzzyset. Neuro-fuzzy systems can apply standarddefuzzification methods, including the centroidtechnique. We will use the sum-product compositionmethod.

  31. The sum-product composition calculates the crispoutput as the weighted average of the centroids ofall output membership functions. For example, theweighted average of the centroids of the clippedfuzzy sets C1 and C2 is calculated as,

  32. How does a neuro-fuzzy system learn? A neuro-fuzzy system is essentially a multi-layerneural network, and thus it can apply standardlearning algorithms developed for neural networks, including the back-propagation algorithm.

  33. When a training input-output example is presentedto the system, the back-propagation algorithmcomputes the system output and compares it withthe desired output of the training example. Theerror is propagated backwards through the networkfrom the output layer to the input layer. Theneuron activation functions are modified as theerror is propagated. To determine the necessarymodifications, the back-propagation algorithmdifferentiates the activation functions of theneurons. • Let us demonstrate how aneuro-fuzzy systemworks on a simple example.

  34. Training patterns

  35. The data set is used for training the five-rule neuro-fuzzy system shown below. Five-rule neuro-fuzzy system

  36. Suppose that fuzzy IF-THEN rules incorporatedinto the system structure are supplied by adomain expert. Prior or existing knowledge candramatically expedite the system training. • Besides, if the quality of training data is poor, the expert knowledge may be the only way to come to a solution at all. However, experts do occasionally make mistakes, and thus some rules used in a neuro-fuzzy system may be false orredundant. Therefore, a neuro-fuzzy systemshould also be capable of identifying bad rules.

  37. Given input and output linguistic values, a neuro-fuzzy system can automatically generate a completeset of fuzzy IF-THEN rules. • Let us create the system for the XOR example. This system consists of 22´ 2 = 8 rules. Becauseexpert knowledge is not embodied inthesystemthis time, we set allinitial weights between Layer 3and Layer 4 to 0.5. • After training we can eliminate all rules whosecertainty factors are less than some sufficientlysmall number, say 0.1. As a result, we obtain thesame set of four fuzzy IF-THEN rules thatrepresents the XOR operation.

  38. Eight-rule neuro-fuzzy system

  39. Neuro-fuzzy systems: summary • The combination of fuzzy logic and neuralnetworks constitutes a powerful means fordesigning intelligent systems. • Domain knowledge can be put into aneuro-fuzzysystem byhuman experts in the form of linguisticvariables and fuzzy rules. • When a representative set of examples is available, a neuro-fuzzysystem can automatically transformit into a robust set of fuzzy IF-THEN rules, andthereby reduce our dependence on expertknowledge when building intelligent systems.

  40. ANFIS:Adaptive Neuro-Fuzzy Inference System The Sugenofuzzy model was proposed for generatingfuzzy rules from a given input-output data set. A typicalSugeno fuzzy rule is expressed in the following form:IF x1is A1AND x2 is A2. . . . . AND xmis AmTHEN y = f (x1, x2, . . . , xm) where x1, x2, . . . , xm are inputvariables; A1, A2, . . . , Amare fuzzy sets.

  41. When y is a constant, we obtain a zero-orderSugeno fuzzy model in which the consequent ofa rule is specified by a singleton. • When y is a first-order polynomial, i.e. y = k0 + k1x1 + k2x2 + . . . + km xm we obtain a first-order Sugeno fuzzy model.

  42. Adaptive Neuro-Fuzzy Inference System

  43. Layer 1is the input layer. Neurons in this layersimply pass external crisp signals to Layer 2. Layer 2is the fuzzification layer. Neurons in thislayer perform fuzzification. In Jang’s model, fuzzification neurons have a bell activationfunction.

  44. Layer 3is the rule layer. Each neuron in this layercorresponds to a single Sugeno-type fuzzy rule. A rule neuron receives inputs from the respectivefuzzification neurons and calculates the firingstrength of the rule it represents. In an ANFIS, the conjunction of the rule antecedents isevaluated by the operator product. Thus, theoutput of neuron i in Layer 3 is obtained as, where the value of m1 represents the firingstrength, or the truth value, of Rule 1.

  45. Layer 4is the normalisation layer. Eachneuroninthis layer receives inputs from all neurons in therule layer, and calculates the normalised firingstrength of a given rule. The normalised firing strength is the ratio of thefiring strength of a given rule to the sum of firingstrengths of all rules. It represents the contributionof a given rule to the final result. Thus, the outputof neuron iin Layer 4 is determined as,

  46. Layer 5is the defuzzification layer. Each neuronin this layer is connected to the respectivenormalisation neuron, and also receives initialinputs, x1and x2. A defuzzification neuroncalculates the weighted consequent value of agiven rule as, where is the input and is the output ofdefuzzification neuron i in Layer 5, and ki0, ki1and ki2 is a set of consequent parameters of rule i.

  47. Layer 6is represented by a singlesummationneuron. This neuron calculates the sum ofoutputs of all defuzzification neurons andproduces the overall ANFIS output, y,

  48. Can an ANFIS deal with problems where wedo not have any prior knowledge of the ruleconsequent parameters? It is not necessary to have any prior knowledge ofrule consequent parameters. An ANFIS learnsthese parameters and tunes membership functions.

  49. Learning in the ANFIS model • An ANFIS uses a hybrid learning algorithm thatcombines the least-squares estimator and thegradient descent method. • In the ANFIS training algorithm, each epoch iscomposed from a forward pass and a backwardpass. In theforward pass, a training set of inputpatterns (an input vector) is presented to theANFIS, neuron outputs are calculated on the layer- by-layer basis, and rule consequent parameters areidentified.

  50. The rule consequent parameters are identified bythe least-squares estimator. In the Sugeno-stylefuzzy inference, an output, y, is a linear function. Thus, given the values of the membershipparameters and a training set of P input-outputpatterns, we can form P linear equations in termsof the consequent parameters as:

More Related