1 / 39

Lecture 11

Lecture 11. Hybrid intelligent systems:. Neural expert systems and neuro-fuzzy systems. Introduction Neural expert systems Neuro -fuzzy systems. Introduction.

mairi
Download Presentation

Lecture 11

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Lecture 11 Hybrid intelligent systems: Neural expert systems and neuro-fuzzy systems • Introduction • Neural expert systems • Neuro-fuzzy systems Computer Intelligence and Soft Computing

  2. Introduction • A hybrid intelligent system is one that combinesat least two intelligent technologies. For example, combining a neural network with a fuzzy systemresults in a hybrid neuro-fuzzy system. • The combination of probabilistic reasoning, fuzzylogic, neural networks and evolutionarycomputation forms the core of soft computing, anemerging approach to building hybrid intelligentsystems capable of reasoning and learning in anuncertain and imprecise environment. Computer Intelligence and Soft Computing

  3. Although words are less precise than numbers, precision carries a high cost. We use wordswhen there is a tolerance for imprecision. Softcomputing exploits the tolerance for uncertaintyand imprecision to achieve greater tractabilityand robustness, and lower the cost of solutions. • We also use words when the available data isnot precise enough to use numbers. This isoften the case with complex problems, andwhile “hard” computing fails to produce anysolution, soft computing is still capable offinding good solutions. Computer Intelligence and Soft Computing

  4. Lotfi Zadehis reputed to have said that a goodhybrid would be “British Police, GermanMechanics, French Cuisine, Swiss Banking andItalian Love”. But “British Cuisine, GermanPolice, French Mechanics, Italian Banking andSwiss Love” would be a bad one. Likewise, ahybrid intelligent system can be good or bad – itdepends on which components constitute thehybrid. So our goal is to select the rightcomponents for building a good hybrid system. Computer Intelligence and Soft Computing

  5. Comparison of Expert Systems, FuzzySystems, Neural Networks and Genetic Algorithms Computer Intelligence and Soft Computing

  6. Neural expert systems • Expert systems rely on logical inferences anddecision trees and focus on modelling humanreasoning. Neural networks rely on parallel dataprocessing and focus on modelling a human brain. • Expert systems treat the brain as a black-box. Neural networks look at its structure and functions, particularly at its ability to learn. • Knowledge in a rule-based expert system isrepresented by IF-THEN production rules. Knowledge in neural networks is stored assynaptic weights between neurons. Computer Intelligence and Soft Computing

  7. In expert systems, knowledge can be divided intoindividual rules and the user can see andunderstand the piece of knowledge applied by thesystem. • In neural networks, one cannot select a singlesynaptic weight as a discrete piece of knowledge. Here knowledge is embedded in the entirenetwork; it cannot be broken into individualpieces, and any change of a synaptic weight maylead to unpredictable results. A neural network is,in fact, a black-boxfor its user. Computer Intelligence and Soft Computing

  8. Can we combine advantages of expert systemsand neural networks to create a more powerfuland effective expert system? A hybrid system that combines a neural networkand a rule-based expert system is called aneuralexpert system(or a connectionist expert system). Computer Intelligence and Soft Computing

  9. Basic structure of a neural expert system Computer Intelligence and Soft Computing

  10. The heart of a neural expert system is theinference engine. It controls the informationflow in the system and initiates inference overthe neural knowledge base. A neural inferenceengine also ensures approximate reasoning. Computer Intelligence and Soft Computing

  11. Approximate reasoning • In a rule-based expert system, the inference enginecompares the condition part of each rule with datagiven in the database. When the IF part of the rulematches the data in the database, the rule is fired andits THEN part is executed. The precise matching isrequired (inference engine cannot cope with noisy orincomplete data). • Neural expert systems use a trained neural network inplace of the knowledge base. The input data does nothave to precisely match the data that was used innetwork training. This ability is called approximatereasoning. Computer Intelligence and Soft Computing

  12. Rule extraction • Neurons in the network are connected by links,each of which has a numerical weight attached to it. • The weights in a trained neural network determinethe strength or importance of the associated neuroninputs. Computer Intelligence and Soft Computing

  13. The neural knowledge base Computer Intelligence and Soft Computing

  14. If we set each input of the input layer to either +1(true), -1 (false), or 0 (unknown), we can give asemantic interpretation for the activation ofany outputneuron. For example, if the object has Wings (+1), Beak (+1) and Feathers (+1), but does not haveEngine (-1), then we can conclude that this object isBird (+1): Computer Intelligence and Soft Computing

  15. We can similarly conclude that this object is notPlane: and not Glider: Computer Intelligence and Soft Computing

  16. By attaching a corresponding question to each inputneuron, we can enable the system to prompt the userfor initial values of the input variables: Neuron: WingsQuestion: Does the object have wings? Neuron: TailQuestion: Does the object have a tail? Neuron: BeakQuestion: Does the object have a beak? Neuron: FeathersQuestion: Does the object have feathers? Neuron: EngineQuestion: Does the object have an engine? Computer Intelligence and Soft Computing

  17. An inference can be made if the known netweighted input to a neuron is greater than thesum of the absolute values of the weights ofthe unknown inputs. where i Î known, j Ï known and nis the numberof neuron inputs. Computer Intelligence and Soft Computing

  18. Example: Enter initial value for the input Feathers:> +1 KNOWN = 1.2.8 = 2.8UNKNOWN = ½-0.8½ + ½-0.2½ + ½2.2½ + ½-1.1½ = 4.3KNOWN < UNKNOWN Enter initial value for the input Beak: >+1 KNOWN = 1.2.8 + 1.2.2 = 5.0UNKNOWN = ½-0.8½ + ½-0.2½ + ½-1.1½ = 2.1KNOWN > UNKNOWN CONCLUDE: Bird is TRUE Computer Intelligence and Soft Computing

  19. An example of a multi-layer knowledge base Computer Intelligence and Soft Computing

  20. Neuro-fuzzy systems • Fuzzy logic and neural networks are naturalcomplementary tools in building intelligentsystems. While neural networks are low-levelcomputational structures that perform well whendealing with raw data, fuzzy logic deals withreasoning on a higher level, using linguisticinformation acquired from domain experts.However, fuzzy systems lack the ability to learnand cannot adjust themselves to a newenvironment. On the other hand, although neuralnetworks can learn, they are opaque to the user. Computer Intelligence and Soft Computing

  21. Integrated neuro-fuzzy systems can combine theparallel computation and learning abilities ofneural networks with the human-like knowledgerepresentation and explanation abilities of fuzzysystems. As a result, neural networks becomemore transparent, while fuzzy systems becomecapable of learning. Computer Intelligence and Soft Computing

  22. A neuro-fuzzy system is a neural network whichis functionallyequivalent to a fuzzy inferencemodel. It can be trained to develop IF-THENfuzzy rules and determine membership functionsfor input and output variables of the system. Expert knowledge can be incorporated into thestructure of theneuro-fuzzy system. At the sametime, the connectionist structure avoids fuzzyinference, which entails a substantialcomputational burden. Computer Intelligence and Soft Computing

  23. The structure of a neuro-fuzzy system is similarto a multi-layer neural network. In general, aneuro-fuzzy system has input and output layers, and three hidden layers that representmembership functions and fuzzy rules. Computer Intelligence and Soft Computing

  24. Neuro-fuzzy system Computer Intelligence and Soft Computing

  25. Each layer in the neuro-fuzzy system is associatedwith a particular step in the fuzzy inference process. Layer 1is the input layer. Each neuron in this layertransmits external crisp signals directly to the nextlayer. That is, Layer 2is the fuzzification layer. Neurons in thislayer represent fuzzy sets used in the antecedentsof fuzzy rules. A fuzzification neuron receives acrisp input and determines the degree to whichthis input belongs to the neuron’sfuzzy set. Computer Intelligence and Soft Computing

  26. The activation function of a membershipneuronisset to the function that specifies the neuron’s fuzzyset. We use triangular sets, and therefore, theactivation functions for the neuronsinLayer 2 areset to the triangular membership functions. Atriangular membership function can be specified bytwo parameters {a, b} as follows: Computer Intelligence and Soft Computing

  27. Triangular activation functions Computer Intelligence and Soft Computing

  28. Layer 3is the fuzzy rule layer. Eachneuron in thislayer corresponds to a single fuzzy rule. A fuzzyruleneuron receives inputs from thefuzzificationneurons that represent fuzzy sets in the ruleantecedents. For instance, neuron R1, whichcorresponds toRule 1, receives inputs fromneurons A1 and B1. In a neuro-fuzzy system, intersection can beimplemented by the product operator. Thus, theoutput of neuron i inLayer 3 is obtained as: Computer Intelligence and Soft Computing

  29. Layer 4is the output membership layer. Neuronsin this layer represent fuzzy sets used in theconsequent of fuzzy rules. An output membership neuron combines all itsinputs by using the fuzzy operation union. This operation can be implemented by theprobabilistic OR. That is, The value of mC1 represents the integrated firingstrength of fuzzy rule neurons R3 and R6 Computer Intelligence and Soft Computing

  30. Layer 5is the defuzzification layer. Each neuronin this layer represents a single output of theneuro-fuzzy system. It takes the output fuzzy setsclipped by the respective integrated firingstrengths and combines them into a single fuzzyset. Neuro-fuzzy systems can apply standarddefuzzification methods, including the centroidtechnique. We will use the sum-product compositionmethod. Computer Intelligence and Soft Computing

  31. The sum-product composition calculates the crispoutput as the weighted average of the centroids ofall output membership functions. For example, theweighted average of the centroids of the clippedfuzzy sets C1 and C2 is calculated as, Computer Intelligence and Soft Computing

  32. How does a neuro-fuzzy system learn? A neuro-fuzzy system is essentially a multi-layerneural network, and thus it can apply standardlearning algorithms developed for neural networks, including the back-propagation algorithm. Computer Intelligence and Soft Computing

  33. When a training input-output example is presentedto the system, the back-propagation algorithmcomputes the system output and compares it withthe desired output of the training example. Theerror is propagated backwards through the networkfrom the output layer to the input layer. Theneuron activation functions are modified as theerror is propagated. To determine the necessarymodifications, the back-propagation algorithmdifferentiates the activation functions of theneurons. • Let us demonstrate how aneuro-fuzzy systemworks on a simple example. Computer Intelligence and Soft Computing

  34. Training patterns Computer Intelligence and Soft Computing

  35. The data set is used for training the five-rule neuro-fuzzy system shown below. Five-rule neuro-fuzzy system x1 Computer Intelligence and Soft Computing

  36. Suppose that fuzzy IF-THEN rules incorporatedinto the system structure are supplied by adomain expert. Prior or existing knowledge candramatically expedite the system training. • Besides, if the quality of training data is poor, the expert knowledge may be the only way to come to a solution at all. However, experts do occasionally make mistakes, and thus some rules used in a neuro-fuzzy system may be false orredundant. Therefore, a neuro-fuzzy systemshould also be capable of identifying bad rules. Computer Intelligence and Soft Computing

  37. Given input and output linguistic values, a neuro-fuzzy system can automatically generate a completeset of fuzzy IF-THEN rules. • Let us create the system for the XOR example. This system consists of 22´ 2 = 8 rules. Becauseexpert knowledge is not embodied inthesystemthis time, we set allinitial weights between Layer 3and Layer 4 to 0.5. • After training we can eliminate all rules whosecertainty factors are less than some sufficientlysmall number, say 0.1. As a result, we obtain thesame set of four fuzzy IF-THEN rules thatrepresents the XOR operation. Computer Intelligence and Soft Computing

  38. Eight-rule neuro-fuzzy system Computer Intelligence and Soft Computing

  39. Neuro-fuzzy systems: summary • The combination of fuzzy logic and neuralnetworks constitutes a powerful means fordesigning intelligent systems. • Domain knowledge can be put into aneuro-fuzzysystem byhuman experts in the form of linguisticvariables and fuzzy rules. • When a representative set of examples is available, a neuro-fuzzysystem can automatically transformit into a robust set of fuzzy IF-THEN rules, andthereby reduce our dependence on expertknowledge when building intelligent systems. Computer Intelligence and Soft Computing

More Related