1 / 229

Soft Computing

Soft Computing. Outline. Intelligent Systems/Historical Perspective Foundation of Soft Computing Evolution of Soft Computing Neural Network / Neuro Computing Fuzzy Logic / Fuzzy Computing Genetic Algorithm / Evolutionary Computing Hybrid Systems BISC Decision Support System Demo

hafwen
Download Presentation

Soft Computing

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Soft Computing

  2. Outline • Intelligent Systems/Historical Perspective • Foundation of Soft Computing • Evolution of Soft Computing • Neural Network / Neuro Computing • Fuzzy Logic / Fuzzy Computing • Genetic Algorithm / Evolutionary Computing • Hybrid Systems • BISC Decision Support System • Demo • Conclusions

  3. Computation? • Traditional Sense: Manipulation of Numbers • Human: Uses Word for Computation and Reasoning • Conclusions <= Word <== Natural Language

  4. Intelligent System? The role model for intelligent system is Human Mind. • Dreyfus: • Minds do not use a theory about the everyday world • Know-how vs know that • Winograd • Intelligent systems act, don't think

  5. Artificial Intelligence • Knowledge Representation • Predicates • Production rules • Semantic networks • Frames • Inference Engine • Learning • Common Sense & Heuristics • Uncertainty

  6. Artificial Intelligence • Applications • Expert tasks • the algorithm does not exist • A medical encyclopedia is not equivalent to a physician • Heuristics • There is an algorithm but it is “useless” • Uncertainty • The algorithm is not possible • Complex problems • The algorithm is too complicated • Technologies • Expert systems • Natural language processing • Symbolic processing • Knowledge engineering

  7. COMMON SENSE Deduction is a method of exact inference (classical logic) All Greeks are humans and Socrates is a Greek, therefore Socrates is a human Inductioninfers generalizations from a set of events (science) Water boils at 100 degrees Abduction infers plausible causes of an effect (medicine) You have the symptoms of a flue

  8. COMMON SENSE Uncertainty “Maybe I will go shopping” Probability Probability measures "how often" an event occurs Principle of incompatibility (Pierre Duhem) The certainty that a proposition is true decreases with any increase of its precision The power of a vague assertion rests in its being vague (“I am not tall”) A very precise assertion is almost never certain (“I am 1.71cm tall)

  9. COMMON SENSE The Frame Problem Classical logic deducts all that is possible from all that is available In the real world the amount of information that is available is infinite It is not possible to represent what does “not” change in the universe as a result of an action Infinite things change, because one can go into greater and greater detail of description The number of preconditions to the execution of any action is also infinite, as the number of things that can go wrong is infinite

  10. COMMON SENSE Classical Logic is inadequate for ordinary life Intuitionism Non- Monotonic Logic Second thoughts Plausible reasoning Quick, efficient response to problems when an exact solution is not necessary Heuristics Rules of thumbs George Polya: “Heuretics" The World Of Objects The Measure Space Qualitative Reasoning

  11. COMMON SENSE Fuzzy Logic Not just zero and one, true and false Things can belong to more than one category, and they can even belong to opposite categories, and that they can belong to a category only partially The degree of “membership” can assume any value between zero and one

  12. Cost “As complexity rises, precise statements lose meaning, and meaningful statements lose precision.” (L.A. Zadeh) • Principle of incompatibility (Pierre Duhem) • The certainty that a proposition is true decreases with any increase of its precision • The power of a vague assertion rests in its being vague (“I am not tall”) • A very precise assertion is almost never certain (“I am 1.71cm tall) Uncertainty Precision

  13. TURING’s TEST ? ? Turing: A computer can be said to be intelligent if its answers are indistinguishable from the answers of a human being Computer

  14. A new age? 1929 HUBBLE’S EXPANDING UNIVERSE 1930 TELEVISION 1940 MODERN SYNTHESIS 1940 SEMIOTICS 1944 DNA 1945 ATOMIC BOMB 1946 COMPUTER 1948 TRANSISTOR 1949 HEBB'S LAW 1953 DOUBLE-HELIX 1955 ARTIFICIAL INTELLIGENCE 1958 LINGUISTICS 1961 SPACE TRAVEL 1967 QUARKS 1974 SUPERSTRING 1978 NEURAL DARWINISM 1982 PERSONAL COMPUTER 1988 SELF-ORGANIZING SYSTEMS • 1859 THEORY OF EVOLUTION • 1854 RIEMANN’S GEOMETRY • 1862 AUTOMOBILE • 1865 HEREDITARITY • 1870 THERMODYNAMICS • 1876 TELEPHONE • 1877 GRAMOPHONE • 1888 ELECTROMAGNETISM • 1894 CINEMA • 1900 PLANCK’S QUANTUM • 1903 AIRPLANE • 1905 EINSTEIN’S SPECIAL RELATIVITY • 1907 RADIO • 1916 EINSTEIN’S GENERAL RELATIVITY • 1920 POPULATION GENETICS • 1923 WAVE-PARTICLE DUALISM • 1926 QUANTUM MECHANICS

  15. Different Historical Paths • PHILOSOPHY (SINCE BEGINNING) • PSYCHOLOGY (SINCE BEGINNING) • MATH (SINCE FREGE) • BIOLOGY (SINCE DISCOVERY OF NEURONS) • COMPUTER SCIENCE (SINCE A.I., 1955) • LINGUISTICS (SINCE CHOMSKY, 1960’s) • PHYSICS (RECENTLY, 1980’s) Thinking About Thought: The Nature of Mind; Piero Scaruffi (Oct 8 - Dec 10, 2002)

  16. The Nature of MindThe Contribution of Information Science The mind as a symbol processor Formal study of human knowledge Knowledge processing Common-sense knowledge Neural Networks Thinking About Thought: The Nature of Mind; Piero Scaruffi (Oct 8 - Dec 10, 2002)

  17. The Nature of MindThe Contribution of Linguistics Competence over performance Pragmatics Metaphor Thinking About Thought: The Nature of Mind; Piero Scaruffi (Oct 8 - Dec 10, 2002)

  18. The Nature of MindThe Contribution of Psychology The mind as a processor of concepts Reconstructive memory Memory is learning and is reasoning. Fundamental unity of cognition Thinking About Thought: The Nature of Mind; Piero Scaruffi (Oct 8 - Dec 10, 2002)

  19. The Nature of Mind The Contribution of Neurophysiology The brain is an evolutionary system Mind shaped mainly by genes and experience Neural-level competition Connectionism Thinking About Thought: The Nature of Mind; Piero Scaruffi (Oct 8 - Dec 10, 2002)

  20. The Nature of Mind The Contribution of Physics Living beings create order from disorder Non-equilibrium thermodynamics Self-organizing systems The mind as a self-organizing system Theories of consciousness based on quantum & relativity physics Thinking About Thought: The Nature of Mind; Piero Scaruffi (Oct 8 - Dec 10, 2002)

  21. MACHINE INTELLIGENCE History “Make it idiot proof and someone will make a better idiot”

  22. KURT GOEDEL (1931) • A CONCEPT OF TRUTH CANNOT BE DEFINED WITHIN A FORMAL SYSTEM • ALFRED TARSKI (1935) • DEFINITION OF “TRUTH”: A STATEMENT IS TRUE IF IT CORRESPONDS TO REALITY (“CORRESPONDENCE THEORY OF TRUTH”) DAVID HILBERT (1928) MATHEMATICS = BLIND MANIPULATION OF SYMBOLS FORMAL SYSTEM = A SET OF AXIOMS AND A SET OF INFERENCE RULES PROPOSITIONS AND PREDICATES DEDUCTION = EXACT REASONING ALFRED TARSKI (1935) BUILD MODELS OF THE WORLD WHICH YIELD INTERPRETATIONS OF SENTENCES IN THAT WORLD TRUTH CAN ONLY BE RELATIVE TO SOMETHING “META-THEORY”

  23. ALAN TURING (1936) • COMPUTATION = THE FORMAL MANIPULATION OF SYMBOLS THROUGH THE APPLICATION OF FORMAL RULES • HILBERT’S PROGRAM REDUCED TO MANIPULATION OF SYMBOLS • LOGIC = SYMBOL PROCESSING • EACH PREDICATE IS DEFINED BY A FUNCTION, EACH FUNCTION IS DEFINED BY AN ALGORITHM NORBERT WIENER (1947) CYBERNETICS BRIDGE BETWEEN MACHINES AND NATURE, BETWEEN "ARTIFICIAL" SYSTEMS AND NATURAL SYSTEMS FEEDBACK, HOMEOSTASIS, MESSAGE, NOISE, INFORMATION PARADIGM SHIFT FROM THE WORLD OF CONTINUOUS LAWS TO THE WORLD OF ALGORITHMS, DIGITAL VS ANALOG WORLD

  24. CLAUDE SHANNON AND WARREN WEAVER (1949) • INFORMATION THEORY • ENTROPY = A MEASURE OF DISORDER = A MEASURE OF THE LACK OF INFORMATION • LEON BRILLOUIN'S NEGENTROPY PRINCIPLE OF INFORMATION ANDREI KOLMOGOROV (1960) ALGORITHMIC INFORMATION THEORY COMPLEXITY = QUANTITY OF INFORMATION CAPACITY OF THE HUMAN BRAIN = 10 TO THE 15TH POWER MAXIMUM AMOUNT OF INFORMATION STORED IN A HUMAN BEING = 10 TO THE 45TH ENTROPY OF A HUMAN BEING = 10 TO THE 23TH LOTFI A. ZADEH (1965) FUZZY SET “Stated informally, the essence of this principle is that as the complexity of a system increases, our ability to make precise and yet significant statements about its behavior diminishes until a threshold is reached beyond which precision and significance (or relevance) become almost mutually exclusive characteristics.”

  25. MACHINE INTELLIGENCE: History 1936: TURING MACHINE 1940: VON NEUMANN’S DISTINCTION BETWEEN DATA AND INSTRUCTIONS 1943: FIRST COMPUTER 1943 MCCULLOUCH & PITTS NEURON 1947: VON NEUMANN’s SELF-REPRODUCING AUTOMATA 1948: WIENER’S CYBERNETICS 1950: TURING’S TEST 1956: DARTMOUTH CONFERENCE ON ARTIFICIAL INTELLIGENCE 1957: NEWELL & SIMON’S GENERAL PROBLEM SOLVER 1957: ROSENBLATT’S PERCEPTRON 1958: SELFRIDGE’S PANDEMONIUM 1957: CHOMSKY’S GRAMMAR 1959: SAMUEL’S CHECKERS 1960: PUTNAM’S COMPUTATIONAL FUNCTIONALISM 1960: WIDROW’S ADALINE 1965: FEIGENBAUM’S DENDRAL

  26. MACHINE INTELLIGENCE: History 1965: ZADEH’S FUZZY LOGIC 1966: WEIZENBAUM’S ELIZA 1967: HAYES-ROTH’S HEARSAY 1967: FILLMORE’S CASE FRAME GRAMMAR 1969: MINSKY & PAPERT’S PAPER ON NEURAL NETWORKS 1970: WOODS’ ATN 1972: BUCHANAN’S MYCIN 1972: WINOGRAD’S SHRDLU 1974: MINSKY’S FRAME 1975: SCHANK’S SCRIPT 1975: HOLLAND’S GENETIC ALGORITHMS 1979: CLANCEY’S GUIDON 1980: SEARLE’S CHINESE ROOM ARTICLE 1980: MCDERMOTT’S XCON 1982: HOPFIELD’S NEURAL NET 1986: RUMELHART & MCCLELLAND’S PDP 1990: ZADEH’S SOFT COMPUTING 2000: ZADEH’S COMPUTING WITH WORDS AND PERCEPTIONS & PNL

  27. What is Soft Computing?

  28. SOFT COMPUTING SC

  29. SOFT COMPUTING “Soft computing is consortium of computing methodologies which collectively provide a foundation for the Conception, Design and Deployment ofIntelligent Systems.” L.A. Zadeh "...in contrast to traditional hard computing, soft computing exploits the tolerance for imprecision, uncertainty, and partial truth to achieve tractability, robustness, low solution-cost, and better rapport with reality” L.A. Zadeh The role model for Soft Computing is theHuman Mind.

  30. SOFT COMPUTING • Neuro-Computing (NC) • Fuzzy Logic (GL) • Genetic Computing (GC) • Probabilistic Reasoning (PR) • Chaotic Systems (CS), Belief Networks (BN), Learning Theory (LT) • Related Technologies • Statistics (Stat.) • Artificial Intelligence (AI): • Case-Based Reasoning (CBR) • Rule-Based Expert Systems (RBR) • Machine Learning (Induction Trees) • Bayesian Belief Networks (BBN)

  31. SOFT COMPUTING • Neural Networks • create complicated models without knowing their structure • gradually adapt existing models using “training data” • Fuzzy Logic • Fuzzy Rules are easy and intuitively understandable • Genetic Algorithms • find parameters through evolution(usually when a direct algorithm is unknown)

  32. Neural Networks • Ensemble of simple processing units • Connection weights define functionality • Derive weights from “training data”(usually gradient descent based algorithms)

  33. Fuzzy Logic • Allow partial membership to sets • Express knowledge through linguistic terms and rules (“Computing with Words”) • Derive sets of Fuzzy Rules from data (usually based on heuristics)

  34. Evolutionary Algorithms • Finding an optimal structure (parameters) for a model is often complicated (due to large search space, complex structure) • Find structure (parameters) through evolution (generate population, evaluate, breed new pop.)

  35. Why Fuzzy Logic? • Uncertainty in the data and laws of nature* • Imprecision due to measurement & human error • Incomplete and sparse information • Subjective and Linguistic rules • So far as the laws of mathematics refer to reality, they are not certain; and so far as they are certain, they don’t refer to reality” • Albert Einstein

  36. Words are less precise than numbers! • When information is too imprecise • Close to reality • Complex problem • “As complexity rises, precise statements lose meaning, and meaningful statements lose precision.” • Lotfi A. Zadeh

  37. Why Neural Network? • Structure Free & Nonlinear Mapping • Multivariable Systems • Trains Easily Based on Historical Data • Parallel Processing & Fault Tolerance • *Much Like Human Brain

  38. Why Evolutionary Computing? • For Multi-objectives and Multi-Criteria Optimization Purposes • Resolving Conflict • Capability to learn. Adapt, and to be self-aware • * Darwinian's law

  39. Why Fuzzy Evolutionary Computing? • To Extract Fuzzy Rules • The Tune Fuzzy Membership

  40. Neural Network Fuzzy Logic Models • A Neural Network capturing presence of Fuzzy Rules • Ideal for Knowledge Acquisition / Discovery • Introduces fuzzy weights to NN internal structure

  41. SOFT COMPUTINGNeural Network NNet

  42. Biological Neuron

  43. Biological Neuron

  44. Biological vs. Artificial Neuron

  45. Analogy between biological and artificial neural networks

  46. Artificial Neuron

  47. Schematic Diagram for Single Neuron b 1 w22 w1 y=f(s) s=xiwi x1 y wk xk y = f [ b+ w1 x1 + w2 x2 + … + wk xk ] . w22

  48. tanh(s) Activation Functions f(s) sgn(s) 1 s -1

  49. Multilayer perceptron with two hidden layers i g n a l s Input Signals O u t p u t S First Second Input hidden hidden Output layer layer layer layer Artificial Neural Network (Feedforward)

More Related