Computational Intelligence – a Possible Solution for Unsolvable Problems

1. Computational Intelligence – a Possible Solution for Unsolvable Problems Annamária R. Várkonyi-Kóczy Dept. of Measurement and Information Systems, Budapest University of Technology and Economics koczy@mit.bme.hu

2. Contents • Motivation: Why do we need something ”non-classical”? • What is Computational Intelligence? • How CI works? • About some of the methods of CI • Fuzzy Logic • Neural Networks • Genetic Algorithms • Anytime Techniques • Engineering view: Practical issues • Conclusions – Is CI really solution for unsolvable problems? Tokyo Institute of Technology

3. Motivation: Why do we need something ”non-classical”? • Nonlinearity, never unseen spatial and temporal complexity of systems and tasks • Imprecise, uncertain, insufficient, ambiguous, contradictory information, lack of knowledge • Finite resources  Strict time requirements (real-time processing) • Need for optimization + • User’s comfort New challanges/more complex tasks to be solved more sophisticated solutions needed

4. Never unseen spatial and temporal complexity of systems and tasks How can we drive in heavy traffic? Many components, very complex system. Can classical or even AI systems solve it? Not, as far as we know. But WE,humans can. And we would like to build MACHINES to be able to do the same. Our car, save fuel, save time, etc.

5. Never unseen spatial and temporal complexity of systems and tasks Help: • Increased computer facilities • Model integrated computing • New modeling techniques • Approximative computing • Hybrid systems

6. Imprecise, uncertain, insufficient, ambiguous, contradictory information, lack of knowledge • How can I get to Shibuya? (Person 1: Turn right at the lamp, than straight ahead till the 3rd corner, than right again ... NO: better turn to the left) (Person 2: Turn right at the lamp, than straight ahead till appr. the 6th corner ... than I don’t know) (Person 3: It is in this direction → somewhere ...) • It is raining • The traffic light is out of order • I don’t know in which building do we have the special lecture (in Building III or II or ...)? And at what time???? (Does it start at 3 p.m. or at 2 p.m? And: on the 3rd or 4th of October?) • When do I have to start from home at at what time? Who (a person or computer) can show me an algorithm to find an OPTIMUM solution?

7. Imprecise, uncertain, insufficient, ambiguous, contradictory information, lack of knowledge Help: • Intelligent and soft computing techniques being able to handle the problems • New data acquisition and representation techniques • Adaptivity, robustness, ability to learn

8. Finite resources  Strict time requirements (real-time processing) • It is 10.15 a.m. My lecture starts at 3 p.m. (hopefully the information is correct) • I am still not finished with my homework • I have run out of the fuel and I don’t have enough money for a taxi • I am very hungry • I have promised my Professor to help him to prepare some demo in the Lab this morning I can not fulfill everything with maximum preciseness

9. Finite resources  Strict time requirements (real-time processing) Help: • Low complexity methods • Flexible systems • Approximative methods • Results for qualitative evaluations & for supporting decisions • Anytime techniques

10. Need for optimization • Traditionally: optimization = precision • New definition: optimization = cost optimization • But what is cost!? presition and certainty also carry a cost

11. Need for optimization Let’s look ”TIME” as a resource: • The most important thing is to go the Lab and help my Professor (He is my Professor and I have promised it). I will spend there as needed, min. 3 hours • I have to submit the homework, but I will work in the Lab., i.e. today I will prepare an ”average” and not a ”maximum” level homework (1 hour) • I don’t have time to eat at home, I will buy a bento at the station (5 minutes) • The train is more expensive then the bus but takes much less time, i.e. I will go by train (40 minutes)

12. User’s comfort • I have to ask the way to the university but unfortunately, I don’t speak Japanese • Next time I also want to find my way • Today it took one and a half hour to get here. How about tomorrow? • It would be good get more help • ....

13. User’s comfort Help: • Modeling methods and representation techniques making possible to • handle • interprete • predict • improve • optimise the system and • give more and more support in the processing

14. User’s comfort Human language Modularity, simplicity, hierarchical structures Aims of the processing preprocessing processing improving the performance of the algorithms giving more support to the processing (new) aims of preprocessing image processing / computer vision: noise smoothing feature extraction (edge, corner detection) pattern recognition, etc. 3D modeling, medical diagnostics, etc. automatic 3D modeling, automatic ... preprocessing processing

15. The most important elements of the solution • Low complexity, approximative modeling • Application of adaptive and robust techniques • Definition and application of the proper cost function including the hierarchy and measure of importance of the elements • Trade-off between accuracy (granularity) and complexity (computational time and resource need) • Giving support for the further processing These do not cope with traditional and AI methods. But how about the new approaches, about COMPUTATIONAL INTELLIGENCE?

16. What is Computational Intelligence? Computer + Intelligence Increased computer facilities Added by the new methods L.A. Zadeh, Fuzzy Sets [1965]: “In traditional – hard – computing, the prime desiderata are precision, certainty, and rigor. By contrast, the point of departure of soft computing is the thesis that precision and certainty carry a cost and that computation, reasoning, and decision making should exploit – whenever possible – the tolerance for imprecision and uncertainty.”

17. What is Computational Intelligence? • CI can be viewed as a corsortium of methodologies which play important role in conception, design, and utilization of information/intelligent systems. • The principal members of the consortium are: fuzzy logic (FL), neuro computing (NC), evalutionary computing (EC), anytime computing (AC), probabilistic computing (PC), chaotic computing (CC), and (parts of) machine learning (ML). • The methodologies are complementary and synergistic, rather than competitive. • What is common: Exploit the tolerance for imprecision, uncertainty, and partial truth to achieve tractability, robustness, low solution cost and better rapport with reality.

18. Computational Intelligence fulfill all of the five requirements:(Low complexity, approximative modelingapplication of adaptive and robust techniquesDefinition and application of the proper cost function including the hierarchy and measure of importance of the elementsTrade-off between accuracy (granularity) and complexity (computational time and resource need)Giving support for the further processing)

19. How CI works?1. Knowledge • Information acquisition (observation) • Information processing (numeric, symbolic) • Storage and retrieval of the information • Search for a ”structure” (algorithm for the non-algorithmizable processing) • Certain knowledge (can be obtained by formal methods): closed, open world: ABSTRACT WORLDS) • Uncertain knowledge (by cognitive methods) (ARTIFICIAL and REAL WORLDS) • Lack of knowledge • Knowledge representation

20. How CI works?1. Knowledge In real life nearly everything is optimization (Ex.1. Determination of the velocity = Calculation of the optimum estimation of the velocity from the measured time and done distance) Ex.2. Determination of the resistance = the optimum estimation of the resistance with the help of the measured intensity of current and voltage Ex.3. Analysis of a measurement result = the optimum estimation of the measured quantity in the kowledge of the conditions of the measurement and the measured data) Ex. 4. Daily time-table Ex. 5. Optimum route between two towns In Ex. 1-3 the criteria of the optimization is unambiguos and easily can be given Ex. 4-5 are also simple tasks but the criteria is not unambiguos

21. Optimum route: • What is optimum? (Subjective, depending on the requirements, taste, limits of the person) • - We prefer/are able to travel by aeroplane, train, car, ... • Let’s say car is selected: • the shortest route (min petrol need), the quickest route (motorway), the most beautiful route with sights (whenever it is possible I never miss the view of the Fuji-san ...), where by best restaurants are located, where I can visit my friends, ... • OK, let’s fix the preferences of a certain person: • But is it summer or winter, is it sunshine or raining, how about the road reconstructions, .... • By going into the details we get nearer and nearer to the solution • Knowledge is needed for the determination of a good descriptive model of the circumstances and goals • But do we know what kind of wheather will be in two months?

22. 2. Model • Known model e.g. analithic model (given by differential equations) - too complex to be handled • Lack of knowledge - the information about the system is uncertain or imperfect We need new, more precise knowledge The knowledge representation (model) should be handable and should tolerate the problems

23. Learning and Modeling New knowledge by learning: Unknown, partially unknown, known but too complex to be handled, ill-defined systems Model by which we can be analyze the system and can predict the behavior of the system + Criteria (quality measure) for the validity of the model

24. Unknown system Criteria Model u Input d c Measure of the quality of the model y Parameter tuning 1. Observation (u, d, y), 2. Knowledge representation (model, formalism), 3. Decision (optimizasion, c(d,y)), 4. Tuning (of the parameters), 5. Environmental influence,(non-observed input, noise, etc.) 6. Prediction ability (for the future input)

25. Iterative procedure: We build a system for collecting information We improve the system by building in the knowledge We collect the information We improve the observation and collect more information

26. Problem Knowledge representation, Model Represented knowledge Independant space, coupled to the problem by the formalism Non-represented part of the problem

27. 3. Optimization • Valid where the model is valid • Given a system with free parameters • Given an objective measure • The task is to set the parameters which mimimize or maximize the qualitative measure • Systematic and random methods • Exploitation (of the deterministic knowledge) and exploration (of new knowledge)

28. Methods of Computational Intelligence • fuzzy logic –low complexity, easy build in of the a priori knowledge into computers, tolerance for imprecision, interpretability • neuro computing - learning ability • evalutionary computing – optimization, optimum learning • anytime computing – robustness, flexibility, adaptivity, coping with the temporal circumstances • probabilistic reasoning – uncertainty, logic • chaotic computing – open mind • machine learning - intelligence

29. Fuzzy Logic • Lotfi Zadeh, 1965 • Knowledge representation in natural language • ”computing with words” • Perceptions • Value imprecisiation meaning precisiation

30. History of fuzzy theory • Fuzzy sets & logic: Zadeh 1964/1965- • Fuzzy algorithm: Zadeh 1968-(1973)- • Fuzzy control by linguistic rules: Mamdani & Al. ~1975- • Industrial applications: Japan 1987- (Fuzzy boom), KoreaHome electronicsVehicle controlProcess controlPattern recognition & image processingExpert systemsMilitary systems (USA ~1990-)Space research • Applications to very complex control problems: Japan 1991-e.g. helicopter autopilot

31. Areas in which Fuzzy Logic was succesfully used: • Modeling and control • Classification and pattern recognition • Databases • Expert Systems • (Fuzzy) hardware • Signal and image processing • Etc.

32. Universe of discourse: Cartesian (direct) product of all the possible values of each of the descriptors • Linguistic variable (linguistic term) [Zadeh]: ”By a linguistic variable we mean a variable whose values are words or sentences in a natural or artificial language. For example, Age is a linguistic variable if its values are linguistic rather than numerical, i.e., young, not young, very young, quite young, old, not very old and not very young, etc., rather than 20, 21, 22, 23, ...” • Fuzzy set: Itrepresents a property of the linguistic variable. A degree of includance is associated to each of the possible values of the linguistic variable (characteristic function) • Membership value: The degree of belonging into the set.

33. An Example • A class of students (e.g. M.Sc. Students taking • the Spec. Course „Computational Intelligence”) • The universe of discourse: X • “Who does have a driver’s license?” • A subset of X = A (Crisp) Set • (X) = CHARACTERISTIC FUNCTION 1 0 1 1 0 1 1 • “Who can drive very well?” (X) = MEMBERSHIP FUNCTION 0.7 0 1.0 0.8 0 0.4 0.2 FUZZY SET

34. c• • x aAbA • b • d • y • a B Crisp set A If xA then also xB. B • y • x AB A Definitions • Crisp set: • Convex set:A is not convex as aA, cA, butd=a+(1-)c A, [0, 1].B is convex as for every x, yB and[0, 1] z=x+(1-)y B. • Subset:

35. Definitions • Relative complement or difference:A–B={x | xA and xB}B={1, 3, 4, 5}, A–B={2, 6}.C={1, 3, 4, 5, 7, 8}, A–C={2, 6}! • Complement: where X is the universe.Complementation is involutive:Basic properties: • Union:AB={x | xA or xB} For (Law of excluded middle)

36. Definitions • Intersection:AB={x | xA and xB}. For • More properties:Commutativity: AB=BA, AB=BA.Associativity: ABC=(AB)C=A(BC), ABC=(AB)C=A(BC).Idempotence: AA=A, AA=A.Distributivity: A(BC)=(AB)(AC), A(BC)=(AB)(AC). (Law of contradiction)

37. Membership function

38. Some basic concepts of fuzzy sets

39. Some basic concepts of fuzzy sets • Support: supp(A)={x | A(x)>0}.supp:Infant=0, so supp(Infant)=0.If |supp(A)|<, A can be definedA=1/x1+ 2/x2+…+ n/xn. • Kernel (Nucleus, Core):Kernel(A)={x | A(x)=1}.

40. Definitions • Height: • height(old)=1height(infant)=0 • If height(A)=1 A is normal • If height(A)<1 A is subnormal • height(0)=0 • (If height(A)=1 then supp(A)=0) • a-cut: Strong Cut: • Kernel: • Support: • If A is subnormal, Kernel(A)=0

41. Definitions • Fuzzy set operations defined by L.A. Zadeh in 1964/1965 • Complement: • Intersection: • Union: (x):

42. Definitions This is really a generalization of crisp set op’s!

43. Fuzzy Proportion • Fuzzy proportion: X is P‘Tina is young’, where:‘Tina’: Crispage, ‘young’: fuzzy predicate. Fuzzy sets expressing linguistic terms for ages Truth claims – Fuzzy sets over [0, 1] • Fuzzy logic based approximate reasoning • is most important for applications!

44.  CRISP RELATION : SOME INTERACTION OR ASSOCIATION BETWEEN ELEMENTS OF TWO OR MORE SETS.  FUZZY RELATION : VARIOUS DEGREES OF ASSOCIATION CAN BE REPRESENTED A B A B               CRISP RELATION FUZZY RELATION  CARTESIAN (DIRECT) PRODUCT OF TWO (OR MORE) SETS X, Y X  Y = { (x,y)  x  X, y  Y } X  Y  Y  X IF X  Y ! MORE GENERALLY: xi = { (x1, x2, …, xn)  xi Xi , i  Nn } 0.5 0.8 1 0.9 0.6 CR FR n i = 1

45. Fuzzy Logic Control • Fuzzification: converts the numerical value to a fuzzy one; determines the degree of matching • Defuzzification converts the fuzzy term to a classical numerical value • The knowledge base contains the fuzzy rules • The inference engine describes the methodology to compute the output from the input

46. Fuzzyfication μ 1 8,4 X The measured (crisp) value is converted to a fuzzy set containing one element with membership value=1 μ(x) = 1 if x=8,4 0 otherwise

47. DefuzzificationCenter of Gravity Method (COG)

48. Specificity of fuzzy partitions Fuzzy Partition A containing three linguistic terms Fuzzy Partition A* containing seven linguistic terms

49. Fuzzyinferencemechanism (Mamdani) • If x1 = A1,iand x2 = A2,iand...and xn = An,ithen y = Bi The weighting factor wji characterizes, how far the input xj corresponds to the rule antecedent fuzzy set Aj,i in one dimension The weighting factor wi characterizes, how far the input x fulfils to the antecedents of the rule Ri.

50. Conclusion The conclusion of rule Ri for a given x observation is yi