Natural Language Computing and Reasoning

1. Natural Language Computing and Reasoning

2. Symptoms X Diagnosis Symptoms Diagnosis Test Attribute Set/ Question

3. Symptoms X Diagnosis • The use of Linguistic variables • Simple relations between variables by fuzzy conditional statement • Complex relations by fuzzy Algorithms IF Symptom (A1 ) is a11 and Symptom (A2 ) is a12 and Symptom (An ) is a1n Then Diagnosis (F1) is a1 IF Symptom (A1 ) is a21 and Symptom (A2 ) is a22 and Symptom (An ) is a2n Then Diagnosis (F1) is a1 Question: IF Symptom (A1 ) is a1 and Symptom (A2 ) is a2 and Symptom (An ) is an Then Diagnosis (F1) is b?

4. Finance • stock prices and characteristics, credit scoring, credit card ranking Military • battlefield simulation and decision making Medicine • diagnosis Internet Marketing Education Banking • university admission • provide knowledge and advice to large numbers of user • fraud detection • store and product display • electronic shopping Other Applications Description Application

8. University admissions Different admission rates and Varying criteria depending on the University strategy; e.g. UC-Berkeley and Stanford University

9. Outline • BISC Decision Support System • Neuro-Fuzzy-Evolutionary Computing: NeF-ECom • Multi-Criteria Decision Analysis with Uncertain and Incomplete Information • Application Areas • ASIS

10. BISC- Decision Support System BISC-DSS Human Knowledge HM First Principle Models Data Knowledge

11. OBJECTIVES Develop soft-computing-based techniques for decision analysis • Tools to assist decision-makers in assessing the consequences of decision made in an environment of imprecision, uncertainty, and partial truth and providing a systematic risk analysis; • Tools to assist decision-makers answer “What if Questions”, examine numerous alternatives very quickly and find the value of the inputs to achieve a desired level of output; • Tools to be used with human interaction and feedback to achieve a capability to learn and adapt through time;

12. DECISION ENVIRONMENT • Information (Can be uncertain) • Granular (Scale and Precision) • Query (Can be imprecise) • Measure (Similarity) • Aggregation (Can be fuzzy) • Ranking (Provide Alternatives) • Optimization (Multi-Objective & Multi-Criteria)

13. BISC DSS: Components and Structure Model and Data Visualization • Model Management • Query • Aggregation • Ranking • Fitness Evaluation Evolutionary Kernel Genetic Algorithm, Genetic Programming, and DNA • Selection • Cross Over • Mutation Input From Decision Makers Experts Knowledge Model Representation Including Linguistic Formulation Data Management • Functional Requirements • Constraints • Goals and Objectives • Linguistic Variables Requirement

14. Query (Request): Q • find if such query exists  degree of match  rank  decision ( i.e. resource allocation) • compare queries  rank  decision (task allocation) • Use Fuzzy Min-Max with degree of preferences

15. Objective function: Cost Function/ Fitness Function This may involve multi-objective, multi-criteria optimization with conflict and fuzzy variables. Therefore, use fuzzy-GA to solve the objective function.

16. University admissions Different admission rates and Varying criteria depending on the University strategy; e.g. UC-Berkeley and Stanford University

17. Actual Model Given Student Rate of Success Predicted Model Using Fuzzy-GA Initial GA Population of Models

18. The BISC Decision Support System Conventional GA: Multi-Objective Multi-Criteria Optimization Max Preferences Mean Actual Prediction 0.5010 0.7961 0.5010 0.5176 0.5210 0.5686 0.4800 0.4588 0.5010 0.7176 0.5010 0.8588 0.5010 0.9490 0.5010 0.6980 0.5010 0.5922 0.5010 0.9373 0.5000 0.7412 0.5210 0.7608 0.5210 0.6353 0.5630 0.6784 0.5210 0.7490 0.5420 0.8667 0.5630 0.7843 Fitness Min. Std Dev. Generation

19. The BISC Decision Support System Interactive-GA Multi-Objective Multi-Criteria Optimization Max Preferences Preferences Actual Predicted 0.5010 0.4609 0.5010 0.4907 0.5210 0.5712 0.4800 0.4709 0.5010 0.5381 0.5010 0.5106 0.5010 0.5513 0.5010 0.5469 0.5010 0.5161 0.5010 0.5061 0.5000 0.5106 0.5210 0.5701 0.5210 0.5425 0.5630 0.5469 0.5210 0.5370 0.5420 0.4444 0.5630 0.5017 Mean Fitness Min. Std Dev. Generation

20. BISC-DSS Software Neuro-Fuzzy-Evolutionary ComputingMulti-Criteria Decision Analysis with Uncertain and Incomplete Information NeF-ECom

21. BISC – DSS Software: Architecture • Aggregation operators • Similarity measures • Norm-Pairs • Fuzzy sets Application Template Fuzzy Search Engine (FSE) User Interface Evolutionary Computing Kernel DB

22. 1 1   Ak Ak 1  Ak Basic concepts Fuzzy sets/ Membership Functions (MFs) LowMediumHigh Triangular Gaussian Low diversitydiverseHigh diversity Trapezoidal

23. Basic concepts Fuzzy similarity measures X and Y are fuzzy measures defined over the same fuzzy sets with MFs: µ1, µ2, …, µm Norm-Pair operators  et  (norm-conorm)

24. Basic concepts Norm-Pairs Fuzzy AND [] Fuzzy OR [] x and y are MF values in [0,1].

25. Basic concepts Aggregation Operators

26. Basic concepts Weighted Aggregation Operators

27. Aggregators Attributes Aggregation tree Advanced Multi-Aggregator Model Basic concepts • Parameters • aggregators • weights • tree structure.

28. Compactification Algorithm InterpretationA Simple Algorithm for Qualitative AnalysisRule Extraction and Building Decision TreeNikravesh and Zadeh (2005)(Zadeh, 1976)

29. Symptoms X Diagnosis • The use of Linguistic variables • Simple relations between variables by fuzzy conditional statement • Complex relations by fuzzy Algorithms IF Symptom (A1 ) is a11 and Symptom (A2 ) is a12 and Symptom (An ) is a1n Then Diagnosis (F1) is a1 IF Symptom (A1 ) is a21 and Symptom (A2 ) is a22 and Symptom (An ) is a2n Then Diagnosis (F1) is a1 Question: IF Symptom (A1 ) is a1 and Symptom (A2 ) is a2 and Symptom (An ) is an Then Diagnosis (F1) is b?

30. Symptoms X Diagnosis Test Attribute Set

31. Table 1 (intermediate results) Group 1(initial) Pass (1) Pass (2) Pass (3)

32. MAXIMALLY COMPACT REPRESENTATION

34. Chromosome Representation Fuzzy Label, Set Value, Scalar & Series Input • Composed of primitive statistical, fuzzy set, aggregator, similarity, arithmetic, and signal processing operators. • Each gene (or algorithm) is represented as a tree, accepts both scalar and series input, and outputs scalar features. • The chromosome produces a feature vector set. Scalar & Fuzzy Label Features

35. Aggregators Attributes Aggregation tree Multi Aggregator Tree Advanced Multi-Aggregator Model • Parameters • aggregators • weights • tree structure.

36. H-DT3 Nikravesh & Zadeh (2003) Attrb. BISC-DSS Nikravesh (2000) Attrib./Feature Selection Nikravesh (2005) C-Rules Zadeh (1976) & Nikravesh (2003) Transf. RCR-PFRL Berenji (2003) & Nikravesh (2003) Compactification Zadeh (1976) Signal C-DT3 Zadeh (1976) & Nikravesh (2003) MA-DT3 Nikravesh (2003) Cases SVM, NN (RBF & MLP), NF Nikravesh (2003)

37. BISC-DSS Software

38. BISC-DSS Software

39. EC: Genetic Algorithms • Requirements • - Individual :problem representation • - Fitness function: for evaluation • - Termination criterion • Principle: • Create randomly an initial population of individuals • Evolve the population: • evaluate and select individuals • use them in genetic operators (crossover, mutation) • generate new generation • - Stop if termination criterion satisfied

40. parent 1 child Crossover parent 2 child parent Mutation 0 0 0 1 1 1 0 1 1 1 0 1 1 1 0 1 0 1 1 1 1 0 0 1 0 1 1 0 0 1 1 0 1 1 0 1 1 0 1 0 EC: Genetic Algorithms Genetic Operators

41. EC: Genetic Programming • Individual = Computer program • Most common representation : tree encoding (nodes = functions, leaves = terminals) • Fitness function = returned value by the root node Chosen node Mutation new individual selected individual resulting individual

42. parent 1 child 1 Chosen node parent 2 child 2 Chosen node EC: Genetic Programming Crossover

44. BISC-DSS: Interaction and Optimization • Comparison, Aggregation, Scoring • MODEL based on • Aggregation operators, • Similarity measures • Norm-Pairs • Fuzzy sets DB Fuzzy Search Engine (FSE) QUERY User Interface ANSWERS Evolutionary Computing Kernel User preferences : (re-ranking, selection) OPTIMIZATION

45. S1 S2  SN xN1 x21 x11 y1 x22 y2 xN2 x12     yK x2K x1K xNK Multi-Criteria Decision Model (1) Scores Database Multi-Attribute Query: K attributes A1, A2,…,AK Similarity calculation Query Query Answering Ranking based Selection based (criteria: number top answers)(criteria: threshold)

46. Multi-Criteria Decision Model (2) Query Data Fuzzification Fuzzy sets For each attribute Norm-pairs [,] Fuzzy similarity calculation Fuzzy similarity measures Aggregation model aggregation Scoring Ranking or Selecting Answers

47. Multi-Criteria Decision Model (3) Data: Xi = (xi1, xi2, …, xiK),Query: Q = (y1, y2, …, yk) Kattributes:A1, A2,…,AK For each attribute Aj: rjfuzzy sets µ1(Aj,.), µ2(Aj,.),…,µrj(Aj,.) sj = similarity(xij, yj), j = 1, 2, …, K Score = SIM(Q,Xi) = Aggregation(s1, s2, …, sk)

48. First Order Aggregation Model (1) • Norm-pair: Min/Max • Fuzzy similarity measure: Jaccard • Aggregation operator: Weighted Mean

49. First Order Aggregation Model (2) Aggregation model = simple weighted aggregation operator user preferences = attribute weighting (Degree of importance of each attribute) Aggregation model parameters = weighting vector Optimization process : find the optimal weights Using GA.

50. wk w1 w2 … First Order Aggregation Model (3) • Model parameters learning using GA • GA-based learning module • - Individuals: weight vectors • - Genetic operators: crossover, Mutation • Fitness function • Termination criterion Specific fitness function Problem specification Optimal weights