CPE/CSC 481: Knowledge-Based Systems

1. CPE/CSC 481: Knowledge-Based Systems Dr. Franz J. Kurfess Computer Science Department Cal Poly

2. Introduction Knowledge Representation Semantic Nets, Frames, Logic Reasoning and Inference Predicate Logic, Inference Methods, Resolution Reasoning with Uncertainty Probability, Bayesian Decision Making Expert System Design ES Life Cycle CLIPS Overview Concepts, Notation, Usage Pattern Matching Variables, Functions, Expressions, Constraints Expert System Implementation Salience, Rete Algorithm Expert System Examples Conclusions and Outlook Course Overview

3. Motivation Objectives Sources of Uncertainty and Inexactness in Reasoning Incorrect and Incomplete Knowledge Ambiguities Belief and Disbelief Probability Theory Bayesian Networks Dempster-Shafer Theory Certainty Factors \Approximate Reasoning Fuzzy Logic Important Concepts and Terms Chapter Summary Overview Reasoning and Uncertainty

4. Logistics • Introductions • Course Materials • textbooks (see below) • lecture notes • PowerPoint Slides will be available on my Web page • handouts • Web page • http://www.csc.calpoly.edu/~fkurfess • Term Project • Lab and Homework Assignments • Exams • Grading

5. Bridge-In

6. Pre-Test

7. Motivation

8. Objectives

9. Introductions • reasoning under uncertainty and with inexact knowledge • heuristics • ways to mimic heuristic knowledge processing • methods used by experts • empirical associations • experiential reasoning • based on limited observations • probabilities • objective (frequency counting) • subjective (human experience ) • reproducibility • will observations deliver the same results when repeated

10. Dealing with Uncertainty • expressiveness • can concepts used by humans be represented adequately? • can the confidence of experts in their decisions be expressed? • comprehensibility • representation of uncertainty • utilization in reasoning methods • correctness • probabilities • relevance ranking • long inference chains • computational complexity • feasibility of calculations for practical purposes

11. Sources of Uncertainty • data • missing data, unreliable, ambiguous, imprecise representation, inconsistent, subjective, derived from defaults, … • expert knowledge • inconsistency between different experts • plausibility • “best guess” of experts • quality • causal knowledge • deep understanding • statistical associations • observations • scope • only current domain?

12. Sources of Uncertainty (cont.) • knowledge representation • restricted model of the real system • limited expressiveness of the representation mechanism • inference process • deductive • the derived result is formally correct, but wrong in the real system • inductive • new conclusions are not well-founded • unsound reasoning methods

13. Uncertainty in Individual Rules • individual rules • errors • domain errors • representation errors • inappropriate application of the rules • likelihood of evidence • for each premise • for the conclusion • combination of evidence from multiple premises

14. Uncertainty and Multiple Rules • conflict resolution • if multiple rules are applicable, which one is selected • explicit priorities, provided by domain experts • implicit priorities derived from rule properties • specificity of patterns, ordering of patterns creation time of rules, most recent usage, … • compatibility • contradictions between rules • subsumption • one rule is a more general version of another one • redundancy • missing rules • data fusion • integration of data from multiple sources

15. Basics of Probability Theory • mathematical approach for processing uncertain information • sample space setX = {x1, x2, …, xn} • collection of all possible events • can be discrete or continuous • probability number P(xi)likelihood of an event xi to occur • non-negative value in [0,1] • total probability of the sample space is 1 • for mutually exclusive events, the probability for at least one of them is the sum of their individual probabilities • experimental probability • based on the frequency of events • subjective probability • based on expert assessment

16. Compound Probabilities • describes independent events • do not affect each other in any way • joint probability of two independent events A and BP(A  B) = n(A  B) / n(s) = P(A) * P (B) where n(S) is the number of elements in S • union probability of two independent events A and BP(A  B) = P(A) + P(B) - P(A  B) =P(A) + P(B) - P(A) * P (B) • where n(S) is the number of elements in S

17. Conditional Probabilities • describes dependent events • affect each other in some way • conditionalprobability of event a given that event B has already occurredP(A|B) = P(A  B) / P(B)

18. Advantages and Problems of Probabilities • advantages • formal foundation • reflection of reality (a posteriori) • problems • may be inappropriate • the future is not always similar to the past • inexact or incorrect • especially for subjective probabilities • knowledge may be represented implicitly

19. Bayesian Approaches • derive the probability of a cause given a symptom • has gained importance recently due to advances in efficiency • more computational power available • better methods • especially useful in diagnostic systems • medicine, computer help systems • inverse or a posteriori probability • inverse to conditional probability of an earlier event given that a later one occurred

20. Bayes’ Rule for Single Event • single hypothesis H, single event EP(H|E) = (P(E|H) * P(H)) / P(E)or • P(H|E) = (P(E|H) * P(H) / (P(E|H) * P(H) + P(E|H) * P(H) )

21. Bayes’ Rule for Multiple Events • multiple hypotheses Hi, multiple events E1, …, Ei, …, EnP(Hi|E1, E2, …, En) = (P(E1, E2, …, En|Hi) * P(Hi)) / P(E1, E2, …, En)or • P(Hi|E1, E2, …, En) = (P(E1|Hi) * P(E2|Hi) * …* P(En|Hi) * P(Hi)) / k P(E1|Hk) * P(E2|Hk) * … * P(En|Hk)* P(Hk)with independent pieces of evidence Ei

22. Advantages and Problems of Bayesian Reasoning • advantages • sound theoretical foundation • well-defined semantics for decision making • problems • requires large amounts of probability data • sufficient sample sizes • subjective evidence may not be reliable • independence of evidences assumption often not valid • relationship between hypothesis and evidence is reduced to a number • explanations for the user difficult • high computational overhead

23. Dempster-Shafer Theory • mathematical theory of evidence • notations • frame of discernment FD • power set of the set of possible conclusions • mass probability function m • assigns a value from [0,1] to every item in the frame of discernment • mass probability m(A) • portion of the total mass probability that is assigned to an element A of FD

24. Belief and Certainty • belief Bel(A) in a subset A • sum of the mass probabilities of all the proper subsets of A • likelihood that one of its members is the conclusion • plausibility Pl(A) • maximum belief of A • certainty Cer(A) • interval [Bel(A), Pl(A)] • expresses the range of belief

25. Combination of Mass Probabilities • m1  m2 (C) =  X  Y=C m1(X) * m2(Y) / 1- X  Y=C m1(X) * m2(Y) where X, Y are hypothesis subsets and C is their intersection

26. Advantages and Problems of Dempster-Shafer • advantages • clear, rigorous foundation • ability oto express confidence through intervals • certainty about certainty • problems • non-intuitive determination of mass probability • very high computational overhead • may produce counterintuitive results due to normalization • usability somewhat unclear

27. Certainty Factors • shares some foundations with Dempster-Shafer theory, but more practical • denotes the belief in a hypothesis H given that some pieces of evidence are observed • no statements about the belief is no evidence is present • in contrast to Bayes’ method

28. Belief and Disbelief • measure of belief • degree to which hypothesis H is supported by evidence E • MB(H,E) = 1 IF P(H) =1 (P(H|E) - P(H)) / (1- P(H)) otherwise • measure of disbelief • degree to which doubt in hypothesis H is supported by evidence E • MB(H,E) = 1 IF P(H) =0 (P(H) - P(H|E)) / P(H)) otherwise

29. Certainty Factor • certainty factor CF • ranges between -1 (denial of the hypothesis H) and 1 (confirmation of H) • CF = (MB - MD) / (1 - min (MD, MB)) • combining antecedent evidence • use of premises with less than absolute confidence • E1  E2 = min(CF(H, E1), CF(H, E2)) • E1  E2 = max(CF(H, E1), CF(H, E2)) • E =  CF(H, E)

30. Combining Certainty Factors • certainty factors that support the same conclusion • several rules can lead to the same conclusion • applied incrementally as new evidence becomes available • Cfrev(CFold, CFnew) = • CFold + CFnew(1 - CFold) if both > 0 • CFold + CFnew(1 + CFold) if both < 0 • CFold + CFnew / (1 - min(|CFold|, |CFnew|)) if one < 0

31. Advantages and Problems of Certainty Factors • Advantages • simple implementation • reasonable modeling of human experts’ belief • expression of belief and disbelief • successful applications for certain problem classes • evidence relatively easy to gather • no statistical base required • Problems • partially ad hoc approach • theoretical foundation through Dempster-Shafer theory was developed later • combination of non-independent evidence unsatisfactory • new knowledge may require changes in the certainty factors of existing knowledge • certainty factors can become the opposite of conditional probabilities for certain cases • not suitable for long inference chains

32. Fuzzy Logic • approach to a formal treatment of uncertainty • relies on quantifying and reasoning through natural language • uses linguistic variables to describe concepts with vague values • tall, large, small, heavy, ...

33. Get Fuzzy

34. Fuzzy Set • categorization of elements xi into a set S • described through a membership function m(s) • associates each element xi with a degree of membership in S • possibility measure Poss{xS} • degree to which an individual element x is a potential member in the fuzzy set S • possibility refers to allowed values • probability expresses expected occurrences of events • combination of multiple premises • Poss(A  B) = min(Poss(A),Poss(B)) • Poss(A  B) = max(Poss(A),Poss(B))

35. Fuzzy Set Example membership tall short medium 1 0.5 height (cm) 0 0 50 100 150 200 250

36. Fuzzy vs. Crisp Set membership tall short medium 1 0.5 height (cm) 0 0 50 100 150 200 250

37. Fuzzy Inference Methods • how to combine evidence across rules • Poss(B|A) = min(1, (1 - Poss(A)+ Poss(B))) • implication according to Max-Min inference • also Max-Product inference and other rules • formal foundation through Lukasiewicz logic • extension of binary logic to infinite-valued logic

38. Example Fuzzy Reasoning

39. Advantages and Problems of Fuzzy Logic • advantages • general theory of uncertainty • wide applicability, many practical applications • natural use of vague and imprecise concepts • helpful for commonsense reasoning, explanation • problems • membership functions can be difficult to find • multiple ways for combining evidence • problems with long inference chains

40. Post-Test

41. Use of References • [Giarratano & Riley 1998] • [Russell & Norvig 1995] • [Jackson 1999] • [Durkin 1994] [Giarratano & Riley 1998]

42. Important Concepts and Terms • natural language processing • neural network • predicate logic • propositional logic • rational agent • rationality • Turing test • agent • automated reasoning • belief network • cognitive science • computer science • hidden Markov model • intelligence • knowledge representation • linguistics • Lisp • logic • machine learning • microworlds

43. Summary Chapter-Topic