Knowledge-based systems

Knowledge-based systems Rozália Lakner University of Veszprém Department of Computer Science

An overview • Knowledge-based systems, expert systems • structure, characteristics • main components • advantages, disadvantages • Base techniques of knowledge-based systems • rule-based techniques • inductive techniques • hybrid techniques • symbol-manipulation techniques • case-based techniques • (qualitative techniques, model-based techniques, temporal reasoning techniques, neural networks) Engineering Application of AI - PhD Course -

Knowledge-based systems Engineering Application of AI - PhD Course -

Structure and characteristics 1 • KBSs are computer systems • contain stored knowledge • solve problems like humans would • KBSs are AI programs with program structure of new type • knowledge-base (rules, facts, meta-knowledge) • inference engine (reasoning and search strategy for solution, other services) • characteristics of KBSs: • intelligent information processing systems • representation of domain of interest  symbolic representation • problem solving  by symbol-manipulation •  symbolic programs Engineering Application of AI - PhD Course -

Structure and characteristics 2 Engineering Application of AI - PhD Course -

Main components 1 • knowledge-base (KB) • knowledge about the field of interest (in natural language-like formalism) • symbolically described system-specification • KNOWLEDGE-REPRESENTATION METHOD! • inference engine • „engine” of problem solving (general problem solving knowledge) • supporting the operation of the other components • PROBLEM SOLVING METHOD! • case-specific database • auxiliary component • specific information (information from outside, initial data of the concrete problem) • information obtained during reasoning Engineering Application of AI - PhD Course -

Main components 2 • explanation subsystem explanation of system’ actions in case of user’ request typical explanation facilities: • explanation during problem solving: • WHY... (explanative reasoning, intelligent help, tracing information about the actual reasoning steps) • WHAT IF... (hypothetical reasoning, conditional assignment and its consequences, can be withdrawn) • WHAT IS ... (gleaning in knowledge-base and case-specific database) • explanation after problem solving: • HOW ... (explanative reasoning, information about the way the result has been found) • WHY NOT ... (explanative reasoning, finding counter-examples) • WHAT IS ... (gleaning in knowledge-base and case-specific database) Engineering Application of AI - PhD Course -

Main components 3 • knowledge acquisition subsystem • main tasks: • checking the syntax of knowledge elements • checking the consistency of KB (verification, validation) • knowledge extraction, building KB • automatic logging and book-keeping of the changes of KB • tracing facilities (handling breakpoints, automatic monitoring and reporting the values of knowledge elements) • user interface ( user) • dialogue on natural language (consultation/ suggestion) • specially intefaces • database and other connections • developer interface ( knowledge engineer, human expert) Engineering Application of AI - PhD Course -

Main components 4 • the main tasks of the knowledge engineer: • knowledge acquisition and design of KBS: determination, classification, refinement and formalization of methods, thumb-rules and procedures • selection of knowledge representation method and reasoning strategy • implementation of knowledge-based system • verification and validation of KB • KB maintenance Engineering Application of AI - PhD Course -

Expert Systems Engineering Application of AI - PhD Course -

Structure and characteristics 1 • expert systems knowledge-based systems • employ expert’ knowledge • applied in a narrow specific field • solve difficult problems (must be demand onspecial knowledge) • specialized human experts are needed • experts must be agreed on the fundamental questions of professional field • learning examples andraw data are needed • expectations from an ES (like a human expert): • make intelligent decision: offer intelligent advice and explanations • question/ answer (“treated as an equal conversation partner”) • explanation of questions • acceptable advice even in case of uncertain situation Engineering Application of AI - PhD Course -

Structure and characteristics 2 • AI programs: intelligent problem solving tools • KBSs AI programs with special program structure separated knowledge base • ESs KBSs applied in a specific narrow field Engineering Application of AI - PhD Course -

Expert system shells 1 • „empty” ESs, contain all the active elements of an ES • emptyKB, powerful knowledge acquicition subsystem • contain services for construction and operation of ES independently of the field of interest • support the development of rapid prototype and the incremental construction • examples: CLIPS, GoldWorks, G2, Level5 Engineering Application of AI - PhD Course -

Expert system shells 2 Engineering Application of AI - PhD Course -

Advantages of KBSs and ESs • make up for shortage of experts, spread expert’ knowledge on available price (TROPICAID) • field of interest’ changes are well-tracked (R1) • increaseexpert’ ability and efficiency • preserve know-how • can be developed systems unrealizabled with tradicional technology (Buck Rogers) • self-consistents in advising, equable in performance • are available permanently • able to work even with partial, non-complete data • able to give expanation Engineering Application of AI - PhD Course -

Disadvantages of KBSs and ESs • their knowledge is from a narrow field, don’t know the limits • the answers are not always correct (advices have to be analysed!) • don’t have common sence (greatest restriction) all of the self-evident checking have to be defined (many exceptions  increase the size of KB and the running time) Engineering Application of AI - PhD Course -

Base techniques of KBSs Engineering Application of AI - PhD Course -

Techniques of KBSs based on the knowledge-representation methods and reasoning strategies applied in the implementation • rule-based techniques • inductive techniques • hybrid techniques • symbol-manipulation techniques • case-based techniques • (qualitative techniques, model-based techniques, temporal reasoning techniques, neural networks) Engineering Application of AI - PhD Course -

Rule-based techniques(a short review) Engineering Application of AI - PhD Course -

Reasoning with rules 1 • knowledge-representation form: rule • rule-base can be according to the structure of KB • simple/unstructured • structured (contexts) • reasoning strategies: • according to the control direction • data-driven/forward chaining • goal-driven/backward chaining Engineering Application of AI - PhD Course -

Reasoning with rules 2 • aim: proving a goal statement or achieving a goal state • the reasoning algorithm: • pattern matching • finding applicable rules (watching condition/conclusion part of rules) • fireable rules  conflict set (match condition/conclusion part of rules) • conflict resolution • selecting the most appropriate rule from conflict set • conflict resolution strategies • firing • executing the selected rule  new knowledge (new facts or new subgoals to be proved) • watching termination conditions • restart of the cycle Engineering Application of AI - PhD Course -

Inductive techniques Engineering Application of AI - PhD Course -

Inductive reasoning • a type of machine learning technics • inferring from individual cases to general information • given a collection of training examples (x, f(x)) • return a function h that approximates f • h is called hypothese • aim: finding the hypothese fits well on the training examples • h is used for prediction the values of the unseen examples Engineering Application of AI - PhD Course -

Decision tree 1 • one of the most known methods of inductive learning: learning decision trees • decision tree: simple representation for classifying examples • elements of the decision tree: • nonleaf (internal) nodes are labelled with attributes (A) • arcs out of a node are labelled with possible attribute values of A • leaf nodes are labelled with classifications (Boolean values –yes/no - in the simplest case) Engineering Application of AI - PhD Course -

Country Age Engine Colour Easy to sell 1. Germany 3-6 diesel white yes Japan yes 2. 6-10 diesel red 3. Japan 3-6 diesel blue no Decision tree 2 We want to classify new examples on property Easy to sell based on the examples’ Country, Age, Engine and Colour. Engineering Application of AI - PhD Course -

Decision tree 3 • a decision tree under construction contains: • nodes labelled with attributes • nodes labelled with classifications (yes/no values) • unlabelled nodes • arcs labelled with attribute values outlet only form nodes labelled with attributes • every unlabelled nodes possess: • a subset of training examples • eligible attributes Engineering Application of AI - PhD Course -

Decision tree 4 • some questions about decision tree: • Given some data (set of training examples and attributes), which decision tree should be generated? • A decision tree can represent any discrete function of the inputs. Which trees are the best predictors of unseen data? • You need a bias (preference for one hypothesis over another). Example, prefer the smallest tree. • Least depth? • Fewest nodes? • How should you go about building a decision tree? The space of decision trees is too big for systematic search for the smallest decision tree. Engineering Application of AI - PhD Course -

Learning decision trees 1 • learning decision tree  ID3 algorithm: • initially decision tree contains an unlabelled node with all of the training examples and attributes • selecting an unlabelled node (n) with non-empty set of training examples (T) and non-empty set of attributes (A) • if T is homogen class  n leaf node, label with the classification • otherwise • choosing the „best” attribute (B) from A • extension of the tree with all of the possible attribute values of B (devide into subclasses) • classification of T to the children nodes according to the attribute values (assign the elements of T to subclasses) • continue with step 2. • building the tree top-down Engineering Application of AI - PhD Course -

Learning decision trees 2 • how to choose the „best” attribute? • attribute divides the examples into homogen classes • otherwise attribute makes the most progress towards this • hill-climbing search on the space of decision trees • searching for the smallest tree  heuristics (maximum information gain) • information gain of an attribute test • measures the difference between the original information requirement and the new requirement (after the attribute test) • information gain (G) it is based on information contents (entropy, E) where: S: set of classified examples, A: attribute S1, … , Sn: subsets of S according to A E: entropy Engineering Application of AI - PhD Course -

Learning decision trees 3 Engineering Application of AI - PhD Course -

Using decision trees 1 • major problem with using decision tree: overfitting • occurs when there is a distinction in the tree that appears in the training examples, but it doesn’t appear in the unseen examples • handling overfitting: • restricting the splitting, so that you split only when the split is useful • allowing unrestricted splitting and pruning the resulting tree where it makes unwarranted distinctions: • examples are devided into two sets: training set and test set • constructing a decision tree with the training set • examining all of the nodes with the test set: whether the subtree under the node is replaceable with a leaf node Engineering Application of AI - PhD Course -

Using decision trees 2 • supporting knowledge acquisition/ fast prototype-making (rule-based/ hybrid systems with inductive services) • each one row in the matrix of training examples is a rule • better: each one path (root  leaf) on the decision tree is a rule IF (Author = known) and (Thread = new) and (Length = short) THEN (Reads = true) IF (Author = unknown) and (Thread = new) and (Length = long) THEN (Reads = true) … IF (Author = known) THEN (Reads = true) IF (Author = unknown) and (Thread = new) THEN (Reads = true) IF (Author = unknown) and (Thread = old) THEN (Reads = false) Engineering Application of AI - PhD Course -

Main components of inductive systems Engineering Application of AI - PhD Course -

Main steps of inductive systems • problem definition (knowledge representation): • attributes (head of the matrix, generate coloumns, define object classes) • training examples (fill the raws of the matrix, define instances) • reasoning (generating a hypothese) • checking the contradiction freeness of the training examples • learning optimal decision tree (DT)  knowledge base • control (operating the system) • classification of user’ (unknown) examples (traversing DT) • analysis of user’ examples (with the help of DT) Engineering Application of AI - PhD Course -

Hybrid techniques Engineering Application of AI - PhD Course -

Characteristics of hybrid systems • supporting various programming techniques: • frame-based techniques • rule-based techniques • data-driven reasoning • goal-driven reasoning • inductive techniques • realization: • using of object-oriented tools Engineering Application of AI - PhD Course -

Frames • knowledge-representation unit developed on epistemology foundations • formal tool using for description of structured objects or events or notions • characteristics of frames: • a frame contains: • the name of the object/event • its important properties (attributes)  stored in slots (slot identifier, type, value – it can be another frame) • classes, subclasses, instances • hierarchical structure (is_a, instance_of relations) • inheritance (classes - subclasses, classes - instances) • procedures controlled by events: daemons Engineering Application of AI - PhD Course -

Formalization of frames 1 • directed graph Engineering Application of AI - PhD Course -

Formalization of frames 2 • description in frame-based environment frame person frame student frame subject is_a class is_a person is_a class f_name: subjects: collection_of subject name: l_name: end precond: collection_of end subject end frame Peter frame ES instance_of student isnstance_of subject f_name: Peter name: Expert_systems l_name: Kis precond: AI subjects: ES end end Engineering Application of AI - PhD Course -

Formalization of frames 3 • object-attribute-value triplets <Peter, f_name, Peter> <Peter, l_name, Kis> <Peter, subjects, [ES]> <ES, name, Expert_systems> <ES, preconditions, [AI]> Engineering Application of AI - PhD Course -

Daemons 1 • active elements of a frame system • standard built-in procedures • assigned to the attributes of the classes and instances • automatically invoked in case of predefined changing in the value of the slot • usual daemons are as follows: • when-needed: describes the steps to be performed when the value of slot is read • when-changed: is invoked when the value of the slot is changed • when-added: contains the actions to be performed when the slot gets its first value • when deleted: is executed when the value of the slot is deleted Engineering Application of AI - PhD Course -

Daemons 2 • the executable part of the daemons is determined by the user or it may even be empty • execution is controlled by events • daemons can invoke (call) each other via changing slot values  spread over and over • the operation of a frame system is described in an indirect way (embedded in the daemons) • daemons can be used for restricted data-driven reasoning Engineering Application of AI - PhD Course -

Daemons versus rules Engineering Application of AI - PhD Course -

Hybrid techniques • rules: used for description of heuristic knowledge • frames: contains both descriptive and procedural knowledge of the given objects/ events/ notions (altogether in one place!  easy to read and modify, the effects of modifications can be held easily) • inference engine of hybrid techniques can contain: • mechanisms insuring inheritance and handling of daemons • mechanisms insuring message changing (object-oriented) • data-driven and/or goal-driven reasoning mechanism • can support the organization of rules and/or frames into hierarchical modules • can support making and using of meta-rules Engineering Application of AI - PhD Course -

Symbol-manipulation techniques Engineering Application of AI - PhD Course -

Programming languages of AI • high-level symbol-manipulation languages are used to support the implementation of AI methods • LISP (LISt Processing) • based on the notion and operations of lists • all of the problems can be described in the form of function calls • PROLOG (PROgramming in LOGic) • high-level declarative language • define relationships between various entities with the help of logic • special type of clause (A  B1 …  Bn): fact, rule, question • reasoning environment with a built-in inference engine • answer to a question with the help of logical reasoning • goal-driven (backward) reasoning Engineering Application of AI - PhD Course -

Comparison of symbol-manipulation and traditional techniques Engineering Application of AI - PhD Course -

Case-based techniques Engineering Application of AI - PhD Course -

Case-based reasoning (CBR) 1 • basic assumption: like was the past like will be the future • the „really” observation can be describe hard with the help of classical rules • it consists of interconnected relationships of more or less generalized events • idea: • solving problems based on solutions for similar problems solved in the past • requires storing, retrieving and adapting past solutions to similar problems Engineering Application of AI - PhD Course -

Case-based reasoning 2 • solve a new problem by making an analogy to an old one and adapting its solution to the current situation • retrieving a case starts with a problem description and ends when a best matching case has been found • all case-based reasoning methods have in common the following process: • identifying a set of relevant problem descriptors • retrieve the most similar case (or cases) comparing the case to the library of past cases • reuse the retrieved case to try to solve the current problem • revise and adapt the proposed solution if necessary • retain the final solution as part of a new case Engineering Application of AI - PhD Course -

Knowledge-based systems