ICT619 Intelligent Systems Topic 7: Case Based Reasoning

1. ICT619 Intelligent SystemsTopic 7: Case Based Reasoning

2. Case Based Reasoning • Introduction • How CBR works • Business Applications of CBR • CBR Development methodology and Tools • Advantagesof CBR systems • Case Study ICT619

3. What is Case Based Reasoning? • An intelligent systems methodology based on using stored past problem solving experience to solve a current problem • Similarity with human problem solving - analogical reasoning or memory-based reasoning • CBR draws on similarities and differences between a given problem and a similar problem solved in the past • CBR learns from experience • Adds a solved problem to the case base for use in future • Difference with ANN learning - does not generalise • Difference with rule-based systems - cases are not rules ICT619

4. Case Based Reasoning Terms • A case denotes a problem situation, experienced or learned in the past and retained in a case base • A case may also be an unsolved case – a problem to be solved at present or in the future • Note that some aspects of the law are like this - based on a library of precedents, expressed as cases ICT619

5. CBR History and Current Status • 1980s: Early work done by Roger Schank - Yale University • Funding support from the US Defence Advanced Research Projects Agency (DARPA) • Started later in Europe - Germany most active • 1990: Commercial applications appear • 1998: Research activity in >35 institutions around the world • 15 reported CBR commercial tools • Many applications in daily use ICT619

6. A Case Attributes … … … Solution … How CBR works • The knowledge base of a CBR consists of cases - units of experience consisting of problems and solutions • Each case in case base is defined in terms of its attributes and the solution found for it ICT619

7. Steps in the case-based reasoning cycle: 1. Retrieve a case matching given problem 2. Adapt the matching case’s solution to produce desired solution 3. Test and revise suggested solution 4. Retain confirmed solution by adding it to case base for future use. ICT619

8. Problem New Case RETRIEVE Retrieved Case Learned Case New Case Case Base REUSE Previous Cases RETAIN General Knowledge Tested/ Repaired Case Solved Case REVISE Suggested Solution Confirmed Solution The CBR cycle (Aamodt 1994) Steps in the CBR Cycle ICT619

9. A Simple Example of Problem Solving Using CBR (Bergmann 1998) • Problem: Find the cause for a fault in a car and a suggest a repair strategy • Case base consists of cases containing: • A description of symptoms • A description of the cause • A description of the repair strategy • Each case describes one particular diagnostic situation • It records several features and their specific values found for that situation • Each case describes one particular situation independently of any another ICT619

10. A Simple Example (continued) ICT619

11. A Simple Example (continued) To solve a specific fault finding problem • A case for it is built up without the solution part • Attribute values are gathered by observing symptoms (eg, engine not starting) and measuring values (eg battery voltage = 6.3V) • Not all attributes values may be known for a problem. • In other problem domains the unsolved problem may have all or part of the solution, but one or more of the attributes missing. ICT619

12. Symbolic Attribute: Fault 0.8 Brake lights not turning on Front lights not turning on 0.4 Engine not starting Frontlights not turning on Numeric Attribute: Battery Voltage 0.9 13.6 V 12.6 V 0.1 6.7 V 12.6 V Similarity computation in simple example • The degree of similarity is expressed using a number in the range 0.0 (no similarity at all) to 1.0 (complete similarity) • Attributes are given weights reflecting their significance In this example: weight 6 given for higher importance and weight 1 for lower importance ICT619

13. Adaptation of matching case in example • Once a matching case has been found its solution part is adapted • The adaptation is based on • Differences and similarities between the given problem and the matched case (Case 1) • How the differences should affect the solution • One heavily weighted attribute is fault • In the example, fault has values “Brake lights not turning on” and “Front lights not turning on” • Significant similarity in nature (light problem), but difference in specifics (type of light) • Differences and similarities in other attributes (eg, make and year of manufacture) are not significant • Adaptation done in matching case’s solution part - “front lights” changed to “brake lights”. ICT619

14. Adaptation of matching case in example Case 1 (Symptoms) - Fault : Front light not turning on - Car : VW Golf - Year : 1993 - Batt. Voltage : 13.6 V - State of lights : OK - State of light switch : OK 0.8 Problem to be solved (Symptoms) - Fault : Brake light not turning on - Car : Audi 80 - Year : 1989 - Batt. Voltage : 12.6 V - State of lights : OK 0.4 0.6 0.9 1.0 Solution - Diagnosis : Front light fuse defect - Repair : Replace front light fuse Important attribute: weight = 6 Less important attribute: weight = 1 Similarity Computation by Weighted Averaging Similarity (problem to be solved, case 1) = 1/20 * [6*0.8 = 1*0.4 + 1*0.6 + 6*0.9 + 6*1.0] = 0.86 ICT619

15. Case 1 (Symptoms) - Fault : Front light not turning on - … Solution - Diagnosis : Front light fuse defect - Repair : Replace front light fuse Problem to be solved (Symptoms) - Fault : Brake light not turning on - Car : Audi 80 - Year : 1989 - Batt. Voltage : 12.6 V - State of lights : OK Adapted Solution - Diagnosis : Brake light fuse defect - Repair : Replace brake light fuse Adaptation of matching case in example ICT619

16. Reuse (by adaptation) of the Solution in Case 1 Case 3 (Symptoms) - Fault : Brake light not turning on - Car : Audi 80 - Year : 1989 - Batt. Voltage : 12.6 V - State of lights: OK Solution - Diagnosis : Brake light fuse defect - Repair : Replace brake light fuse Storage of new Experience ICT619

17. Representation of Cases • CBR heavily dependent on structure and content of case base • Case search and retrieval of matching processes needs to be efficient • Actual case representation depends on domain and task requirements • Also influenced by the structure of the already available case data ICT619

18. Some of the representation approaches Flat feature-value list • A simple structure • Sometimes sufficient for solving problems in a given domain • Allows relatively easy storage and retrieval in a CBR system Object-oriented (OO) representations • Reflect the case structure in a hierarchical fashion • A case consists of a set of objects • Objects described by a set of attributes • Each object belongs to an object-class. Object-classes are organised in an inheritance ICT619

19. Car … … Transmission system Brake system Motor Ignition system Fuel injection system Spark plug Spark coil Some of the representation approaches Object-oriented (OO) representations (con't) ICT619

20. Some of the representation approaches Graph representations • Graph representations consist of • a set of nodes • arcs joining the nodes • A more flexible but complex structure compared with a hierarchical representation ICT619

21. 3 Concrete Number of lanes Deck type Performance/ Cost attributes: Cost < $20 million Length Purpose Medium Auto Graph representation of a case (Dhar & Stein 1997) Some of the representation approaches Graph representations (con't) ICT619

22. Computation of Similarity for Matching • Different attributes usually carry different levels of significance • So simple comparison for retrieving similar cases not useful • Attribute values assigned weight values to reflect their significance • This may be done • A priori based on user experience • Depending on importance they assume for a specific problem instance - “discriminating power” • Attributes may be numeric or symbolic Numeric valued attributes • The nearest-neighbour algorithm is commonly used for measuring inter-case distances • But the nearest-neighbour algorithm becomes less and less reliable with increasing number of attributes ICT619

23. Computation of Similarity for Matching Symbolic attributes • Similarity may be measured heuristically by using set/subset relationships • Cases represented hierarchically depending upon their attributes • For example, two cases B and C with symbolic attributes x’and x’’ have the case A as parent if x’ and x’’ are special instances (subsets) of A’s attribute x. • Distance between two symbolic cases may be measured by their distances to their common parent. ICT619

24. A C B E D Case D is more similar to case E (common parent B) than to case C (a more distant common parent A) Computation of Similarity for Matching • The distance calculation module of a CBR system may take the form of a statistical model, a rule-based system or a neural network. ICT619

25. Case Storage • Efficient storage and retrieval of cases is essential for large case bases • Storage method depends on the case representation scheme, and the size of the case base.The two main approaches are: • Linear lists, for small case bases • Index structures consisting of trees or nets, for large case bases. • Internal vs. External Storage • For small case bases and non-shared data, the main memory stores the case base. • Databases if the case base is large, or if the data is shared with other applications ICT619

26. (Partial) Problem Description Identify Attributes Initially match Search & Select Best Matching Case Case Retrieval ICT619

27. Case Retrieval Identification of Attributes • Generates relevant problem descriptors from the user input • Unknown descriptors disregarded or asked to be explained by user • Descriptors may be inferred by using contextual general knowledge Initial Match • Cases that match all input attributes are good candidates for selection • Cases that match a given fraction of problem features may also be retrieved • Similarity assessment may be more knowledge-intensive Select • Best match chosen from good candidates found in initial match • Involves closer inspection and ranking • Knowledge-intensive selection methods typically generate explanations ICT619

28. Case Reuse • The principal issue for reuses is how to adapt the solution part from the best matching case to make it suit given problem • Possibilities: • No modification of the solution: simply copies matching case’s solution part! • Manual/interactive solution adaptation by the user • Automatic solution adaptation • Automatic solution adaptation carried out by • Using the past case solution – known as transformational analogy • Using the method that constructed the past case solution – derivational analogy. ICT619

29. Case Reuse • In transformational analogy, rules or operators are used to adjust the past case solution with respect to differences in the two situations • In derivational analogy, the retrieved case holds information about the method used for solving the retrieved problem. Applies retrieved method to the new case. ICT619

30. Case Revision • Consists of two tasks: (1) evaluation of case solution generated by the reuse, and if it fails evaluation, (2) repair of the case solution using domain-specific knowledge • The evaluation task may take the form of • Applying the solution in the real environment to verify its correctness, quality & user acceptance • Computer simulation (try out before you commit) • The case repair task involves • detecting errors in the current solution and modifying the solution so that failures do not occur • Case retention • Tested, verified and (if necessary) revised solutions accepted as a correct solution • and retained by adding it to the case base ICT619

31. Business Applications of CBR(Allen 1994) • Many focused on case retrieval for decision support • Case retrieval avoids the step of case adaptation • Aid decisions to be based on the most similar available precedents • Customer service help desk • Volatile nature of the problem domain • Knowledge acquisition and maintenance too expensive for traditional expert systems • Example - Compaq Computers’ SMART system • Automation of business processes • Wide-scale distribution of technical and managerial expertise • Example - NEC’s SQUAD a corporate-wide system for capture and distribution of software quality control knowledge • Some 3000 cases added to the system per year since 1982 ICT619

32. Business Applications of CBR • Design and configuration • Support reuse and modification of standard designs • Used by Nippon Steel and Lockheed. • Applications of CBR systems have also been reported in the following areas: • Technical fault diagnosis • Classification and prediction • Control and Monitoring • Planning • Bank loan analysis • The CBR approach to automating planning and scheduling is an active area of research ICT619

33. Phases of CBR Development • Methodologies for CBR system development share the following phases: Case-base Design • A general representation for cases developed using source materials - written documentation and expert accounts, and database records • Involves a coordinated effort by user, managers and system developers • Tasks: • Compilation of a lexicon of terms used to describe problem attributes • Selection of appropriate attributes for indexing cases • Specification of database schemas used to store cases • Definition of case base authoring standards. ICT619

34. Phases of CBR Development Initial Case-Base Development • A “seed” case base developed as a baseline • This case base reviewed and refined by users and developers until a valid case base covering an adequate part of the case space developed Ongoing Development and Maintenance • Initial case base further refined through execution of the revise and retain steps during use • Case accuracy and utility are monitored • Case base managed like an organisational database ICT619

35. CBR Tools • Commercial tools like expert system shells, available for building CBR systems • Facilitate quick development of applications • A typical CBR development environment, as reported in 1994, provides • Default database schemes for case representation • Problem solving tool for case-based decision support • Forms used for editing cases, attributes, and solutions • Utilities include those for • manual and automated indexing of cases • automatic import of cases from records in relational database tables • conceptual clustering of cases for analysis. • Some examples of CBR tools • ReMind from Cognitive Systems Inc. • CBR Express from Inference Corporation • Esteem from Esteem Software Inc • CasePower from Inductive Solutions Inc. • ReCall from Isoft ICT619

36. Advantages of CBR systems • Solves difficult to model problems • In many application areas, eg in business, problems are often unstructured and difficult to model • Reduced knowledge acquisition effort • Reliance on experts modest, especially if good data already available • Easier for experts to describe case attributes rather than providing heuristic rules for solution • Reduced maintenance effort • Carried out by addition or deletion of cases • Reliance on experts is modest, cases easy to understand (cf. rules in a rule base) • High scalability and flexibility • Case base easy to expand and refine • Enhancement and refinement happens as part of the overall operation and use • Mistakes corrected relatively easily by adapting cases • Performance improves over time through refinement • Changes in environment get reflected through the addition of new and/or deletion of outdated cases ICT619

37. Weaknesses of CBR systems • Critics of CBR complain that it uses ad hoc or anecdotal evidence as its main operating principle - weak for the reasons human ad hoc adoption of cases from memory is weak • Without statistically relevant data for backing and implicit generalization, there is no guarantee that the generalization is correct • Some of the work in CBR may be 'handed off' to human operator - eg case database needs begin by hand-crafted cases, which could be difficult to write • Response time may suffer as number of cases in the case base grows (depends on indexing method) ICT619

38. Case Study - Customer Support (Dhar & Stein 1997, pp.236-243) • A very successful intelligent system for customer support system developed at Compaq Computer Corporation • Given the complexity of a modern personal computer system, effective customer support is a major undertaking • The customer support engineer must enter a call for support into a logging system, analyse the customer’s data, resolve the problem, and deliver the solution. • This task is particularly challenging due to • Dynamic nature of the problem domain - with an increasing range of products, support staff faces an increasing variety of questions. • Widening scope of problem domain - the use of more and more third party hardware and software that must be integrated with Compaq’s products • Distributed expertise - due to the width and diversity of the problem domain, few support staff ever experience the full range of problems ICT619

39. Case Study (continued) • Compaq had to either increase its support staff number or reduce the number of incoming calls and the time taken to resolve a call, while maintaining a high support standard • Compaq needed to develop an intelligent support system that would be a central repository of problem solving expertise. The requirements • The system must integrate knowledge that was highly distributed in nature. • The system must handle a large and changing array of models, products and configurations. • Users (service engineers, dealers) must be able to find solutions quickly so that the customer could be got back to within a few minutes of time. • Users need not require deep knowledge about all problem areas. • The system must be able to handle incomplete or inexact input since many customers may not be able to describe the problem fully or accurately. • The system must be accessible to many users at different locations. • To gain user confidence, some explanation capability was needed. • The system must be easily expandable over time to allow new kinds of problems to be added. ICT619

40. Case Study (continued) • Some positive aspects of the system development problem were: • Most problems were relatively independent in nature and non-interacting with other problems. • The system did not need to provide an exact diagnosis, just the likely problem area. Possible solutions • Apart from a CBR system, there were two other possible solution choices: • A standard database system - a DBMS would lack the rich structures needed for representing the problems. Also, such a system would be too rigid to allow users access to open-ended data. ICT619

41. Case Study (continued) • An expert system - this choice faced the following difficulties • The problem domain was not stable enough for experts to express solution heuristics with a high degree of confidence • The rule base would require frequent updating due to continuing changes in the problem domain • Due to the distributed nature of the expertise, the knowledge extraction process would be difficult • A case-based reasoning system • Required enough prototypical cases to cover the problem domain • This had to be done either by using the experts or from data gleaned from customer calls • One weakness of a CBR system is the response time as the size of the case base grows • However, it could deal with noisy or partial data, as missing attributes tend not to affect similarity computation too adversely ICT619

42. Case Study (continued) The solution implemented • The SMART system based on case-based reasoning and integrated with the call-logging system • The following year Compaq developed a second CBR system called QUICKSOURCE, which was also aimed at the customer apart from dealers and internal staff • Case bases for both systems were built using prototypical cases of previous problems • The structure of each case was as follows: • A description of the problem in English • A set of questions. The answers of the questions could be of the form yes/no, numeric, or an item from a list. Each question also has a match weight and a mismatch weight to reflect its importance • A set of actions (the solution part) ICT619

43. Case Study (continued) System operation • A customer support staff collects the problem information with a textual description • Simple problems are resolved by him/her straightaway • Unresolved problems cause the invocation of the CBR system. • The system performs an initial search for similar cases using keywords from the textual description • A list of matching cases along with their distance scores is displayed on the screen • A list of relevant questions pops up on the screen • As the user answers the questions, the list of cases and their scores change ICT619

44. Case Study (continued) System operation (continued) • If a perfectly matching case is found, the problem is solved • If a perfectly matching case is not found, the case is marked “unresolved” and passed on to the case builder experts. It is then reviewed and solved • The system gets refined with each new problem solved added to its case base • To keep the growth of the case base under control, new cases are only added if they are judged to be unique ICT619

45. Case Study (continued) Results Some of the benefits achieved by the CBR system were: • A larger number of problems could be resolved during the customer’s interaction with the support system than it was previously possible. • The success rate on test cases was 50% higher than that without the CBR system • Ready access to the case base meant, support staff was more likely to search for relevant information more actively instead of passing problems on to specialists • QUICKSOURCE resulted in 20% fewer calls to Compaq support centre from dealers and customers as only the harder problems got directed to it • The case base served as a valuable repository of product performance information for Compaq • The company became less susceptible to departures of experienced support staff ICT619

46. REFERENCES • Aamodt, A., & Plaza, E., Case-Based Reasoning: Foundational Issues, Methodological Variations, and System Approaches, AICom – Artificial Intelligence Communications, IOS Press, Vol.7:1, 1994, pp.39-59. • Allen, B. P., Case-Based Reasoning Business Applications, Communications of the ACM, Vol.37, No.3, March 1994, pp.40-42. • Bergman, R., Introduction to Case-Based Reasoning¸URL http://www.cbr-web.org/CBR-Web/cbrintro. • Dhar, V., & Stein, R.,”Solving Problems by Analogy” in Seven Methods for Transforming Corporate Data into Business Intelligence., Prentice Hall 1997, pp. 149-166, 236-243. ICT619