Case Base Maintenance (CBM)

1. Case Base Maintenance (CBM) Fabiana Prabhakar CSE 435 November 6, 2006

2. Introduction • The growing use of CBR applications has brought with it increased awareness of the importance of case-base maintenance (CBM). • Large scale CBR systems are becoming more prevalent, with case library sizes ranging from thousands to millions of cases. • Large case-bases raises concern about the utility problem for case retrieval, underlining the potential need to control case-base growth through case deletion policies.

3. Definition • CBM is the process of refining a CBR system’s case base to improve the system’s performance.

4. Standard CBR learning • The system always add each new case to the case base. • Domain expert adds a variable number of new cases. • Indexing of the cases.

5. Knowledge-based Systems Utility Problem • The cost associated with searching for relevant knowledge outweighs the benefit of applying the knowledge.

6. Traditional Deletion Policies • A simple deletion policy is random deletion. According to this policy a random item is removed from the knowledgebase once the knowledge-base size exceeds some predefined limit. • Minton’s utility metric [Minton, 1990]. Chooses a knowledge item for deletion based on an estimate of its performance benefits. Utility=(ApplicationFreq*AverageSavings)-MatchCost

7. Remembering to Forget • Competency Preserving Case Deletion Policy for CBR Systems (Smyth and Keane, 1995)

8. Coverage and Reachability • Coverage of a case is the set of target problems that can be solved by such case. • Reachability of a target problem is the set of cases that can be used to provide a solution for the target problem.

9. Case Competence Categories • Pivotal Cases: its deletion directly reduces the competence of the system. A case is pivotal if it is reachable by no other case but itself. • Auxiliary Cases: do not effect competence at all. A case is auxiliary case if the coverage it provides is subsumed by the coverage of one of its reachable cases.

10. Case Competence Categories (Cont.) • Spanning Cases: do not directly affect the competence. Their coverage spaces link regions of the problem space that are independently covered by other cases. If cases from this linked regions are deleted, then the spanning case might be necessary. • Support Cases: a special class of spanning cases. They exist in groups. The deletion of the group is analogous to removing a pivotal case.

11. Case Competence Categories (Cont.)

12. Case Competence Categories (Cont.) • The case categories provide a means of ordering cases for deletion in terms of their competence contributions. • Auxiliary cases (they make no direct contribution to competence) • Support cases • Spanning cases • Pivotal cases.

13. Modeling Case Competence • Competence categories are computed at start-up. • During future problem solving as cases are learned, the case categories must be updated: • Re-compute the coverage and reachability sets of the appropriate cases; • Adjust the categories accordingly.

14. The Footprint Deletion Policy • Ideally a deletion policy should work to remove irrelevant cases guiding the case-base toward an optimal configuration of cases. • Competence Footprint is this optimal case-base. It provides the same competence of the entire case-base but with fewer cases.

15. The Footprint Deletion Algorithm DeleteCase(Cases): If there are auxiliary cases then SelectAuxiliary(AuxiliaryCases) ElseIf there are support cases then With the largest support group SelectSupport(SuportGroup) ElseIf there are spanning cases then SelectSpanning(SpanningCases) ElseIf there are pivotal cases then SelectPivot(PivotalCases) Endif

16. The Footprint Utility Deletion Policy • Combine Footprint and Utility Deletion: • Minton’s utility metric – An item is selected based on an estimate of its performance benefits. Utility = (ApplicationFreq * AverageSavings) – MatchCost • The footprint method is used to select candidates for deletion. If there is only one such candidate then it is deleted. • If, however, there a number of candidates, then rather than selecting the one with the least coverage or largest reachability set, the candidate with the lowest utility is chosen. • In other words the utility metric is used within the SelectPivot, SelectSpanning, SelectSupport, and SelectAuxiliary procedures.

17. Further Applications • The competence modeling approach may be used during the initial case acquisition stage of system development. It is often undesirable to store every available case in the initial case-base. • Utility Problem; • Irrelevant cases may introduce noise into the retrieval stage and lead to the selection of suboptimal cases or difficulties in tuning the similarity metric. • The competence modeling approach may be used during the authoring process.

18. CBR systems Authoring Process • Case base authoring can be a long, difficult, and tedious process, and the only advice given to the author is often of the “choose representative cases” variety. • This can ultimately lead to the development of poor case bases, which offer limited coverage of the target problem space, and which include significant redundancy.

19. CASCADE (Case Authoring Support & Development Environment) • Keeps the knowledge engineer informed about how case authoring is progressing, and in particular, how case base competence is evolving. • Extends the case competency model proposed by Smyth and Keane.

20. Competence Groups • A competence group is a collection of related cases. • The key idea underlying the definition of a competence group is that of shared coverage. Two cases exhibit shared coverage if their coverage or reachability sets overlap.

21. The Evolution of Competence • In general as cases are added to the case base one of four things can happen: • New groups are created; • Existing competence groups grow in size and coverage; • A number of existing groups merge to form a new ‘super’ group; • Existing groups can grow in size but without increasing coverage. • Conversely, as cases are deleted, groups may disappear altogether, or they may split into smaller ‘sub’ groups.

22. The Competence Visualization Tool

23. Competence Regions

24. The Competence Visualization Tool – Examples

25. The Competence Visualization Tool – Examples (Cont.)

26. Conclusion • Experience with the growing number of large-scale CBR systems has led to increasing recognition of the importance of case-base maintenance. • Multiple researches have addressed pieces of the CBM problem, considering such issues as maintaining consistency and controlling case-base growth. • The authoring process can be improved in order to avoid the development of poor case bases.

27. References • Smyth B., and Keane, M. “Remembering to forget: A competence-preserving case deletion policy for case-based reasoning systems”, in Proc. of the 14th International Joint Conf. on Artificial Intelligence, Montreal, Morgan Kauffmann, Canada, 1995, pp. 377-382. • D.B. Leak and D.C. Wilson “Categorizing case-based maintenance: Dimensions and directions”, in Proc. of the 4th European Workshop on Case-Base Reasoning, Dublin, Ireland, Springer Verleg: 1998, pp. 196-207. • McKenna, E. & Smyth, B. “An Interactive Visualisation Tool for Case-Based Reasoners”, Journal of Applied Intelligence: Special Issue on Interactive Case-Based Reasoning, 2000