1 / 17

Case Based Reasoning

Case Based Reasoning . Lecture 6: CBR Acquisition, Testing and Maintenance of Case-Bases . Outline. Knowledge Containers Knowledge Acquisition Testing Maintenance Reading. CBR Knowledge Containers. Representation Language representation of cases

ricky
Download Presentation

Case Based Reasoning

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Case Based Reasoning Lecture 6: CBR Acquisition, Testing and Maintenance of Case-Bases

  2. Outline • Knowledge Containers • Knowledge Acquisition • Testing • Maintenance • Reading

  3. CBR Knowledge Containers • Representation Language • representation of cases • features and values of problem/solution • Cases • lesson to be learned • context in which lesson applies

  4. CBR Knowledge Containers • Retrieval Knowledge • features used to index cases • relative importance of features in similarity • Adaptation Knowledge • rules to capture when and how to adapt • alteration to apply • Other knowledge containers now proposed • maintenance, explanation, …

  5. CBR Knowledge Acquisition • Acquiring cases may be simpler than rules • but does require care and planning • Can be obtained from database, archive, . . . • Opportunity for multiple knowledge sources • several experts • Retrieval and Adaptation Knowledge • Knowledge acquisition bottleneck is back!

  6. Learning Retrieval and Adaptation Knowledge AstraZeneca Case-base Index Knowledge Similarity Knowledge Database of previous formulations Case-Based Design System Adaptation Knowledge Adaptations

  7. Characterising a “good” case base • A good case base should contain • a representative set of cases • a well-distributed set of cases • A CBR system is characterised by its competence • problem solving ability

  8. Acquiring Case Features • Homogenous case bases • cases have the same attributes • e.g., every flat is described by the same attributes • identify case features by considering cases and/or challenging domain experts • Heterogeneous case bases • cases may have different attributes • e.g., printer diagnostic case base – each printer can have a different problem • may be fruitless to imagine all possible features, instead rely on historical records • Effect on similarity?

  9. Acquiring Representative Cases • CBR systems have an advantage over rule-based systems • work for weak or ill-defined theory domains • can aquire new cases during use • can be delivered with incomplete case bases • But if a case base is underpopulated • users may reject the system because it fails to deal adequately with a majority of problems • e.g., in a help-desk application • majority of cases have to deal with most common problems

  10. Testing CBR Performance • Obtain a number of representative test cases • Not in case-base • Use each of your test cases as target cases • Evaluate the performance of the system: • Did the system retrieve useful cases? (accuracy) • Was the retrieval time acceptable? • If relevant, was the adaptation successful? • Record the results • Repeat with different cases • membership, size, …

  11. Testing CBR Performance • Problem-solving accuracy • “Leave one out” testing • a case is removed as query • case-base has query case missing • Does reused retrieved solution match query solution? • Repeat for every case in case-base Case-base

  12. Testing CBR Performance • Retrieval accuracy • “Leave one in” testing • if a case is used as target • Is the retrieved case identical? • Useful for help desk applications Case-base Same case

  13. Testing Case Coverage • Are the possible values in the ranges of all attributes well represented? • If the distribution is uneven • then you have outlying cases. • try to remove them • obtain more cases to improve the coverage • Are the different combinations of attribute values well represented? • do gaps correspond to impossible combinations?

  14. Testing Case Coverage • Boundaries • e.g. clinical trials show that when a person weighs 100 lbs, the correct drug dosage is 1 mg • 6 evenly distributed cases in the case base • If a man weighting 190 lbs presents himself, the nearest matching case would be the 200 lbs case • the recommended dosage after a simple linear adaptation would be 1.9 mg Dose mg Weight lbs

  15. Testing Case Coverage • Boundaries • Clinical trials are extended to cover young children • A 30 lbs baby nearly died from a 0.3 mg dose • A sudden change (feature shift) in the recommended dosage at low weights • More cases need to be clustered around the point of feature shift Dose mg New case 30 lbs Weight lbs

  16. Case-Base Maintenance • Case redundancy • duplicates or unnecessary near neighbours in case-base may evolve during retain • redundant cases may not harm decision making, but can slow down the system • consult domain experts • are potentially redundant cases harming performance? • or may they be useful in future? • Case utilisation statistics • how many times is each case retrieved? • if case never retrieved over a period of time • may be redundant • if case retrieved very frequently • may indicate poor case coverage

  17. Summary • Knowledge containers • Not just case knowledge • Knowledge acquisition • Cases and others • Testing CBR system • Coverage – accuracy on unseen problems • Performance - efficiency • Maintenance

More Related