HLTHINFO 730 Healthcare Decision Support Systems Lecture 5: Production Rule Systems

1. HLTHINFO 730Healthcare Decision Support SystemsLecture 5: Production Rule Systems Lecturer: Prof Jim Warren

2. Expert Systems • Expert System – a system that is (in some sense) a synthetic expert • Generally for a very narrow ‘domain’ of knowledge • Reasons in a way that emulates a human expert • Uses • For safety and completeness – as a check, even though the user is an expert too • To extend the user – because they have less than ideal expertise themselves • To train or test the user – possibly working on canned data and/or with the system providing explanation of its recommendations

3. Rules and heuristics • Expert systems are most commonly built from rules • E.g., IF light is red, THEN stop • Rules may be more or less ‘certain’ • E.g., IF Temperature>37 THEN infection may be present • Confidence in a rule may be quantified as a probability • Heuristics are ‘rules of thumb’ • See http://www.phrases.org.uk/meanings/307000.html for origin of the phrase • Adequate, but not logically fool-proof, ways of reasoning based on experience • Heuristic knowledge may help the search for a solution • E.g., “allow extrapolation of therapeutic benefits from one drug to another if they are of the same physiologic class” - http://www.annals.org/cgi/content/full/124/1_Part_1/56

4. Knowledge Engineering (KE) • KE – developing (and also potentially maintaining) a knowledge-based system (KBS), especially an Expert System • A key aspect is Knowledge Acquisition (KA) • Getting the rules and heuristics (and their confidence levels) • Best to have the participation of a live expert (or preferably a panel of them) • Suboptimal to work from a textbook or printed guideline, esp. if the knowledge engineer has only a technical (not clinical) background • This can lead to a ‘bottleneck’ (if the expertise were easy to come by, we probably wouldn’t build the system!)

5. KA techniques • In addition to reviewing documents (e.g., guidelines)… • Interviewing • Just ask the expert how they solve problems in the domain; may first develop a number of reference cases to guide the conversation • Protocol analysis • Have the expert ‘talk aloud’ as they solve a problem (or have two experts talk to each other as they solve the problem) • Make a video (be sure audio is good!)

6. KA synthesis and verification • Once you’ve identified some initial concepts and rules, begin formalisation and review with experts • Process diagrams – make a flowchart of the process that’s followed for key decisions • Ontology formulation – collect all the concepts in the domain into a database (Protégé is a tool to support this) • Often a concept will have more than one term (word or phrase used to refer to the concept) • Create influence diagrams (show the relationship of factors to one another, to actions, and to outcomes)

7. Influence diagrams • Arrows indicate influence (generally positive correlation / increased probability) * * from http://www.cs.ru.nl/~peterl/aisb.pdf

8. Computedose Program flowcharts • Simple notation to show order of steps in a process • Use diamond to indicate a decision and include two outbound branches • Use rectangle with double sidebars to indicate details are defined elsewhere • E.g.,

9. ‘Production’ rules • A Production Rule System emulates human reasoning using a set of ‘productions’ • Productions have two parts • Sensory precondition (“IF” part) • Action (“THEN” part) • When the state of the ‘world’ matches the IF part, the production is fired, meaning the action is executed • The ‘world’ is the set of data values in the system’s working memory • For a clinical expert systems, this is usually data about a patient, which, ideally, has come from (and may go back to) an electronic medical record, or it may be entered interactively (or usually a little of each) • So production rules link facts (“IF” parts, also called antecedents) to conclusions (“THEN” parts, also called consequents)

10. MYCIN innovations • MYCIN • Developed at Stanford from 1972 to 1980 • Helped physicians diagnose a range of infectious blood diseases • Separated the methods of reasoning on productions (‘inference engine’) from the rules themselves (the ‘knowledge base) • Became the first expert systems shell when distributed as ‘empty MYCIN’ (EMYCIN) • Also provided explicit certainty factors (crude analog to probability), used backward chaining and provided an explanation facility

11. Example MYCIN rule IF the stain of the organism is gram negative     AND the morphology of the organism is rod         AND the aerobicity of the organism is anaerobic             THEN there is strongly suggestive evidence (0.8) that the class of the organism is Enterobacter iaceae. • This rule has three predicates (yes/no, or Boolean, values that determine if it should fire) • In this case each predicate involves the equality of a data field about a patient to a specific qualitative value(e.g., [stain of the organism] = ‘gram negative’) • Note that human expertise is still needed – e.g., to decide that the morphology of the organism is ‘rod’ (nonetheless to understand its vocabulary!) • Notice it produces a new fact (regarding [class of the organism]) • Note this is ‘symbolic reasoning’ – working with concepts as compared to numbers (it’s not like y = x1 + 4.6 x2)

12. Forward v. Backward chaining • Forward chaining is reasoning from facts to the conclusions resulting from those facts • E.g., if you see that it is raining before leaving from home (the fact) then you should decide to take an umbrella (the conclusion) • Backward chaining involves reasoning in reverse from a hypothesis - from a potential conclusion to be proved, to the facts that support the hypothesis • E.g., if you have not looked outside and someone enters with wet shoes and an umbrella, your hypothesis is that it is raining • In order to support this hypothesis you could ask the person if it was, in fact, raining • If the response is yes, then the hypothesis is proved true and becomes a fact

13. Backward chaining example • Suppose that the goal is to conclude the colour of my pet Fritz, given that he croaks and eats flies, and that the rule base contains the following four rules: • If X croaks and eats flies - Then X is a frog • If X chirps and sings - Then X is a canary • If X is a frog - Then X is green • If X is a canary - Then X is yellow • This rule base would be searched and the third and fourth rules would be selected, because their consequents (Then Fritz is green, Then Fritz is yellow) match the goal (to determine Fritz's colour). It is not yet known that Fritz is a frog, so both the antecedents (If Fritz is a frog, If Fritz is a canary) are added to the goal list. The rule base is again searched and this time the first two rules are selected, because their consequents (Then X is a frog or canary) match the new goals that were just added to the list. The antecedent (If Fritz croaks and eats flies) is known to be true and therefore it can be concluded that Fritz is a frog, and not a canary. The goal of determining Fritz's colour is now achieved (Fritz is green if he is a frog, and yellow if he is a canary, but since he croaks and eats flies, he is a frog, and, therefore, he is green).

14. Backward chaining in practice • Unlike the previous example, often we don’t have all the information at first to reach the goal • The system may ask questions interactively and then make more inferences • A long chain of reasoning may be abandoned if its certainty becomes too low • A practical system may ask a specific battery of questions up front just to get things started • Some rules, or parts of rules, may be there for heuristic purposes to smooth the interaction (e.g., to not look for smoking as a factor in patients under 10 years of age – not 100% valid, but possibly useful) • As production rule systems get more complex, their behavior can become very hard to predict and they become impractical to verify

15. Shells • There are some free expert systems shells nowadays (these things were hot in the 1980s, but are giveaways now!) • CLIPS (C Language Integrated Production System) developed by NASA • http://www.ghg.net/clips/CLIPS.html • Commonly used in university AI courses • Runs under Windows XP or MacOS X • e2gLite – an expert system as a Java applet (runs in your browser!) by expertise2go • http://www.expertise2go.com/webesie/e2gdoc/ • Just put e2glite.jar and a .kb file in a directory together, along with a .html file that invokes the applet, and you can run it locally via your browser

16. e2gLite example • A small .kb file (i.e., knowledge base) • We’ll do a hands-on tutorial with this system REM Test Knowledge Base (weather.kb) RULE [Is it going to rain?] If [precipitation] = "expected“ Then [the recommendation] = "wear a raincoat“ PROMPT [precipitation] MultChoice CF "According to the weather forecast, precipitation is:“ "expected“ "not expected“ GOAL [the recommendation]

17. Explanation Facility • Ability to explain its recommendations has always been considered part of an ‘expert system’ • Because it’s something a human expert is also expected to do • Useful for • Tutoring – the student can learn what rules applied to the case at hand • Use in practice – the doctor (or other expert user) can confirm that the concur with the reasoning (and possibly then translate it for the patient as appropriate) • Accuracy/safety/debugging – the user might see an error in the explanation • Readability • Unfortunately, an automatically generated explanation can be rather cryptic • Aided by use of good predicate names in the production rules • Provenance • Ideally, the system should allow links to supporting literature to support the rules themselves • And it’s important that we know who was involved in the knowledge engineering, and how the system has been tested

18. Testing • White box testing • Testing with an understanding of the way the system is put together • Design test cases that try out all the rules • Black box testing • Test without knowledge of (or sympathy for!) the structure of the system • Ideally done by people removed from the main KA/KE process • Obviously, for clinical expert systems, clinical people are the appropriate black box testers (this can be expensive)

19. Maintenance • An expert system can be dead, but it can never be complete • A non-trivial rule base cannot be proved correct • Must always have a feedback process for end-users readily to report issues for investigation • And medical knowledge doesn’t sit still • MYCIN is now an expert that’s 28 years out of date! • So production expert systems need a routine process of review, revision and re-distribution • Last step easier if the system’s web based, but at least users need a way to find out about the latest changes • And when you change something, you have to re-test everything

20. Some Medical Expert Systems

21. Some current clinical expert systems • DxPlain • Provides justification of why to consider any of various diseases for a set of findings • Knowledge base includes over 2200 diseases and 4900 findings (http://www.lcs.mgh.harvard.edu/projects/dxplain.html) • Slightly clunky slideshow available from http://dxplain.org/dxp/dxp.sdemo.pl?login=demslcshome • Clinical Knowledge Summaries • From the UK NHS - http://cks.library.nhs.uk/ • Organises advice on ‘scenarios’ (e.g., http://cks.library.nhs.uk/hypertension/view_whole_guidance#NodeIdn180741n180870n180871) • A compromise between an online guideline and a running expert system • PREDICT CVD/Diabetes • Developed by School of Population Health and Engima Publishing • Advises on management of CVD event risk

22. Further reading • History and applications of AI by George Luger - http://www.cs.unm.edu/~luger/ai-final/chapter1.html • Introduction to expert systems by expertise2go - http://www.expertise2go.com/webesie/tutorials/ESIntro/ • Clem McDonald, “Medical Heuristics: The Silent Adjudicators of Clinical Practice,” Annals of Internal Medicine 124(1): 56-62, 1996 (from http://www.annals.org/cgi/content/full/124/1_Part_1/56)