INFO-I530 (Foundation of Health Informatics)

1. INFO-I530 (Foundation of Health Informatics) Medical Data and Decision-Making Lecture #4

2. Lecture in a Nutshell • Medical Data and Semiology • Introduction • The Nature of Medical Data • Interpreting Medical Data • Quantitative Semiology • Medical Reasoning and Decision Making • Introduction • Reasoning • Steps in Medical Decision • Medical Judgment • Uncertainty and Judgment Bias • Probability Theory and Decision Analysis • Symbolic Reasoning and Expert Systems • Learning

3. Medical Data and Semiology

4. Introduction • A symptom or a clinical or radiological sign is the result of a complex decision-making process. • Medical information therefore only exists in an interpretative framework that must constantly be updated to avoid diagnostic or therapeutic errors. • Semiology is the part of medicine that studies the signs of diseases. • The physician's hypothesis guides data collection, and the selection of information judged as "useful" changes with his or her experience. • Subjectivity plays a predominant role in medicine  missing data in EHR • Important to evaluate the quality of medical data and to be able to appreciate its informational value.

5. The Nature of Medical Data • Medical Data Types • Min four elements of medical data: • the patient's name (e.g., M. Jones) • the attribute or the parameter in question (e.g., age) • the value for that parameter (e.g., 40) • the time of the observation (e.g., 16 June 1995 at 10:52 A.M.) • Quantitative variables • Precision depends on the measurement • Context of the measurement (BP in standing versus laid) • Qualitative variables • May have several values • May be binary or boolean qualitative variable • The Variability of Medical Data • The rate at which medical observations can be successfully reproduced depends on the methods used for measurement (analytic variability), the observers (intra- and inter observer variability), and the subject being observed (intra- and inter individual variability).

6. The Nature of Medical Data cont. intra- inter- intra- inter- intra- inter- instrumental observer individual Analytical or metrological biological total variability Sources of variability in medical data

7. The Nature of Medical Data cont. • Precision refers to the fidelity of the measurement; if the measurement is repeated on the same subject, the same result will be obtained. • The fidelity of the measurement of clinical information is difficult to establish (the view points of physicians on identical patient cases) • Accuracy refers to the tendency of measured values to be symmetrically grouped around the variable's true value. The dispersion of measurements is greater around the central value for imprecision. In the case of inaccuracy, measurements are grouped around a value that is not the one sought.

8. Interpreting Medical Data • Cognitive Processes: • Abduction starts from a suggested hypothesis that explains a given phenomenon and proceeds with successive verifications to confirm or refute the hypothesis. • It explains in part the intra- and inter observer variability. • Interpreting Numbers and Words: • The absence of a standardized medical vocabulary based on clear definitions makes medical observations fuzzy. • The comprehension of numerical terms is also subject to differing interpretations. • Interpreting Associations • The abductive process and diagnostic judgment are based on the frequency of associations between a diagnosis and a particular trait of a description. • Common events are much more easily perceived than rare ones.

9. Interpreting Medical Data cont. Qualifiers and estimated probabilities

10. Quantitative Semiology • Semiology is the study of signs. Quantitative semiology uses tools to evaluate the informational value of medical signs and data. • The Diagnostic Value of a Test: A sign S has a diagnostic value if it can separate sick patients from healthy ones (or disease A from disease B); in other words, if its frequency is significantly different between healthy patients and sick patients (or those who have contracted diseases A and B). The probability P of having disease D if the sign S is present, written P(D|S), is greater than the probability of having D, or the anterior probability of D, written P(D). Modification of the probability of a patient being sick by searching for a sign

11. Quantitative Semiology cont. • The presence of a sign does not necessarily signify the presence of a disease, and the absence of a sign is not synonymous with the absence of the disease. • The contingency table (based on a reference test called the gold standard): • true positives (TP), where both the sign and the disease are present, • false positives (FP), where the sign is present but the disease is absent, • false negatives (FN), who do not present the sign but do have the disease, • true negatives (TN) have neither the sign nor the disease. The situations as a function of results from a test

12. Quantitative Semiology cont. • The same reasoning holds true for known quantitative values if we opt to define a normality threshold. • The threshold is determined by statistical methods, according to the distribution of the parameter in the population under study. These statistical fluctuations show that the repetition of doses with healthy patients may give abnormal results. Probability of finding at least one abnormal result for a person in good health Normality threshold of a test

13. Quantitative Semiology cont. Discriminating ability of two tests. Test 1 is more discriminating than test 2 because there is less of an overlap between the distributions of the parameter in the healthy population and the diseased population.

14. Quantitative Semiology cont. • Sensitivity and Specificity of a SignSensitivity and specificity are two indicators of the informational value of a medical sign. • Binary Qualitative Sign: • For a binary qualitative sign (present or absent of the disease), the sensitivity and specificity may be estimated from the contingency table. • The sensitivity (Se) or true positive rate (TPR) is defined as the proportion of patients who show the sign (S). Sensitivity, measured in a sample of patients, is an estimation of the probability of the presence of the sign in those carrying the disease (theoretical sensitivity), written P(S|D). • The specificity (Sp) or true negative rate (TNR) is defined as the proportion of healthy patients who do not have the sign. Specificity, measured in a sample of patients who do not have the disease (D), is an estimation of the probability of the absence of the sign in the healthy population (theoretical specificity), written p(S|M).

15. Quantitative Semiology cont. • An ideal sign would have a sensitivity and a specificity of 1: • A sign whose specificity is equal to 1 is called pathognomonic, such as Koplick's sign for measles. • A good diagnostic test must have high sensitivity and specificity. The following formula represents the likelihood ratio (LR): • Sensitivity, calculated on a sample of patients who have the disease D, and specificity, calculated on a sample of patients who do not have the disease D, are independent of the prevalence of D, which expresses the probability P(D) of the disease in the population.

16. Quantitative Semiology cont. • Continuous Quantitative Variables: • Any improvement in specificity comes at the expense of sensitivity, and vice versa. • For each value a in the selected threshold, we can calculate a value for sensitivity and specificity and find the threshold with the best sensitivity / specificity couple. Influence of the sensitivity and specificity threshold of SGOT for the diagnosis of MI (a) Se=52.1% & Sp=91.3% (b) Se=45.8% & Sp=93.5%

17. Quantitative Semiology cont. • The informational value of a test compared to a diagnostic may be represented graphically by the curve of [Se, 1- Sp] pairs, obtained by varying the normality threshold. This curve is called the ROC curve (receiver-operating characteristic). • Comparing the ROC curves of different tests lets us appreciate the informational value. The test with the best discriminating ability is the one that corresponds to the highest curve A format for ranking tests. Test A is better than test B. ROC curve of a test

18. Quantitative Semiology cont. • Consideration for the Cost of Tests: • The method described above does not take into consideration the respective costs of the two possible diagnostic errors: incorrectly ignoring the disease D or incorrectly diagnosing D. • The cost must be considered from two perspectives: • The health cost is measured in terms of mortality or morbidity. • The financial cost includes the cost of treatment and the costs incurred by the death or the invalidity of the patient. • Measuring the Predictive Value of a Sign • When making a medical decision, it is useful to consider the posterior probability of the disease once the result of the test is known. • P(D) or prevalence of the diseases is now included in the calculations. Predictive values of a sign

19. Quantitative Semiology cont. • Knowing that P(D^S)=P(D).P(S|D), if P(D) is the prevalence of the disease in the general population, we can estimate the positive and negative predictive values of a sign using Bayes' formula. • The probability P(D|S) that an individual who has the sign has contracted the disease is called the positive predictive value (PPV) or the diagnostic value: • The diagnostic value is equal to: • 1 if Sp = 1 , that is, if the sign is pathognomonic (look at the formula) • P(D) if Se = 1 - Sp, that is, if the sign has no diagnostic value for disease D (if the probability of having the disease does not change with the presence or the absence of the sign). • If Se(A) > Se(B) and Sp(A) > Sp(B), then the diagnostic value of A > B. • The prevalence is often difficult to determine in diagnostic situations where the patient cannot be considered as having been selected at random.

20. Quantitative Semiology cont. Effect of prevalence in the population and the sample studied

21. Quantitative Semiology cont. Relation between the prevalence and the positive predictive value(sensitivity and specificity at 90%)

22. Medical Reasoning and Decision Making

23. Introduction • Decision-making is the physician's essential activity. • There are too many possible hypotheses to consider each one individually. • Only fragmentary knowledge of the consequences of each decision is available. • Medical decisions are made under uncertainty. They present a judgment that is generally a preference for a solution or a treatment that has been deemed optimal. • Computers may assist in medical decision-making and improve the quality of diagnosis or the efficiency of therapy. • Creating systems to assist in medical decision-making requires considerable thought in order to formalize the problem and the possible solutions.

24. Reasoning • DeductionDeductive reasoning is based on the principles of logical implication. It allows us to infer conclusions, the degree of truth of which is only a function of the degree of truth of the premises. The results of logical inference may be used as a premise for further deductions. Thus, if A implies B and if B implies C, then, via transitivity, A implies C. • InductionInductive reasoning makes generalizations based on specific examples to formulate general rules. Inductive reasoning produces inferences that are valid to a certain degree of credibility or probability. • AbductionAbductive reasoning is referred to as the scientific method. It attempts to establish links between observations such as cause and effect. Hypotheses may help formulate a rule to establish a link between the preceding and subsequent facts (e.g., a diagnosis hypothesis), or they may concern the formulation of a new rule (e.g., a scientific discovery).

25. Reasoning cont. Different types of reasoning

26. Steps in Medical Decision • Identify the ProblemDiagnostic decisions begin with the primary interpretation of clinical data. Abductive reasoning is used. • Structure the ProblemSeveral interpretations of the same data or parts of that data are possible. Diagnostic hypotheses are formulated by structuring the information. Reasoning may be deductive (e.g., for a pathognomonic sign), inductive (e.g., diagnosing a transmissive disease in a population of subjects at risk), or abductive. • Choose the SolutionStarting from a number of working hypotheses, the expected signs and symptoms may be obtained by deduction and, if necessary, by the complementary examinations required to obtain them. By induction and/or abduction, the physician may eliminate hypotheses that do not correspond to the observations. The results of complementary examinations may help to reduce the uncertainty over the clinical situation and eliminate hypotheses or solicit new ones.

27. Steps in Medical Decision cont. Structure of the decision-making process

28. Medical Judgment • Two clinical technicians confronted with the same situation may use very different decision-making strategies. • Judgments are based on criteria (A, B, C, etc.) and the relationships that exist on the one hand, between the uncertain situation and the selected criteria and, on the other hand, between the judgment and the criteria. The Brunswick lens model

29. Uncertainty and Judgment Bias • Medical judgment may be hindered by the cognitive bias that appears throughout the decision-making process: • When acquiring data, the order in which information is provided is a possible source of error, since the first information supplied may dominate the rest. • Human judgment does not always fully account for data reliability (data sources are considered as being perfectly reliable). • Collecting information is based on expectations depending on the context as extrapolated by the decision-maker. • Conservatism is the difficulty of revising opinions, the tendency to favor a particular interpretation, and to rationalize or ignore contrary evidence. • The inconsistency of any judgment represents the contradiction that arises from giving different opinions on identical cases. • Justifiability makes us inclined to apply a rule if we can find a reason to justify it, even if it is not appropriate.

30. Probability Theory and Decision Analysis • All of the causes for uncertainty discussed previously illustrate the need for a precise scientific framework to represent and quantify uncertainty. • Probabilities are normally the axiomatic bases of decision theory because they measure the credibility of uncertain propositions. • Bayes' theorem may be used to evaluate the probabilities of different diagnostic hypotheses. • Unfortunately, the difficulties involved in estimating the a priori probabilities and conditional probabilities limit the practical use of the Bayes method. • In Bayes' theorem the following hypotheses should be made: • Diseases are mutually exclusive (a disease may only present one of the Di diagnoses). • The different signs and symptoms involved in the diagnosis are independent.

31. Probability Theory and Decision Analysis cont. Diagnosis Illustration of Bayes' theorem for diagnosis. Five diagnostics Di are possible a priori. Knowledge of the sign S modifies the values of the probabilities for each Di.

32. Probability Theory and Decision Analysis cont. Treatment • Therapy may be curative (e.g., antibiotic treatment or removing a tumor), preventive (e.g., eliminating or reducing a risk factor), palliative (e.g., treating pain) or supportive (e.g., psychological assistance). • Nonetheless, in some cases these benefits may be evaluated using information supplied by controlled therapeutic tests, whose results may be accessed by querying knowledge bases such as the COCHRANE base. • If Pi represents the risk that a certain event E arises in an intervention group, and Pc represents the risk of that same condition in a control group (Pc is called the baseline risk), the ratio of Pi/Pc is called the relative risk (RR) associated with the test. • RR expresses a measure of the reduction of the risk Pi in the intervention group compared to the baseline risk Pi calculated in the control group. • Controlled trials normally provide this measurement.

33. Probability Theory and Decision Analysis cont. Decision • Decision trees are useful for representing logical sequences and for structuring clinical decision problems. Two type of nodes: Decision nodes, which are under the control of the decision-maker, are represented by squares. Contingency nodes, represented by circles, are not controlled by the decision-maker. • Utilities are quantities used to assign degrees of preference to different solutions and to select the optimal one the maximum expected utility. A utility may be a probability of survival, a financial risk, or any other function that involves objective or subjective criteria. The strategy offering the highest utility is chosen. Decision node and contingency node

34. Probability Theory and Decision Analysis cont. Calculating the utility of a strategy. The preferred strategy is the one that offers the highest utility.

35. Probability Theory and Decision Analysis cont. Decision tree: Utilities have been set subjectively on a scale of 0 to 1

36. Symbolic Reasoning and Expert Systems • Research has been directed toward solving problems where no algorithmic solution exists. This method has led researchers to suggest means for representing and using symbolic and declarative knowledge that enriches and complements numerical and algorithmic. The methods and techniques of artificial intelligence (AI) have provided a privileged framework for cognitive research and have led to the development of expert systems. • Two principal knowledge models: • The empirical model concerns the associations between diseases and symptoms. They may be provided by an expert or derived from the analysis of a database. • Models based on physiological and pathological knowledge. This type of knowledge, when available, lets us employ reasoning that describes the mechanisms involved in morbid processes. • Production Rules: These indicate that if the conditions or premises are met, the conclusions may be affirmed. Credibility factor (CF) can be applied for each rule whichvaries between -1 and 1.Arden syntax is a proposed standard for the representation of production rules. MYCIN expert system

37. Symbolic Reasoning and Expert Systems cont. Combination of credibility factors (CF) in MYCIN. x. y and z are the CF of the conclusions of rules supplied by the expert.

38. Symbolic Reasoning and Expert Systems cont. AND/OR tree created by all the rules of MYCIN. AND nodes are marked with a star. The Si are the signs to be sought in the patient. Outlined numbers rate the credibility of the user for each sign. The CF supplied by the expert (CF of rules) are framed

39. Learning • Learning is of interest in order to develop decision-support systems: "a system capable of modifying its future answers as a function of past experience". • Programmed learning: This corresponds to all current programs where a code controls what must be done. This is the case for robots executing prerecorded commands. • Learning "by heart": All situations have been memorized. The system must find and apply the behavior corresponding to the situation. • Statistical learning: The system only retains a classification of the situations based on the greatest number of cases. • Learning by example: The system must be able to generalize. It must abstract the knowledge required to solve similar cases from examples. It uses inductive reasoning. • Learning by discovery: The system uses inductive and/or abductive methods. It must be able to create new hypotheses and new concepts.

40. Summary • Medical Data and Semiology • Introduction • The Nature of Medical Data • Interpreting Medical Data • Quantitative Semiology • Medical Reasoning and Decision Making • Introduction • Reasoning • Steps in Medical Decision • Medical Judgment • Uncertainty and Judgment Bias • Probability Theory and Decision Analysis • Symbolic Reasoning and Expert Systems • Learning