Yuval Shahar, M.D., Ph.D. Medical Informatics Research Center

1. Distributed Knowledge-Based Abstraction, Visualization, and Exploration of Time-Oriented Clinical Data Yuval Shahar, M.D., Ph.D. Medical Informatics Research Center Ben Gurion University, Beer Sheva, Israel & Stanford University, CA, USA

2. Many medical tasks, especially those involving chronic patients, require extraction of clinically meaningful concepts from multiple sources of raw, longitudinal, time-oriented data Example: “Modify the standard dose of the drug, if during treatment, the patient experiences a second episodeof moderate anemia that has persisted for at least two weeks” Examples of clinical tasks: Diagnosis Searching for “a gradual increase of fasting blood-glucose level” Therapy Following a treatment plan based on a clinical guideline Quality assessment Comparing observed treatments with those recommended by a guideline Research Detection of hidden dependencies over time between clinical parameters The Need for Intelligent Integration of Multiple Time-Oriented Clinical Data

3. Clinical databases store raw, time-stampeddata Care providers and decision-support applications reason about patients in terms of abstract,clinically meaningful concepts, typically over significant time periods A system that automatically answers queries or detects patterns regarding either raw clinical data or concepts derivable from them over time, is crucial for effectively supporting multiple clinical tasks The Need for Intelligent Mediation:The Gap Between Raw Clinical Data and Clinically Meaningful Concepts

4. The Temporal-Abstraction Task • Input: time-stamped clinical data and relevant events (interventions) • Output: interval-based abstractions • Identifies past and present trends and states • Supports decisions based on temporal patterns, such as: “modify therapy if the patient has a secondepisode of Grade II bone-marrow toxicity lasting more than 3 weeks” • Focuses on interpretation, rather than on forecasting

5. BMT ( ) ( ) ² Temporal Abstraction:The Bone-Marrow Transplantation Domain PAZ protocol Expected CGVHD . M[0] M[1] M[2] M[3] M[1] M[0] Granu- Platelet locyte counts ² ² ² ² counts ² ² ² • • • ² ² • ² ² ² ² • ² ² ² • • • • • • • • 150K • ² ² • 2000 • • 100K • • 1000 400 0 50 100 200 Time (days) M[i]= Myelotoxicity (bone-marrow toxicity) of severity Grade i

6. The Bone-Marrow Transplantation Example, Revisited

7. Uses of Temporal Abstractions • Therapy planning and monitoring (e.g., to support guideline-based care) • Creating high-level summaries of time-oriented medical records • Supporting explanation modules for a medical DSS • Representing goals of therapy guidelines for quality assurance at runtime and quality assessment retrospectively; guideline intentions regarding both the (care provider) process and (patient) outcomes can be captured as temporal patterns to be achieved or avoided • Visualization and exploration of time-oriented clinical data: the KNAVE project

8. The Temporal-Abstraction Ontology • Events (interventions) (e.g., insulin therapy) - part-of, is-arelations • Parameters (measured raw data and derived concepts) (e.g., hemoglobin values; anemia levels) - abstracted-into, is-arelations • Patterns (e.g., crescendo angina; chronic GVHD) - component-of, is-a relations • Abstraction goals (user views)(e.g., therapy of diabetes) - is-arelations • Interpretation contexts (effect of regular insulin) - subcontext, is-arelations • Interpretation contexts are induced by all other entities

9. Temporal-Abstraction Output Types • State abstractions (LOW, HIGH) • Gradient abstractions (INC, DEC) • Rate Abstractions (SLOW, FAST) • Pattern Abstractions (CRESCENDO) - Linear patterns - Periodic patterns

10. Temporal-Abstraction Knowledge Types • Structural(e.g., part-of, is-a relations) - mainly declarative/relational • Classification (e.g., value ranges; patterns) - mainly functional • Temporal-semantic (e.g., “concatenable”property) - mainly logical • Temporal-dynamic (e.g., interpolation functions) - mainly probabilistic

11. Dynamic Induction of Contexts ee ss es se

12. Induction of Interpretation Contexts

13. Context intervals serve as a frame of reference for interpretation of clinical data; abstractions are meaningful only in a particular clinical context Context intervals focus and limit the computations to only those relevant to a particular context Contexts enable the use of context-specific medical knowledge contexts support maintenance of several concurrent views (or interpretations) of the data Facilitate maintenance of clinical knowledge The same context-forming entity (e.g., a hepatitis episode) can induce several clinical contexts The same context (e.g., “a chemotherapy effect”) might be induced by several entities (e.g., multiple medication types) The Benefits of Interpretation Contexts

14. Local and Global Persistence Functions t j j Bel ( j ) 2 1 I I 2 1 1 j th 0 Time

15. Temperature Hemoglobin Level Fever Fever Fever Fever Fever Anemia Anemia Anemia Anemia Abstraction of Periodic Patterns Periodic Pattern Linear Component Linear Component Linear Component Linear Component Week 1 Week 3 Week 2

16. The RÉSUMÉ System Architecture . Temporal-abstraction mechanisms Domain knowledge base Temporal fact base E v e n t s Event ontology C o n t e x t s Context ontology A b s t r a c t e d i n t e r v a l s Parameter ontology + • P r i m i t i v e d a t a External patient database Events • • • Primitive data + + +

17. Medical domains: Guideline-based care AIDS therapy Oncology Monitoring of children’s growth Therapy of insulin-dependent diabetes patients Non-medical domains: Evaluation of traffic-controllers actions summarization of meteorological data Test Domains for the RÉSUMÉ System

18. Acquisition of Temporal-Abstraction Knowledge

19. Editing The Ontology Using Protégé Tools

20. KNAVE=Knowledge-Based Navigation of Abstractions for Visualization and Explanation Interactive queries regarding both raw data and multiple levels of time-oriented abstractions derivable from these data Visualization and manipulation of query results Dynamic exploration of the results using the domain’s temporal-abstraction ontology The semantics of all operators do not depend on any specific domain, but the interface uses each domain’s ontology to compute and display specific terms and explore their relations KNAVE accesses the data through the IDAN temporal-abstraction mediator, which uses the ALMA system, a constraint-based re-implementation of RESUME, for temporal-reasoning Knowledge-Based Visualization andExploration of Time-Oriented Data:The KNAVE-I and KNAVE-II Projects(Shahar and Cheng, 1999, 2000; Shahar et al., 2003)

21. The IDAN Temporal-Abstraction Mediator(Boaz and Shahar, 2003) Knowledge Knowledge - Service acquisition tool Medical Expert Temporal- Standard Medical KNAVE - II Abstraction Vocab ularies Service Controller Clinical User Data Access Temporal - Service Abstraction Service (ALMA)

22. The KNAVE-II Browsing and Exploration Interface [Shahar et al., AIM 2006] Overall pattern Intermediate abstractions Medical knowledge browser Raw clinical data Concept search

23. Moving Data Panels Around

24. Global Temporal-Granule Zoom (I)

25. Global Temporal-Granule Zoom (II)

26. Global Calendar-Based Zoom

27. Global Content-Based Zoom (I)

28. Global Content-Based Zoom (II)

29. Local Time-Sensitive Zoom

30. Exploration Operators • Motion across semantic links in the domain’s knowledge base by using the semantic explorer; in particular, relations such as: - part-of - is-a - abstracted-from - subcontext • Motion across abstraction types: state, gradient, rate, pattern • Application of aggregation operators such as mean and distribution • Dynamic change of temporal-granularity (e.g., days, months) • Explanation by context-sensitive display of relevant knowledge • “What- if” queries allow hypothetical assertion or retraction of data and examination of resultant patterns

31. Semantic Exploration of Temporal Abstractions

32. Explanation: A Classification Function

33. Explanation: A Persistence Function

34. Eight clinicians with varying medical/computer use backgrounds A second study used six additional clinicians and more difficult queries Each user was given a 15 minute demonstration of the interface and two warm-up queries to answer The evaluation used an online database of more than 1000 bone-marrow transplantation patients followed for 2 to 4 years Each user was asked to answer 10 queries common in oncology protocols, about individual patients, at increasing difficulty levels A cross-over study design compared the KNAVE-II module versus two existing methods (in the 2nd study, control-group users chose which one): Paper charts An electronic spreadsheet (ESS) Measures: Quantitative: time to answer and accuracy of responses Qualitative:the Standard Usability Score (SUS) and comparative ranking Functionality and Usability Evaluation of KNAVE-II(Palo Alto Veterans Administration Health Care System)

35. Direct Ranking comparison: KNAVE-II ranked first in preference by all users Detailed Usability Scores: The Standard Usability Scale (SUS) mean scores: KNAVE-II 69, ESS 48, Paper 46 (P=0.006) (more than 50 is user friendly) Time to answer: In the first study, users were significantly faster using KNAVE-II as the level of difficulty increased, up to a mean of 93 seconds difference versus paper, and 27 seconds versus Excel, for the hardest query (p = 0.0006) The second evaluation, using more difficult queries and more advanced features of KNAVE-II, emphasized the differences even further: The comparison with Excel showed a similar trend for moderately difficult queries (P=0.007) and for hard queries (p=0.002); on the average, study participants answered each of the two hardest queries 277 seconds faster using KNAVE than when using the ESS Correctness: In the first study, using KNAVE-II significantly enhanced correctness versus using paper, especially as level of difficulty increased (P=0.01) In the second study, 91.6% (110/120) of all of the questions asked within queries of all levels produced correct answers using KNAVE-II, versus only 57.5% (69/120) using the ESS(p<0.0001); the correctness scores for KNAVE-II versus the ESS were significantly higherfor all queries The KNAVE-II Evaluation Results(Martins et al., MEDINFO 2004; AIM 2008)

36. Supports, among other types,aggregationqueries: Who [Patients] Query Find allmalepatients who wereolder than 50and weretreated by Dr. Johnson; and, starting within thetwo weeks following a BMTprocedure, had aone weekepisode ofWBC_Statewhose value was lower than“Normal” Show all patients whoseHGB_Statecan be interpreted as “moderateanemia” during more than 5% ofFebruary 2006 When[Time intervals] Query Select time intervals during whichmore than 10%of the specific patients haveHGB state value greater than “normal” The VISITORS System:Interactive Exploration of Multiple Patients

37. Demographic Constraints: Young (age≤20) or Old (age≥70) Male patients The VISITORS Query-Builder Interface

38. Knowledge-based constraints Hemoglobin [HGB] State was abstracted as Normal or higher for at least seven days after the two weeks period starting from the allogenic bone marrow transplantation, WBC counts were abstracted as Gradient = Increasing during the same period

39. The VISITORS Main Display Interface Subject groups Medical knowledge browser Multiple-subjects raw data Distribution of derived patterns over time Concept search

40. The VISITORS System: Zooming into a Panel

41. Modifying Display Parameters for a Raw-Data Concept

42. Displaying the Distribution of an Abstract Concept over Time

43. Interactive Temporal Data Mining:Temporal Association Charts[Klimov & Shahar, IDAMP 2007] Abstractions for the same subject group are connected; support and confidence indicated by width and hue The data of each subject are connected by a line

44. Temporal Association Charts: Support and Confidence Links Three data mining parameters displayed for each temporal association link: support = 55.60% of patient group confidence = 61.00% probability actual number of patients = 25

45. Temporal Association Charts:Direct Manipulation Using a Time-Value Lens 91% of patients have the “moderately_low” value of the HGB_STATE_BMT CurrentWBC minimal value1.4 103 cells/ml

46. Temporal Association Charts:Direct Manipulation Using a Time-Value Lens (Cont) Now only 44.4% of patients have the “moderately_low” value of the HGB_STATE_BMT New WBC minimal value5.43 103 cells/ml

47. Temporal Association Charts: Using a Relative Time Line Time line is relative to the BMT_Al procedure

48. Temporal Association Charts: Using a Relative Time Line (Cont) During the second month after the BMT_Al procedure During the first month after BMT_Al

49. Temporal Association Charts: Using a Relative Time Line (Cont) 1st month 2nd month 2nd month 1st month

50. Due to local variations in terminology and data structure, linking to a new clinical database requires creation of A term-mapping table A unit-mapping table A schema-mapping table The mapping tools use a vocabulary-server search engine that organizes and searches within several standard controlled medical vocabularies (ICD-9-CM , LOINC, CPT, SNOMED, NDF) Clinical databases are mapped into the standard terms and structure that are used by the clinical knowledge base, thus making the knowledge base(s) highly generic Adding a New Clinical Database to The IDAN Temporal Mediator Architecture