KMeD: A Knowledge-Based Multimedia Medical Database System

1. KMeD: A Knowledge-Based Multimedia Medical Database System Wesley W. Chu Computer Science Department University of California, Los Angeles http://www.cobase.cs.ucla.edu

2. KMeD A Knowledge-Based Multimedia Medical Distributed Database System A Cooperative, Spatial, Evolutionary Medical Database System Knowledge-Based Image Retrieval with Spatial and Temporal Constructs October 1, 1991 to September 30, 1993 July 1, 1993 to June 30, 1997 May 1, 1997 toApril 30, 2001 Wesley W. Chu Computer Science Department Alfonso F. Cardenas Computer Science Department Ricky K. Taira Department of Radiological Sciences

3. Students John David N. Dionisio Chih-Cheng Hsu David Johnson Christine Chih Collaborators Computer Science Department Alfonso F. Cardenas UCLA Medical School Denise Aberle, MD Robert Lufkin, MD Ricky K. Taira, MD Research Team

4. Significance • Query multimedia data based on image content and spatial predicates • Use domain knowledge to relax and interpret medical queries • Present integrated view of multiple temporal and evolutionary data in a timeline metaphor

5. Overview • Image retrieval by feature and content • Query relaxation • Spatial query answering • Similarity query answering • Visual query interface • Timeline interface • Sample cases

6. Image Retrieval by Content • Features • size, shape, texture, density, histology • Spatial Relations • angle of coverage, shortest distance, overlapping ratio, contact ratio, relative direction • Evolution of Object Growth • fusion, fission

11. Characteristics of Medical Queries • Multimedia • Temporal • Evolutionary • Spatial • Imprecise

12. 01 01 O O’ O O Om On Evolution: Object O evolves into a new object O’ Fusion: Object 01, …, Om fuse into a new object Fission: Object O splits into object 01, …, On

13. Case a: Case b: The object exists with its supertype or aggregated type. The life span of the object starts after and ends with its supertype or aggregated type. Case c: Case d: The life span of the object starts with and ends before its supertype or aggregated type. The life span of the object starts after and ends before its supertype or aggregated type.

14. Lesion Micro- Lesion Micro- Lesion

19. Query Modification Techniques • Relaxation • Generalization • Specialization • Association

20. More Conceptual Query Specialization Generalization Conceptual Query Conceptual Query Specialization Generalization Specific Query Specific Query Generalization and Specialization

21. Type Abstraction Hierarchy • Presents abstract view of • Types • Attribute values • Image features • Temporal and evolutionary behavior • Spatial relationships among objects • Provides multi-level knowledge representation

22. TAH Generation for Numerical Attribute Values • Relaxation Error • Difference between the exact value and the returned approximate value • The expected error is weighted by the probability of occurrence of each value • DISC (Distribution Sensitive Clustering) is based on the attribute values and frequency distribution of the data

23. TAH Generation for NumericalAttribute Values (cont.) • Computation Complexity: O(n2), where n is the number of distinct value in a cluster • DISC performs better than Biggest Cap(value only) or Max Entropy(frequency only) methods • MDISC is developed for multiple attribute TAHs. Computation Complexity: O(mn2), where m is the number of attributes

24. Query Display Yes Relax Attribute Answers Database No Query Modification TAHs Query Relaxation

25. Cooperative Querying for Medical Applications • Query • Find the treatment used for the tumor similar-to(loc, size)X1 on 12 year-oldKorean males. • Relaxed Query • Find the treatment used for the tumor Class Xon preteenAsians. • Association • The success rate, side effects, and cost of the treatment.

26. Type Abstraction Hierarchies forMedical Domain Tumor (location, size) Age Ethnic Group Class X [loc1loc3] [s1 s3] Class Y [locY sY] Preteens Teen Adult Asian African European 11 12 10 9 Japanese Filipino Korean Chinese X3 [loc3 s3] X1 [loc1 s1] X2 [loc2 s2]

27. TAH Lateral Ventricle TAH SR(t,b) TAH Tumor Size TAH SR(t,l) Knowledge Level SR(t,l) SR(t,b) Schema Level Lateral Ventricle Tumor Brain SR: Spatial Relation b: Brain t: Tumor l: Lateral Ventricle Knowledge-Based Image Model Representation Level (features and contents)

28. Queries Query Analysis and Feature Selection Knowledge Based Query Processing Knowledge-Based Content Matching Via TAHs Query Relaxation Query Answers

29. User Model To customize query conditions and knowledge-based query processing • User type • Default Parameter Values • Feature and Content Matching Policies • Complete Match • Partial Match

30. User Model (cont.) • Relaxation Control Policies • Relaxation Order • Unrelaxable Object • Preference List • Measure for Ranking

33. Query Preprocessing • Segment and label contours for objects of interest • Determine relevant features and spatial relationships (e.g., location, containment, intersection) of the selected objects • Organize the features and spatial relationships of objects into a feature database • Classify the feature database into a Type Abstraction Hierarchy (TAH)

34. Similarity Query Answering • Determine relevant features based on query input • Select TAH based on these features • Traverse through the TAH nodes to match all the images with similar features in the database • Present the images and rank their similarity (e.g., by mean square error)

35. Spatial Query Answering • Preprocessing • Draw and label contours for objects of interest • Determine relevant features and spatial relationships (e.g., location, containment, intersection) of the selected objects • Organize the features and spatial relationships of objects into a feature database • Classify the feature database into a type abstraction hierarchy (TAH)

36. Spatial Query Answering (cont.) • Processing • Select TAH based on t he query conditions and context • Search nodes to match the query conditions • Return images linked to the TAH node

37. Similarity Query Answering • Preprocessing • Select objects and specify features of interest in the image • Create a feature database of the selected objects for all images • Classify the feature databases as type abstraction hierarchies

38. Similarity Query Answering (cont.) • Processing • Determine relevant features based on query input • Select TAH based on these features (interact with user to resolve ambiguity) • Traverse through the TAH nodes to match all the images with similar features in the databases • Present the images and rank their similarity (e.g., by mean square error)

40. Visual Query Language and Interface • Point-click-drag interface • Objects may be represented iconically • Spatial relationships among objects are represented graphically

41. Visual Query Example Retrieve brain tumor cases where a tumor is located in the region as indicated in the picture