Capturing and Modeling Neuro-Radiological Knowledge on a Community Basis: The Head Injury Scenario

1. Capturing and Modeling Neuro-Radiological Knowledge on a Community Basis: The Head Injury Scenario. Alexander Garcia, Zhuo Zhang, Menaka Rajapakse, Christopher J. O. Baker, and Suisheng Tang Data Mining Department Institute for Infocomm Research Singapore

2. Outline • Motivation • MiBank – Head Injury Database • Ontology Development • Collective Intelligence • The “facebook” approach • Medical Image Annotator • Discussion and Conclusions

3. Motivation • National Neurological Institute Singapore (NII) has 500+ head injury patients each year with Brain, Scalp, Skull, Internal bleeding requiring rapid diagnosis. • Clinical radiology reports comprise of multiple series of Computed Tomography (CT) Images with unstructured text associated to images) • Computationally a weak association between images and words, cannot retrieve similar images. • Conceptually a tightly coupled association between Image and Diagnosis • MiBank database of DICOM files, (http://dicom.i2r.a-star.edu.sg/pacsone/)

4. Features of MiBank • Browser • Search by category, patient, report, note, study • Annotation with free-text • Forum discussion • DICOM viewer • Image upload & download

5. MiBank: Medical Image Databank: Head Injury Web site: http://dicom.i2r.a-star.edu.sg/pacsone/ 505 studies, 1775 series, 31561 images. Pass word protect, DICOM viewer, searchable

6. Possible query in MiBank Show me all cases who have skull fracture with acute subdural hematoma But do not have brain edema. Details in next page Impossible query with no-predefined terms Show all cases who have skull fracture with midline shift and acute subdural hematoma But do not have brain edema.

7. Current Limitations of MiBank • Can not query based on image features explicitly; • Can not associate the description in R-report to specific instance of an image. Need to see all instances for bone fracture Sample: Radiology Report • A fracture of the right frontal bone. • Mild midline shift to the left is present. • An acute extraduralhematoma, measuring 1.9 cm in maximal thickness, is noted. • A 1 cm thick acute subdural hematoma is also present over the right cerebral hemisphere.

8. What do we want… What do we need? • Properly annotated data: images, radiology reports • Meaningful associations between reports, images, and across images • …an ontology …. • Retrieve patients with right midnight shifts of less than 3mm for whom there has been no reported haematoma • Retrieve all images similar to this one

9. Header InfoMining Semantic Query Ontology OriginalDICOM data Categorization Image retrieve Web Interface Head Injurydatabase(Relational) Indexing Customized online report Reportdata Statistic report Text mining Search Engine Discussion forum Visualization

10. The Role of the Ontology • Community defined controlled vocabulary for annotation of radiology images. • Hierarchical descriptions of medical terms relevant to anatomy, pathology and head injury specific features found in medical images. • Consensus model of head injury terminology generated through community engagement for knowledge reuse in medical information systems. • Query model for semantic search

11. Ontology Development Garcia et al

12. Ontology Development P h a s e 1 P h a s e 2

13. Text Processing / Baseline Ontology FMA Non FMA “Plain scans were acquired. Note is made of the MRI dated 2/3/2004 and CT dated 18/2/2004.Evidence of previous left high parietal craniectomy noted. Hypodensity in the left parietal-occipital region is compatible with gliosis at site of previous surgery. A large left-sided scalp hematoma is seen. Underlying linear radiolucency in the left frontal bone was seen. This suggests an undisplaced fracture. Underlying acute subdural hematoma is seen with a maximal depth of 1.2 cm. Acute subarachnoid blood is also noted collecting mainly in the ipsilateralcerebral hemisphere, sylvian fissure as well as tentorium. There is diffuse cerebral edema. Mass-effect is seen with midline shift to the right, and developing hydrocephalus. Basal cisterns are effaced”. FMA / Galen / R-report terms: anatomy, pathology, trauma, injury

14. Requires excessive amount of time Experts – easily bored – no short term result. Results in the creation of unstructured knowledge stores that are difficult to reuse and maintain. Skimping on validation may include errors, omissions, inconsistencies & irrelevances Experts are not always capturing the evidence – rather explaining context Storing the knowledge that is not machine-readable Capturing Knowledge: Phase 1 Not an easy task Disadvantages • Inside expert’s head • Difficult to describe • concepts and relations • Difficult for non- • experts to understand.

15. Ontology Development Maintenance Evolution P h a s e 1 P h a s e 2

16. Capturing Knowledge: Phase 2 • Knowledge Elicitation via Collective Intelligence • The capacity to provide useful information based on human contributions which gets better as more people participate. • Data Types • mix of structured, machine-readable data and unstructured data from human input • Collective Knowledge Resources • intelligent collection? • collaborative bookmarking, searching • “database of intentions” • clicking, rating, tagging, buying - Amazon • what we all know but hadn’t got around to saying in public before • blogs, wikis, discussion lists -

17. Retrieving images of the diving trip to Australia. Albert and Alex have to be in the photo. facebook Tags Make The Difference ! • The Premise: From unstructured and unrelated annotation to structured meaningful annotation • Simple tagging it possible to derive meaningful associations • Need to have a tool to gather knowledge that is directly linked to supporting evidence.

18. Main challenge in medical image retrieval is that it heavily depends on expert’s knowledge of data structures and annotation is poor. So the objective of MIA is knowledge capture. MIA is designed for medical image annotation and its users are domain experts who require a consistent vocabulary for annotation tasks, knowledge sharing and machine automation. User community consists of Radiologists, Neurosurgeons (specifically, NNI doctors). Medical students, junior doctors, image processing researchers. MIA is a designed to both facilitate the building of appropriate ontology by domain experts and effective maintenance and evolution of the ontology, given new use cases /images. Medical Image Annotator: MIA MIA User Interface Our contribution: the use of WEB 2.0 technology to support knowledge capture, and the approach to community engagement in the development of the ontology; more concretely in the maintenance and evolution

19. MIA: Platform Architecture Easy to extend, any OWL file can be loaded • Ontologies can be edited online: • * add node * rename node * delete node Ajax to update ontologies on server side to provide dynamic content on a web page so no page-refresh, no re-loading Image .owlfile OntologyEditor Database AJAX Owl Parser Java script (DHTML) Ontology & Image Management Console Tree Constructor OntologyViewer Client-side browser Server-side processors • OWL files can be loaded dynamically • OWL  relational database  OWL • Users can keep their own version of ontology • Consolidated ontology will be generated based on community inputs.

20. Knowledge Capture in Action

21. Knowledge Capture in Action

22. Knowledge Capture in Action

23. Medical Image Annotator: MIA Advantages • Fast and easy • Domain experts lead the process • Always rooted in reality or a medical use case • Maintenance and evolution of the controlled vocabulary is assured. • Excellent training for new doctors / radiologists • Facilitates Data Mining of Radiology reports

24. Ontology Evolution • Different trainee and clinical doctors building ontologies with extensions on different sub trees • Consolidated ontology is currently manually curated • Goal is automatically align & merge ontologies

25. Query with the Head Injury Ontology • Simple ‘ontology-term’ assisted query • Search for images: based merely on simple combination of ontology terms (and / or) • Form based interface linked to SQL Queires • Ontology reasoning (A-box) • Content navigation over R-reports using defined object properties (Knowlegtor) • Use of subsumption and object properties

26. Head Injury Ontology

27. Find patient records for ‘Fracture’

28. Discussion and Conclusions • Medical images should be better annotated in order to facilitate information retrieval • Collective knowledge is real… “FAQ-o-Sphere” • Controlled vocabularies (CVs) and/or ontologies are being developed by communities • Simple tagging combined with knowledge elicitation methods supports ontology development • Collective knowledge capture requires dedicated infrastructure that supports specific tasks • Querability can be improved through the use of explicit tags and CVs/ontologies

29. Challenges for the Community • How to get knowledge from all those intelligent people on the Internet • How to give everyone the benefit of everyone else’s experience • How to leverage and contribute to the ecosystem that has created today’s web. Social + Semantic Web Social Web Life Science

30. Acknowledgments • Bonarges Aleman-Meza – Social Web • Tom Gruber - Semantic-Social Web • MIA Developers - Zhang Zhuo and Menaka Rajapakse • Suisheng Tang M.D. and Project PI, - Coordinator of domain experts and builder of baseline ontology • Tchoyoson Lim – Radiologist NNI (National Neuroscience Institute, Singapore)