QueryAtlas

1. QueryAtlas Steve Pieper, Greg Brown, David Kennedy, Maryanne Martone, Jyl Boline, Burak Ozyurt, Wendy Plesniak, Michael Halle, Anna Tang, and Florin Talos

2. QueryAtlas: a Slicer3 module The Query Atlas is a 3D Slicer module that uses atlas-based anatomical representations, linked to underlying semantic descriptions, as an interactive 3D spatial reference for viewing experimental results and for performing information searches to study and interpret them.

3. Slicer3 • Slicer3: • Cross-Platform Integrated Visualization and Analysis: Structure / Function • Multi-Site Development: BWH, MIT, GE, Kitware, Isomics, UCSD, JHU, MGH… • Multi-Project Scope: fBIRN, mBIRN, NAC, NA-MIC, NCIGT, RO1s… • Impact: • fBIRN and mBIRN Informatics (XNAT) and Data (XCEDE) Compatibility • Leverage Active Dissemination and Training Efforts of Partner Grants

4. Slicer3 • Slicer3: Complete re-design using “Community-centered” approach • Considers “user” experience for diverse community: • End-users of the desktop application • Developers working on core software, using different programming styles • Contributors of algorithms who want a light-weight API

5. Slicer3 GUI designed with input from application scientists for improved usability Toolbar with module navigation Module GUI panel Slice viewer control 3D viewer control Status & progress

6. QueryAtlas Overview • Use the QueryAtlas to: • Load volumetric analyses from XCEDE compliant or XNAT databases; • Visualize volumes, models and parametric map overlays. • Annotate models and labelmaps with anatomy labels • Formulate queries in the context of particular study results (fMRI activations or group statistics, for example) • View and manage collections of query results

7. QueryAtlas Overview • Target Audiences: • Neuroscientists: Address “Information Overload” with Ontology-Aware Tools • Tool Builders: Provide Reference Implementation of XCEDE Compliant Application Software • Non-Specialists: Educational Tool and Compelling Demonstration of Data Richness • Leverages Cross-Testbed BIRN Resources

8. QueryAtlas Overview • Complete QueryAtlas Xcede 2.0 Catalog: • To support BIRN research, the QueryAtlas is Xcede 2.0 catalog compliant. A complete QueryAtlas Xcede catalog describes and contains uris for: • an anatomical volume (brain.mgz); • results of a FreeSurfer morphological analysis ( a label map, aparc+aseg.mgz and models, lh.pial and/or rh.pial); • the model's annotataion overlay (lh.aparc.annot and/or rh.aparc.annot) derived from the anatomical; • results of the FIPS processing (an example_functional.nii dataset and brain activation statistics volume datasets of interest, zstatX.nii, zfstatX.nii); and • a FreeSurfer matrix (anat2exf.register.dat) to register the anatomical to the example functional volume. • ** The anatomical volume, the example functional volume and the matrix that registers them in FreeSurfer are required when the catalog is parsed in order to compute the registration matrix in Slicer's coordinate system that correctly aligns all statistics volumes with the scene's anatomical information.

9. QueryAtlas Overview Relationship to BIRN informatics efforts distortion correction tools acquisition protocols local databases PubMed (.xcat) XNAT visualization & Interpretation (Slicer3/QueryAtlas) (.xar) IBVD . . . workflows analysis tools BrainInfo

10. Supported data • Currently, the QueryAtlas supports: • Annotated FreeSurfer datasets (aparc+aseg.mgz, lh/rh.aparc.annot) • Xcede 2.0 catalog .xcat files for fBIRN FIPS/FreeSurfer data • XNAT archive .xar files for mBIRN Qdec data • Can be extended to include other atlas-based annotated label maps or model files: (requires extending readers and data handling methods, and the controlled vocabulary)

11. Overview Supported workflows:

12. Interface Overview 1. Scene loading and Annotation setup 2. Annotation and Display options 3. Ontology Mapping and Browsing 4. Define Search Terms 5. Build queries, View Manage and Save Results

13. Bookmark file read/write for Firefox Set Mozilla Firefox 2 location through the Application Settings Interface View -> ApplicationSettings -> Slicer Settings

14. Loading Qdec data • Load an XNAT .xar archive from command-line, • or load a .qdec file from the GUI • Set up annotations • Select among • population statistics overlays

15. Interactive annotations on Qdec data “way-finding” with interactive annotations on the FreeSurfer inflated brain mapped with different population statistics Does the correlation between thickness and Left NonWM Hypointensities differ from zero? Does the average thickness differ from zero? Does the average thickness differ from zero? click to play movie

16. Loading FIPS/FreeSurfer data (XCEDE 2.0 catalog descriptions) • Close any currently loaded scene. • Load an XCEDE catalog (.xcat file) • from command-line or from the GUI • (example phII BIRN SIRP paradigm, combined morphology and functional analyses) • Set up Slice layers for convenient display • Set up annotations

17. Loading FIPS/FreeSurfer data (XCEDE 2.0 catalog descriptions) • Select among brain activation overlays included in the catalog (zstat.nii, zfstat.nii…) C7 C14 C17

18. Loading FIPS/FreeSurfer data (Go to Volumes Module to control data display) • Select “brain” as the active volume, and set its threshold to “Auto” • Select each statistics • file as the active • volume and set its • threshold and window/ • level to adjust the • display appropriately • Navigate back to the QueryAtlas

19. Interactive annotations Combined structural image data and functional statistics informed by local or formal anatomical annotations • In the annotation and Display Options GUI Panel, select the termset used for annotations • Toggle LH and RH model visibility • In the three Slice Viewers, toggle Slice Plane visibility click to play movie

20. Controlled vocabulary • Extensible mechanism for linking atlas-based labels in various local naming systems to symantically compatible terms used in the larger biomedical informatics community • CSV format files, leverages current BIRNLex ontology and mapping to other termsets and NeuroNames resources from BrainInfo • Specifically provides mapping between local termsets (FreeSurfer) and BIRNLex, NeuroNames, UMLS, IBVD • Extensible to include other local and formal termsets • Distributed with Slicer’s QueryAtlas module • TBD: Authoring/editing policy and mechanisms, means of tracking BIRNLex changes

21. Features: Quick translation & search 3D Viewer’s Context menu (right click) provides a quick jump to ontology mapping and quick search. Ontology Mapping & structure term selection panel Controlled vocabulary leverages current BIRNLex ontology and mapping to other termsets

22. Features: SPL Ontology browser 3D Viewer’s Context menu launches SPL Ontology Browser if local term maps to BIRNLex term or NeuroNames term (requires Java 1.5+) Interactive tree view of BIRNLex and NeuroNames structures entry matching terms f.o.v Tree view: pan, zoom, expand, collapse info

23. Features: SPL Ontology browser • SPL Ontology browser: • information visualization tool • written in Java using Jython scripting language • based on open-source Prefuse toolkit • contains subset of BIRNlex ontology (root = Regional part of Organ) • contains all of NeuroNames, including synonyms • converted to JSON format for simplified parsing and better human readability: • NeuroNames CSV file to JSON is python script • BIRNlex to JSON uses Jython and Jena ontology toolkit • Slicer sends commands using a socket-based mechanism • (Currently, communication is one-way)

24. Features: Detailed search Select terms of interest and pass to structure search terms in the Search Terms GUI panel…

25. Features: Detailed search • Select among structure terms to use • Specify additional (group, species, other) terms • Select search target, and inspect resulting links • Reserve and accumulate useful links

26. Features: Detailed search • Save useful links to Firefox bookmark file (or load past results)

27. Example data & more information • FIPS/FreeSurfer example dataset: • http://wiki.na-mic.org/Wiki/index.php/Slicer3:Module:QueryAtlas#Example_data • Qdec example dataset: • http://www.na-mic.org/Slicer/Download/Data/TEST4.qdec • Simple tutorial describing features and use: • http://wiki.na-mic.org/Wiki/index.php/Slicer3:Module:QueryAtlas

28. Future work • Ability to query richer information repositories and infrastructure using available web services • Information display tools for intuitive inspection and integrated comprehension of search results (images, data tables, text) • Tools for transacting with BIRN HID and other XCEDE-compliant data resources • Extending software and controlled vocabulary to include new atlas information • Adding experimental natural language parsing tools for query formulation • Developing mechanisms for tracking BIRNLex and approach to community authoring and editing of controlled vocabulary

29. Acknowledgement This research was supported by Grant 5 MOI RR 000827 to the FIRST BIRN and Grant 1 U24 RR021992 to the FBIRN Biomedical Informatics Research Network (BIRN, http://www.nbirn.net), that is funded by the National Center for Research Resources (NCRR) at the National Institutes of Health (NIH). This work was also supported by NA-MIC, NAC, NCIGT. NeuroNames ontology and URI resources are provided courtesy of BrainInfo, University of Washington.