130 likes | 275 Views
Stephen R. Aylward, Director of Medical Imaging, Kitware. Overview. Open-source toolkits Medical image processing (ITK) Image-guided surgery (IGSTK) Microscopy (FARSIGHT) Image display (VTK) Information visualization (VTK) Scientific computing (VTK) Computer vision (VidTK) Open-API
E N D
Overview Open-source toolkits • Medical image processing (ITK) • Image-guided surgery (IGSTK) • Microscopy (FARSIGHT) • Image display (VTK) • Information visualization (VTK) • Scientific computing (VTK) • Computer vision (VidTK) Open-API • Databases (MIDAS) • Open-source applications • Medical images (VolView) • Scientific data (ParaView) • NEW! • Open-source application building • ITK, VTK, Qt (Maverick)
Overview Open-source toolkits • Medical image processing (ITK) • Image-guided surgery (IGSTK) • Microscopy (FARSIGHT) • Image display (VTK) • Information visualization (VTK) • Scientific computing (VTK) • Computer vision (VidTK) Open-API • Databases (MIDAS) • Open-source applications • Medical images (VolView) • Scientific data (ParaView) • NEW! • Open-source application building • ITK, VTK, Qt (Maverick)
IGSTK Features • PI: Kevin Clary, Georgetown (NIH) • Toolkit for application builders • Building blocks for C++ programmers • Trackers, display layouts, scenes, … • Scene graph based on ITK’s SpatialObjects • Objects are tied to trackers • Displays show objects in the scenes • State machine for application control • All events, not just GUI • Tracker coordinates, type of data loaded, tracker unplugged • Patient safety is a top priority • State machine can be visualized and validated • Confirm application never enters an invalid state • Some cost added for application developers.
FARSIGHT Features • PI: Badri Roysam, RPI (NIH) • Toolkit for biologists • Python scripting • 3D/4D Image analysis and visualization • Scene graph based on ITK’s SpatialObjects • Arrangement of cells wrt vasculature • Scene spans time and space • Flexibility is a top priority • New methods for processing and display • Created by biologists (scripts) and computer scientists (new ITK/VTK methods)
Information Visualization using VTK (OverView/ParaView) Figure 3 - The OverView application displaying a large graph linking miRNA sequences with the proteins they affect. Figure 4 - The OverView informatics application is shown retrieving PubMed articles from a search string ("HIV"), clustering them, and displaying the articles in geospatial, graph, landscape, and tabular views.
Maverick • PI: Kitware (AFRL) • Toolkit for application development • Qt modules for C++ programmers • “Qt designer” to define applications • Signals / slots (GUI-driven) • Scene graph • Memory (mavScene – SpatialObjects / MRML) • Disk (MRML) • Display (mavSceneViewers) • Extensible • Slicer Execution Model • GUI (User) is the priority • Workflow of interdependent modules (maximize user-interaction re-use) • Consistent look-and-feel (GUI design standard)
Roadmap (dreams / milestones) • Maverick is released (weeks) • FARSIGHT and IGSTK momentum continues • Slicer converted to Qt and supports multiple workflows • …CTK evolves and is integrated into above systems
What's most important for open-source? NO DUPLICATION! - Specialization for an audience Community support • License • Transparent • Value added • User support • Funding beyond delivery Pick ONE audience • Clinical / biomedical research • 1. Extensible application • 2. Custom workflow applications • Application builder • Algorithms research • Scripting • Pharma • Integration into existing workflows
What's most important for open-source medical imaging applications? Medical imaging • We are processing organs, tumors, vessels; not images. • Portable data on disk • Crosses applications • Patient record • Data representation in memory • ITK Spatial Objects (scene graph) • IsInside • ValueAt • Transform • GUI scene • Present using intuitive names, hierarchy, etc. Why don’t commercial CAD companies fear open-source CAD efforts • Integration / turn-key • GUI • Patents • Clinical testing • Outreach • Cross-institution distribution • These are out targets
Dreams • Imagine… • Custom application from my institute walks me through a workflow for planning a brain tumor biopsy procedure • Save scene • Load scene into Slicer to merge with new DTI data • Save scene • Load scene into surgical suite running IGSTK (robotics) • Biopsy pre-chosen sites • Send biopsy to lab • Lab processes using a new research module loaded into their FARSIGHT-based application • Multiple biomarkers (multiple scales: MRI, DTI, cell, proteomics) are compared with a database of cases to predict best course of treatment. • Radiation oncology program loads scene and biopsy results...
Reality • Legacy applications, formats, data, expertise, … • Legacy funding… • Good news: different audiences • Opportunity for sharing • Solution: CTK • Scene definition (Disk and Memory) • IO Library for scene • Scene converters for other types of scenes/data • API and classes for methods that will run in Slicer, MITK, OpenMAF, Maverick, XIP, ...
CTK Implementation • Scenes (Disk, Memory, Display) • IGSTK / ITK / MITK (SpatialObjects) • Slicer (MRML) • XIP (OpenInventor) • OpenSG • XNAT • Modular methods • Slicer Execution Model • Implement as ITK/VTK command-line programs • Automatically come available as shared libraries • Passes events to parent program • Missing: interaction with parent GUI (mouse clicks) • Can be done, but no standard set • GUI Modules?