Application Hosting A Standardized API for Launching and Communicating with 'Plug-in' Applications

1. Application HostingA Standardized API for Launching and Communicating with 'Plug-in' Applications DICOM Working Group 23 Lawrence Tarbox, Ph.D., Chair Mallinckrodt Institute of Radiology Washington University in St. Louis School of Medicine

2. any WG23 Host caBIG Agfa Syngo Cedara Advantage WG-23 Goal • Portable applications that ‘plug into’ any host that implements the standardized ‘socket’

3. Goals A Standardized API that is: • Language independent • Platform independent • IP independent • Extensible • Secure

4. Suggested Staging • Stage one – Access to DICOM Datasets and Results Recording • Stage Two – Access to Non-Interactive Application Services (e.g. print, archive) • Stage Three – Access to Interactive Application Services (e.g. GUI, ‘skins’, rendering) • Stage Four – Standard Workflow Descriptions, and Interactions Between Hosted Software

5. Targets for Stage One • Basic Launch and Control of a Hosted Application • Load, Unload, Start, Abort • Simple Interchange of Data Between a Hosting System and Hosted Applications • Data inputs and outputs described using DICOM Semantics • DICOM messages/objects need not be used directly, instead the API could give access to parts of the objects • Manual Configuration

6. Highlights Since MI-2006 • Multiple Drafts of Supplement 118 • Initial Reading by WG-6 • Joined forces with the NCI caBIG XIP Project • Open Source Reference Implementation • Demonstrations at RSNA • Targeted Use Cases in Clinical Research • Accelerated Schedule

7. WG23 / XIP Relationship • WG-23 addresses clinical integration and vendor inter-operability by defining standardized “plugs” and “sockets” (APIs) • caBIG XIP addresses an open-architecture, open-source, integrated environment for rapid application development based onWG 23 APIs XIP developed Application Standard API … Commercial Vendor #1 Commercial Vendor #2 Unix, Mac, PC Internet Server  Prototype & Collaboration   Clinical 

8. What is ? • The “eXtensible Imaging Platform” is an open source environment for rapidly developing medical imaging applications from an extensible set of modular elements. • It allows developers to easily develop, evaluate, and new approaches to medical imaging problems in a translational research setting. • WG-23 interfaces will be used to gain application portability and platform independence. • The National Cancer Institute (NCI) contracted for the development of XIP through its caBIG program.

9. Who is Contributing to ? • The caBIG In Vivo Imaging Workspace, Software SIG • Released the XIP RFP • Provides primary feedback to the XIP development team • Washington University in St. Louis, Electronic Radiology Lab • Main coordinating site • Will leverage other activities in caBIG, DICOM, IHE, clinical trials • Siemens Corporate Research (SCR) • Contributing a suite of tools – ivRAD – that will form the basis for XIP • Experts in moving ideas from prototypes to commercial reality • DICOM WG-23 • Standardizing the interfaces between a hosting system (e.g. workstation) and hosted post-processing applications (a.k.a. “Plug-ins”) • Representation from both vendor and user communities • ITK/VTK community • Providing image processing and visualization libraries with the assistance of Kitware

10. What is Included in • XIP Rapid Application Development Tools and Libraries (RAD) • Development and application build environment • Extensive and extensible set of libraries for imaging and visualization (XIP Libraries) • Uses the Open Inventor framework • Includes code generating wizards to create new objects and wrap existing libraries • XIP Workstation (WS) • A reference implementation of a medical imaging workstation developed using XIP RAD and DICOM WG-23 APIs • Includes two key components: • XIP Application – a use case specific “plug-in” application integrated via the DICOM WG-23 Interface • XIP Host – the hosting environment that provides XIP Applications with access to services such as data stores, remote processing, etc.

11. XIP Application XIP Libraries Host Independent XIP Host Adapter API caGrid Analytical Svc. caGrid Data Svc. AIM MetaData . . . ITK VTK XIP LIB . . . Remote Processing Annotation and Markup Data Access . . . WG23 WG23 WG23 WG23 Host-Specific XIP Libraries XIP Host (Can be replaced with any WG23-compatible Host) Framework & Architecture Open Inventor Scene Graph / Pipeline+ Optional GUI Engine Open Inventor Nodes, Engines, Manipulators for Accessing Data Services, Host Services, Remote Services, etc. Open Inventor Nodes, Engines, Manipulators for Rendering, Local Processing, etc. ‘Glue’ Logic that Ties Host Dependent Libraries to the Host Environment

12. Open Inventor • Open Inventor ® is an object-oriented 3D toolkit offering a comprehensive solution to interactive graphics programming problems. URL: http://oss.sgi.com/projects/inventor/ • Its programming model is based on the Model/View/Controller design pattern and the concept of Pipelines. • Open Inventor: • is built on top of OpenGL® • defines a standard file format for 3D data interchange • introduces a simple event model for 3D interaction • provides animation objects called Engines • provides high performance object picking • is window system and platform independent • is a cross-platform 3D graphics development system • encourages programmers to create new customized objects

13. Open Inventor modules in C++ modules represent Engines, Nodes and Manipulators • Engines enable the creation of processing pipelines • Nodes support the concept of scene graphs, which are hierarchical structures of objects describing what needs to be visualized in 2D/3D • Manipulators handle input devices, measurements and coordinate transforms in response to user interaction

14. Integrating existing toolkits into • Automatic Wrapper generation for 2D/3D libraries/toolkits such as ITK and VTK – User can review parsed results and choose to support only the desired data types, hide some methods, exclude some classes, etc. • Wrapped ITK functions include Region Growing, Neighborhood, Isolated, Confidence, Watershed, Thresholding, Edge Detection, Laplacian, Gaussian • Support for ITK Data Meshes and Vector Fields

15. XIP FeaturesModules for DICOM loading and 2D/3D Display • XIP modules extend Open Inventor to facilitate medical imaging application development: • Database access (read/write) • DICOM query/retrieve • Image/Volume types • Lookup tables • Transfer function editor • MPR intersection lines/manipulators • camera control (pan/zoom/rotate) • 2D Image display • ROIs, Annotations, Measurements • …

16. XIP FeaturesModules for Fused Volume Rendering • Volumes are stored separately and fused at rendering time, not as a preprocessing step • Support for unlimited number of fused large volumes • Each volume has independent control of: • Transformation (rot, trans, scale, shear…) • Color/opacity Transfer function • Crop box • Cut-planes • Rendering mode (VRT, MIP, MinIP, DRR, SSD) • Voxel Resolution • Sampling rate • Performance • 20 frames/sec during interactivity • 1 sec for final diagnostic-quality update

17. XIP FeaturesModules for Client/Server Remote Visualization •  Visual creation and configuration of distributed services • Thin Client and Smart Client configurations are supported • Support for caGRID remote grid computing services • XIP modules for DICOM Query, Sorting, remote data transmission • XIP can serialize the entire state to a file, thereby facilitating support for client/server state management and recovery as well as workflow management.

18. An Open Platform for WG-23 Application Development

19. Volunteers Solicited WG 23 welcomes your input. We would be even happier with your assistance in creating this new standard or helped in the reference implementation. • Join the mailing list and contribute ideas • Join us at future meetings • Participate in NCI caBIG IVI WS SW SIG

20. Q & A