Development of Open Source Software for C omputer- A ssisted I ntervention Systems

1. Development of Open Source Software for Computer-Assisted Intervention Systems Peter Kazanzides, Anton Deguet, Ankur Kapoor, Ofri Sadowsky, Andy LaMora, Russell Taylor Johns Hopkins University www.cisst.org October 30, 2005

2. Outline • Background and Motivation • Where are we going? • CISST Libraries • Foundation libraries • Real time support • Development process and tools

3. NSF Engineering Research Center for Computer-Integrated Surgical Systems and Technology • Multi-institution, multi-disciplinary center • Johns Hopkins University + Medical Institutions • MIT + Brigham & Women’s Hospital • CMU + UPMC • Others: Morgan State, Georgetown, Harvard, Penn, Columbia • Funding • Year 7: Core NSF Grant = $3.9M; Total = ~$8.3M • Years 1-10: Core NSF Grant = $30M; Total = ~$55M • University researchers, clinicians, industry • Research, Systems, Education, Outreach

4. Historical Background • ERC-developed software was captured in: • CIS library • Common tools, such as logging • Vectors, matrices, transformations • Interface to tracking systems • Numerical methods, registration, … • MRC library • Common interface to different robots • Essentially a “wrapper” around API for hardware that provides low-level control

5. Read Sensors Compute Goal on Trajectory Compute Joint Goals cisstMRC MRC Robot Controller Architecture Supervisory/Trajectory Control (~100 Hz) Servo Control (~1000 Hz) Application (non-real-time) Application Read Sensors Interpolate Setpoint Hardware API Compute Control

6. Motivation for CISST Package • Improve process, design, testing, portability, maintainability for open source release and to facilitate clinical use: • Programming standards • Design reviews • Portable build tools • Automated testing • User documentation • Enable the development of real-time software for physical devices such as robots and trackers

7. Motivation for Real Time Support • Motivated by transition from motion controller boards (with processor and vendor’s software) to I/O boards (no processor) and research software I/O Boards (non-intelligent) Motion Controller Boards (intelligent hardware)

8. Example: Teleoperation of Snake Robot Control PC (RTAI/Linux) Master Robots I/O and Amps LoPoMoCo Control PC (RTAI/Linux) Slave Robots I/O and Amps LoPoMoCo

9. Servo Control and Amps (Galil) DMC-2143 Example: Image-Guided Robot for Rodent Research PC (Windows) Robot 3D Slicer

10. Where are we going? • C++ Software Libraries • cisst libraries • other libraries Distributed Interface static linking • Frameworks • Based on system complexity • Component of larger system frozen spots dynamic loading hot spots • Binary components • hardware interfaces • research algorithms

11. Foundation libraries cisstCommoncisstVector cisstNumerical cisstInteractive Real Time Support cisstOSAbstractioncisstDeviceInterfacecisstRealtime Interventional Devices cisstTracker cisstMRC… CISST Libraries • (Mostly) Stable • Open source* Sept 2005 • cisstInteractive ~Nov 2005 • Beta version • Open source ~Jan 2006 • Work in process • cisstTracker ~Jan 2006 • cisstMRC~June 2006 *www.cisst.org, current license based on Slicer 2.x, goal is Slicer 3.0

12. cisstVector • Vectors, Matrices and Transformations • Extensive use of C++ templates (metaprogramming) • Fixed size and dynamic • Fixed size especially suited for real-time code • Operations on slices and sub-regions • References to vectors and matrices • Improves interoperability with other libraries • Matrices in row-major or column-major format • C++ wrapping of NetLib (numerical methods, including CLAPACK, MINPACK)

13. Fixed Size Vectors and Matrices • Templated by: • Element type (int, double, etc.) • Dimension (number of rows, columns, etc.) • Efficiency considerations (for templated dimension): • Loop is easy, but not efficient for small vectors: int Sum() { sum = 0 for (i=0; i < _size; i++) sum += data[i]; return sum; } • Recursive function also not efficient: int RecursiveSum(int size){ return (size == 1) ? data[0] : RecursiveSum(size-1) + data[size-1]; } int Sum() { return RecursiveSum(_size); } • Recursive template (template metaprogramming) is efficient: • Compiler “unwraps” recursive template into straight-line code

14. Fixed Size Vectors and Matrices • Operations provided by “recursive engines”: • Classify operations by: • Number of operands • Type of operands • Type of result • Storage location for result • Small number of engines handle all operations • Example: same engine for addition, subtraction, …

15. Python Interface • Automatic wrapping of C++ for Python (Swig) • Object registry to share objects between languages • Can load cisst libraries into Python shell • Can start Python shell from C++ program • GUI features provided by cisstInteractive (using wxWidgets)

16. Can lead to complexity! Device Interface • Device Hierarchies: Tracker Tool GetPosition Robot Tool GetPosition MoveToPosition Should be able to use robot in place of a tracker

17. Device Interface • Our solution: query with string to obtain command object (Command Pattern) Initialize (e.g., from XML) ddiDeviceInterface Configure Provides GetMethodByName Abstract Base Class Return list of implemented operations (strings) Return command object for specified operation (string) Tracker Tool Provides Robot Tool Provides … { “GetPosition” } { “GetPosition”, “MoveToPosition” }

18. Device Interface class myTask : public rtsTask { private: ddiDeviceInterface* dev; ddiCommand* cmd; public: void Startup(); · · · dev = ptr to device devConfigure(…); cmd = devGetMethodByName(“GetPosition”); · · · Non-real-time · · · cmdExecute(data); · · · Real-time void Run(); };

19. Real Time Support • Devices and Tasks should be interchangeable: • Example: Servo control via an intelligent device or via a software task • Solution: Task class derived from device class, but also includes a device ddiDeviceInterface rtsTask

20. Real Time Support • Maintain time history of important state data • State Data Table (SDT), indexed by time and data id • Task communication with Command Pattern: • Read from Task SDT or Device • Write to Task Mailbox or Device • Command object can handle remote communications Command objects High-level task (low frequency) Low-level task (high frequency) Device Interface mailbox mailbox SDT SDT Command objects Hardware

21. map <string, ddiCommandBase * > ddiDeviceInterface Operations rtsTimeIndex osaThread Thread Device IndexReader â IndexWriter â osaThreadBuddy ThreadBuddy rtsTask vector < osaMailBox > MailBoxes StateDataTable Position Output Voltage Desired Position Ticks vector < rtsTimeTicks > rtsStateDataTable vector <string> StateVectorDataNames Legend: [0] [1] [2] vector < rtsStateDataArrayBase * > StateVector Member of Inherits from Instance of (snapshot) ddiCommandBase rtsStateDataArrayBase Device _device* Data template<_device> template<_type> vector <_type> ddiCommand rtsStateDataArray bool (_device::*)() Action cisstRealTime

22. Compile Link Test Results CppUnit PyUnit Development Tools Build Environment CMake CMake Build Instructions (e.g., VC++, gcc/make) Documentation LaTeX Formatted Documentation (e.g., pdf, html) Doxygen Libraries Wrapped Source SWIG Library Binaries (static & dynamic, e.g., cisstVector, cisstCommon) CVS Repository (source control) Dart2 (dashboard) Test App Compile CTest Test Programs Applications Applications Compile Link Optional Interpreter (IRE) Scripts CVSTrac (bug/feature requests)

23. Development Summary • Tools adequately manage implementation, (unit) testing and maintenance phases of development • Automated testing will be hard for physical devices • Much documentation is manually created (not enforced by the process) • Requirements, risk analysis, (high-level) design, validation • User guide, tutorial, quickstart • We needed Dart sooner than we expected! • gcc is getting pickier! • Windows static/shared libraries (dll export)