Model-Guided Therapy and the role of DICOM in Surgery

1. Model-Guided Therapy and the role of DICOM in Surgery Heinz U. Lemke, PhD Chair of Working Group 24 “DICOM in Surgery“

2. Content • Introduction (problems and solutions) • Model guided therapy with TIMMS • Classification and model classes • Virtual human model examples • Conclusion

3. Image Manager - Report Video Endoscopy Monitor C-Arm Images MD’s Staff RN, Tech EEG Monitoring MRI Image - PACS C-Arm Fluoroscopy Left side of OR Laser generator EMG Monitoring Image view boxes Digital endoscopic OR suite facilitates MISS Teleconferencing - telesurgery Computer Assisted Digital OR Suite for Endoscopic MISSProblems: Multiple Data Sources Courtesy of Dr. John Chiu

4. Model Guided Therapy and the Patient Specific Model • Model Guided Therapy (MGT) is a methodology complementing Image Guided Therapy (IGT) with additional vital patient-specific data. • It brings patient treatment closer to achieving a more precise diagnosis, a more accurate assessment of prognosis, as well as a more individualized planning, execution and validation of a specific therapy. • By definition, Model Guided Therapy is based on a Patient Specific Model (PSM) and allows for a patient specific intervention via an adapted therapeutic workflow.

5. Model Guided Therapy and data structures • Model Guided Therapy based on patient specific modelling requires appropriate IT architectures and data structures for its realisation. • For PSMs, archetypes and templates allow different levels of generalisation and specialisation, respectively.

6. Mechatronics (Navigation, ablation, …) EBM Workflow IHE Omics EMR Data bases (Atlas, P2P repositories, data grids, ...) Model Based Patient Care Modalities (X-ray,CT, US, MR,SPECT, PET,OI) Model Creation and Diagnosis (Data fusion, CAD, …) Model Maintenanceand Intervention (Simulation, decision support, validation, …) Biosensors (physiology, metabolism, serum, tissue, …) IT Communication Infrastructure

7. Content • Introduction (problems and solutions) • Model guided therapy with TIMMS • Classification and model classes • Virtual human model examples • PM data structures (SDTM and OpenEHR) • Conclusion

8. Interventional Cockpit/SAS modules Images and signals Modelling tools Computing tools WF and K+D tools Rep. tools Devices/ Mechatr. tools Validation tools WF`s, EBM, ”cases” Models (Simulated Objects) IT Model-Centric World View Repo- sitory Engine Data Exch. Control Therapy Imaging and Model Management System (TIMMS) Modelling Simulation Kernel for WF and K+D Management Visualisation Rep. Manager Intervention Validation IO Imaging and Biosensors Therapy Imaging and Model Management System (TIMMS) ICT infrastructure (based on DICOM-X) for data, image, model and tool communication for patient model-guided therapy Models and intervention records Data and information

9. Model Guided Therapy with TIMMS • For a therapeutic intervention it is assumed that human, mechatronic, radiation or pharmaceutical agents interact with the model. • MGT provides the scientific basis for an accurate, transparent and reproducible intervention with the potential for validation and other services. • TIMMS is an IT meta architecture allowing for interoperability of the agents to facilitate a MGT intervention.

10. Model Guided Therapy The basic TIMMS patient model must have the following features: • The TIMMS patient model must have components which represent the patient as an n-dimensional and multiscale (in space and time) data set. • The TIMMS patient model must facilitate interfacing to the surgeon and other operative personnel, the TIMMS engines, TIMMS repositories, and the IT infrastructure. • The TIMMS patient model must be capable of linking these components, which may be static or dynamic, in a meaningful and accurate way. • For dynamic components, the TIMMS patient model must be able to process morphological and physiological data and perform the necessary mathematical functions to maintain the model in an up-to-date state.

11. Model Guided Therapy • The TIMMS patient model must be capable of being incorporated by the TIMMS executing workflow and responding to its changes. • The TIMMS patient model must be amenable to be developed using readily available, standardized informatics methodology. Tools may include UML, XML, Visio, block diagrams, workflow diagrams, MATLAB, Simulink, DICOM (including surgical DICOM), Physiome, CDISC SDTM, openEHRand similar products and tools. • The TIMMS patient model must comply to software engineering criteria, for example, to open standards and service-oriented architectures to allow for multi-disciplinary information exchange. • The TIMMS patient model must allow for further extensions to incorporate advances in molecular medical imaging, genomics, proteomics and epigenetics. • The TIMMS patient model must be amenable to be used for clinical trials, predictive modeling, personal health records and in the long term contribute to a Model Based Medical Evidence (EBME) methodology.

12. Interventional Cockpit/SAS modules Images and signals Modelling tools Computing tools WF and K+D tools Rep. tools Devices/ Mechatr. tools Validation tools WF`s, EBM, ”cases” Models (Simulated Objects) IT Model-Centric World View Repo- sitory Engine Data Exch. Control Therapy Imaging and Model Management System (TIMMS) Modelling Simulation Kernel for WF and K+D Management Visualisation Rep. Manager Intervention Validation IO Imaging and Biosensors Therapy Imaging and Model Management System (TIMMS) ICT infrastructure (based on DICOM-X) for data, image, model and tool communication for patient model-guided therapy Models and intervention records Data and information

13. Generic and patient specific n-D modelling tools Modelling tools • Geometric modelling • Prosthesis modelling • Properties of cells and tissue • Segmentation and reconstruction • Biomechanics and damage • Tissue growth • Tissue shift • Properties of biomaterials • ...

14. Model Guided Therapy • MGT in its simpliest instantiation is an intervention with a subset, a single or a set of voxels representing locations within the patient body. With this view, it is an extension from Image (pixel) Guided Therapy (IGT) to model (voxel) guided therapy. Examples of model guided therapy are: a) interventions within a subset of a voxel, e.g. cells, organelles, molecules, etc. b) interventions with a voxel, e.g. small tissue parts of an organ or lesion, etc. c) interventions with a set of voxels, e.g. part of functional structures of organs, organ components, soft tissue, lesions, etc.

15. Model Guided Therapy In a simple PSM, voxels may be associated with several dimensions of data • 1-D signals (e.g. EEG) • 2-D projection and tomographic images • 3-D reconstructions • Temporal change • Tissue/cell type • Ownership to organ, lesion, system, prothesis, chronic condition, etc. • Spatial occupancy/extension • Permeability (blood brain barrier) • Flow (e.g. electric, heat, liquid, perfusion, diffusion, etc.)

16. Model Guided Therapy In a simple PSM, voxels may be associated with several dimensions of data • Level of oxygenation (e.g. level of hypoxia) • Pharmacokinetics (e.g. effect of tissue on pharmaceutical agent, flow parameters, time to peak, etc.) • Pharmacodynamics (effect of pharmaceutical agent on tissue, ablation parameters) • Biological marker types (in vitro and/or in vivo molecular spectrum) • Reference coordinate system (e.g. Schaltenbrand/Warren, Talaraich/Tourneaux) • Value (life critical to life threatening) • Neighbourhood (e.g. 3³, 5³, 7³, etc.) • ...

17. Example: ENT model elements Source: G. Strauss

18. Example: ENT model elements Source: G. Strauss

19. Content • Introduction (problems and solutions) • Model guided therapy with TIMMS • Classification and model classes • Virtual human model examples • Conclusion

20. Strategies for multiscale modelling • Modelling is essential for understanding the knowledge of human characteristics such as, anatomy, physiology, metabolism, genomics, proteomics, pharmacokinetics, etc. • Because of the complexity of integrating the knowledge about the different characteristics the model of a human has to be realised on different levels (multiscale in space and time) and with different ontologies, depending on the questions posed and answered delivered. • The problems associated with using reduced-form components within large systems models stem primarily from their limited range of validity.

21. Source: J. Bassingthwaighte

22. Patient specific and associated modelling functions In the Model-Centric World View a wide variety of information, relating to the patient, can be integrated with the images and their derivatives, providing a more comprehensive and robust view of the patient. By default, the broader the spectrum of different types of interventional/surgical workflows which have to be considered, the more effort has to be given for designing appropriate multiscale PSM’s and associated services.

23. Patient specific and associated modelling functions Management of n-D and multi resolutional knowledge (model of the biologic continuum in space and time) is still a research and development challenge. If solved successfully, it will transform surgery into a more scientifically based activity.

24. Content • Introduction (problems and solutions) • Model guided therapy with TIMMS • Classification and model classes • Virtual human model examples • Conclusion

25. Patient Specific CMB Human Laser Scan (CAESAR DB) Multimodal Imaging(MRI, CT, Angio,..DT-MRI) Visible Human Anatomical Templateorgan surface meshes Roberts JHU Spitzer 2006 Virtual Anatomy PKPD FEM Mesh (Roberts JHU)

26. Content • Introduction (problems and solutions) • Model guided therapy with TIMMS • Classification and model classes • Virtual human model examples • Conclusion

27. Solutions and Research Focus(medical) • Transition from image guided to model guided therapy (e.g. through workflow and use case selection/creation/repositories) • Concepts and specification of patient specific models in a multiscale domain of discourse • Concepts and design of a canonical set of low level surgical functions • Prototyping

28. Interventional Cockpit/SAS modules Images and signals Modelling tools Computing tools WF and K+D tools Rep. tools Devices/ Mechatr. tools Validation tools WF`s, EBM, ”cases” Models (Simulated Objects) IT Model-Centric World View Repo- sitory Engine Data Exch. Control Therapy Imaging and Model Management System (TIMMS) Prototyping Modelling Simulation Kernel for WF and K+D Management Visualisation Rep. Manager Intervention Validation IO Imaging and Biosensors Therapy Imaging and Model Management System (TIMMS) ICT infrastructure (based on DICOM-X) for data, image, model and tool communication for patient model-guided therapy Models and intervention records Data and information

29. Solutions and Research Focus(technical) • Concepts and data structure design of patient specific models (e.g. with archetypes and templates) • Model management with open architectures (e.g. SOA) • SOA modulariation with repositories, engines, LLM´s and HLM´s • LLM´s as adaptive (cognitive/intelligent) agents • HLM´s as application modules (competitive differentiation) • LLM´s possibly as open source • Kernel (engine and repository) for adaptive workflow and K+D management • Cooperative and competitive R+D framework for engine and repository building • Therapy based open standard ( e.g. S-DICOM) • Transition from CAD to CAT modelling

30. Interventional Cockpit/SAS modules Images and signals Modelling tools Computing tools WF and K+D tools Rep. tools Devices/ Mechatr. tools Validation tools WF`s, EBM, ”cases” Models (Simulated Objects) IT Model-Centric World View Repo- sitory Engine Data Exch. Control Therapy Imaging and Model Management System (TIMMS) Archetypes and Templates Modelling Simulation Kernel for WF and K+D Management Visualisation Rep. Manager Intervention Validation IO Imaging and Biosensors Therapy Imaging and Model Management System (TIMMS) ICT infrastructure (based on DICOM-X) for data, image, model and tool communication for patient model-guided therapy Models and intervention records Data and information

31. Solutions and Research Focus(medical and technical) • Transition from image guided to model guided therapy (e.g. through workflow and use case selection/creation/repositories) • Use cases for adaptive workflow, exception handling and K+D management for selected interventions • Cooperative and competitive R+D framework for low (open source) and high level (competitive differentiation) surgical function computerisation • Information/model flow from diagnosis (e.g. CAD) to CAT (i.e. interdisciplinary cooperation) • Development of standards for patient modelling in WG24 “DICOM in Surgery”

32. Interventional Cockpit/SAS modules Images and signals Modelling tools Computing tools WF and K+D tools Rep. tools Devices/ Mechatr. tools Validation tools WF`s, EBM, ”cases” Models (Simulated Objects) IT Model-Centric World View Repo- sitory Engine Data Exch. Control Candidate components for open source Open Source Modelling Simulation Kernel for WF and K+D Management Visualisation Rep. Manager Intervention Validation IO Imaging and Biosensors Therapy Imaging and Model Management System (TIMMS) ICT infrastructure (based on DICOM-X) for data, image, model and tool communication for patient model-guided therapy Models and intervention records Data and information

33. WG 24 “DICOM in Surgery“ Project Groups PG1 WF/MI Neurosurgery PG2 WF/MI ENT and CMF Surgery PG3 WF/MI Orthopaedic Surgery PG4 WF/MI Cardiovascular Surgery PG5 WF/MI Thoraco-abdominal Surgery PG6 WF/MI Interventional Radiology PG7 WF/MI Anaesthesia PG8 S-PACS Functions PG9 WFMS Tools PG10 Image Processing and Display PG11 Ultrasound in Surgery

34. Definition of Surgical Workflows (S-WFs) Micro Laryngeal Surgery (MLS) (PG2 ENT/CMF) Foreign Body Excision (PG2 ENT/CMF) Total Hip Replacement Surgery (PG3 Orthopaedic) Total Endoscopic Coronary Artery Bypass (TECAB) (PG4 Cardiovascular) Mitral Valve Reconstruction (MVR) (PG4 Cardiovascular) Laparoscopic Splenectomy (PG5 Thoraco-abdominal) Laparoscopic Cholecystectomy (PG5 Thoraco-abdominal) Laparoscopic Nephrectomy left (PG5 Thoraco-abdominal) Angiography with PTA and Stent (PG6 Interventional Radiology) Hepatic Tumor Radio Frequency Ablation (PG6 Interventional Radiology) Trajugular Intrahepatic Portosystemic Shunt (PG6 Interventional Radiology)

35. CARS 2008 Computer Assisted Radiology and Surgery CARS / SPIE / EuroPACS 9th Joint Workshop onSurgical PACS and the Digital Operating RoomBarcelona, 28 June, 2008 12th Meeting of the DICOM Working Group WG 24 “DICOM in Surgery“ Barcelona, 28 June 2008 http://www.cars-int.org

37. WG24 “DICOM in Surgery” Secretariat: Howard Clark, NEMA Secretary: Franziska Schweikert, CARS/CURAC Office fschweikert@cars-int.org General Chair: Heinz U. Lemke, ISCAS/CURAC, Germany Co-Chair: Ferenc Jolesz, Harvard Medical School, Boston(Surgery/Radiology) Co-Chair: tbd (Industry)