From Image Registration in Oncology to Complex Workflows on the GRID

1. From Image Registration in Oncologyto Complex Workflows on the GRID Xavier Pennec, PhD, INRIA-Sophia, projet Epidaure Johan Montagnat, PhD, I3S, Rainbow team, Tristan Glatard, I3S, Rainbow + INRIA, Epidaure teams Pierre-Yves Bondiau, MD, PhD, Centre Antoine Lacassagne, Nice

2. Overview • The Medical application: • Registration for oncology • The scientific question: • Evaluation / comparison of registration algorithm performances • The technical challenge: • Running the workflow on the GRID AGIR - Sophia

3. Image Registration for Oncology • Registration / segmentation are basic components of medical image analysis • Registration: finding homologous points / tranformation • Segmentation: give anatomical label to each image point • Registration for brain radiotherapy • Planning • Fusion of image modalities (multimodal, rigid) • Warp atlas to patient image for segmentation(mono-modal, non-rigid) • Definition of Target volumes and Organs at risk: dose optimization • Follow-up (monomodal rigid) http://www.healthgrid.org/docs/pdf/WhitePaperdraft_v1.1-3reviewedv2.pdf(ch 3/4) AGIR - Sophia

4. Inter-subject registrationAffine transformation Image Registration for Oncology MR T1 Images 256x256x120 voxels Atlas to patient registrationfor radiotherapy planning Correct size and position but high remaining variability in cortex and deep structures AGIR - Sophia

5. Image Registration for Oncology Registration in 5 min on 15 PCs Anatomically meaningful deformation Adaptive non-stationary visco-elastic inter-subject registration AGIR - Sophia

6. Atlas Propagate the segmentation of structure of interest from the atlas to the patient image AGIR - Sophia

7. Image Registration for Oncology • Define target volume and organs at risk thanks to the segmentation • Optimize the irradiation process to • maximize the dose within the tumor • minimize it within neighboring organs at risk AGIR - Sophia

8. Image Registration for Oncology • There is no universal registration algorithm • More than 600 references on medical image registration in 1997 • More than 100 papers each year… (70 at MICCAI 2004 only) • Registration algorithms as Grid services • Use up to date algorithm • Evaluation / comparison of algorithm performances • Challenges • Inter-operability (coordinate systems, transformation format…) • Ontology describing data, registration problems and algorithms AGIR - Sophia

9. Overview • The Medical application: • Registration for oncology • The scientific question: • Evaluation / comparison of registration algorithm performances • The technical challenge: • Running the workflow on the GRID AGIR - Sophia

10. Variability of a registration algorithm • Fixed internal parameters • Multiscale resolution • (Typical variance…) • External parameters • Data (image) 1 • Data (image) 2 • Acquisition noise • Patient effects • Varying internal parameters • Initial transformation • (…) Registration algorithm Final transformation • Robustness: ability to find the right transformation (success/failure) • Precision: Repeatability w.r.t. some parameters (e.g. initialization) • Accuracy: Variability w.r.t. the ground truth for typical data AGIR - Sophia

11. Quantifying the registration errors • Uncertainty = deviation from the real transformation • Maximum error: bound • Mean Error: covariance matrix, std dev. • On the transformation ( rotation sr [rad], translation st [mm]) • On test points (TRE sx) • Robustness: • size of the basin of attraction • Probability of convergence AGIR - Sophia

12. Performance evaluation and validation • Synthetic data (simulation): • Available ground truth • Difficult to identify and model all sources of variability • Real data in a controlled environment (Phantom): • Possible gold standard • Performances evaluation in specific conditions • Difficult to test all clinical conditions • May hide a bias • Image database representative of the clinical application • Usually no ground truth • Should span all sources of variability AGIR - Sophia

13. Performance Evaluation without Gold Std • Bronze standard: The exact result is an unknown variable • Unbiased estimation: use redundant information • use many different registration algorithms(average biases, so that precision ~ accuracy) • Use many different data (redundant information to ensure precision) • Average transformations (maximal consistency) • Data intensive application: • High number of images across different databases • High number of registration algorithms AGIR - Sophia

14. Multiple a posteriori registration • Best explanation of the observations (ML) : • Robust Fréchet mean • Robust initialisation and Newton gradient descent • Result AGIR - Sophia

15. Example bronze std AGIR - Sophia

16. Performance Evaluation without Gold Std • Data intensive application: • High number of images across different databases • High number of registration algorithms • Grid validation protocol (PhD Tristan Glatard) • Find available data that match the problem description • Find the algorithms that can deal with them • Find and organize the resources to do the job AGIR - Sophia

17. Target image : - Image1 - Image2 - ... Floating image : - Image1 - Image2 - ... Registration algorithms Other components data links Crest lines extraction input Baladin output CrestMatch Format conversion Format conversion PFMatchICP Results management Yasmina Results management PFRegister Format conversion Format conversion Results management Results management Bronze Std workflow The bronze standard workflow AGIR - Sophia

18. Overview • The Medical application: • Registration for oncology • The scientific question: • Evaluation / comparison of registration algorithm performances • The technical challenge: • Running the workflow on the GRID AGIR - Sophia

19. Workflow manager • Workflow description • components / links • Taverna is the most powerful • Workflow Execution • Use the available parallelism (different notions of grid….) • Taverna has severe limitations • Control issues AGIR - Sophia

20. Workflow description • Description of processing components (web services) • Interface (e.g. WSDL), independent of their implementation • Example: <messagename="registrateWithCrestMatchRequest"> <partname="reference"type="xsd:string"/> <partname="floating"type="xsd:string"/> <partname="crest-ref"type="xsd:string"/> <partname="crest-float"type="xsd:string"/> <partname="input-comment"type="xsd:string"/> </message> <messagename="response"> <partname="result-image"type="xsd:string"/> <partname="result-voxel-transfo"type="xsd:string"/> <partname="result-real-transfo"type="xsd:string"/> <partname="reference-image"type="xsd:string"/> <partname="floating-image" type="xsd:string"/> <partname="comment"type="xsd:string"/> </message> <SOAP:addresslocation="http://colors.unice.fr:18002"/> AGIR - Sophia

21. Workflow description • Description of processing components (web services) • Interface (e.g. WSDL), independent of their implementation • Description is syntactic, not semantic • Description of links between components • Control links (from e-business): • BPEL4WS – WSCDL • Data links (from e-science) • Scufl (Taverna) – MoML (Kepler) <sequence> <flow> <switch> <while> <wait> BPEL tags <processor> <source> <sink> <link> Scufl tags AGIR - Sophia

22. Taverna • Chosen workflow management tool: Taverna • Developed in the UK project myGrid (bioinformatique) • Open source : http://taverna.sourceforge.net • Based on web-services • Most powerful workflow manager for description • Current research (e.g. in myGrid, UK) • Semantic annotation of services through ontologies • Automatic transcription into translating units • Limitation of translating units needed for algorithm compatibility • Systematic discovery of available components AGIR - Sophia

23. Set 0 Set 1 I0 J0 I1 J1 I2 J2 Set 0 Set 1 I0 J0 I1 J1 I2 J2 Ref Img Flo Img A0 A0 A1 A1 A2 A2 B0 B0 B1 B1 Taverna • Limitations of the data iteration strategy description • Scufl: • dot and cross products operators • In our case: register all images of • the same patient • the same modality • A different exam date AGIR - Sophia

24. Grid Resources EGEE User Interface Taverna workflow manager command line interface Registration Web-Service SOAP (over HTTP) ssh tunnelling C3 C1 C4 C2 D0 Taverna: Execution • Interaction of Taverna with the grid (EGEE) • Exloiting the parallelism of the workflow • Splits and synchronize, e.g. • C1: Initialization • C2: Register Algo 1 • C3: Register Algo 2 • C4: avarage results • Taverna is OK for one data… AGIR - Sophia

25. D0, D1, D2 • Synchronous interaction • Asynchronous interaction C3 C1 C4 Submission service Taverna Web-Service Grid Taverna Monitor1 Monitor2 Grid C2 query1 query1 query2 computation1 result1 Fetching service computation2 computation1 query2 result2 computation2 result2 result1 Exploiting parallelism • Data parallelism: • components are not multithread in Taverna! • Patch with submission/fetching services • Data order is not preserved (send 1/2/3, receive 3/1/2) • Need a track record for each result AGIR - Sophia

26. D0, D1, D2 C3 C1 C4 C2 Exploiting parallelism • Data + component parallelism: streaming (Pipelining) • Nw sequential steps, ND Data sets, Mean time T per component • Execution time = ND.Nw.T vs (ND+Nw-1).T • Example for registration: • nD = 50 ; nW = 4 ; T = 30min • Execution time = 100h vs 26.5 h • Streaming is not possible with Taverna AGIR - Sophia

27. A new workflow execution engine • Development of a new execution engine • compatible with Taverna description (Scufl) • Allowing data and Component parallelism • Implementing result traceability • Article submitted, soft to be available at http://www.i3s.unice.fr/~glatard AGIR - Sophia

28. Controlling the execution • Taverna and the new execution engine handle: • The traceability of results (execution tree for each data) • Taverna handles: • Re-submissions and delays • Alternative but predefined locations of web-services • Remaining issues • Nor Taverna nor EGEE handles • Job submission errors • Cancelled or lost jobs • Timeouts • How to do that without stopping the workflow execution? • Is it a middleware or a workflow manager issue? AGIR - Sophia

29. Conclusion - perspectives • Prototype of a new execution engine for Taverna • Exploiting streaming parallelism • Control of traceability • Open questions • Including ontologies • Granularity of jobs on the grid • Reliable interface with the EGEE infrastructure (timeouts/errors) • The Bronze standard application • Verification phase (standardization / converters) • Coupling with ontologies • Benchmark for • registration algorithms • Compression • Workflow execution engines on the grid AGIR - Sophia

30. References • Bronze Standard • Granger et al, MICCAI 2001 & ECCV 2002. • Nicolau et al, IS4TM 2003. • Worflows on GRIDS • T. Glatard & al. Grid-enabled workflows for data intensive applications. IEEE Int. Symp. On Computer-based Medical Systems CBMS’05. • T. Glatard & al. An optimized workflow enactor for data-intensive grid applications, Submitted to IEEE/ACM Intern. Work. On Grid Computing 2005 (associated to Supercomputing 2005). AGIR - Sophia

31. AGIR - Sophia

32. Grid registration services • Scenario 1: user accesses to registration services through the grid on his own data • Scenario 2: the user test his algorithm on standard image databases Computer resources Image data resources Registration service GRID User AGIR - Sophia

33. Grid registration services Interoperability challenges • Image format (input / output) • Dicom (communication module ?) • Basic 3D image format ? • Transformation formats • Standardized displacement field / resampled image • Internal representation + std resampling function • Algorithm parameters / options • Define std param. w.r.t. classes of registration problems • Interactivity • State of advancement (reporting) • Interactive corrections AGIR - Sophia

34. Grid registration services Ontology of Algorithms (registration service) • Type of data • Images (2D, 3D, time series) • Point clouds, landmarks • Type of spatial transformation • Rigid / similarity / affine • Non rigid (global / local) (splines, def. Fields, polyrigids…) • From Data to Transformation • Comparison metric (SSD, Correlation coefficient)takes into account the intensity transformation • Optimization procedure • Interactivity AGIR - Sophia

35. Grid registration services Ontology of Registration Problems (image databases) • Modality involved (specifies the type of data) • Monomodal (CT, MR, US, Video, point measures…) • Multimodal (combination of above) • Atlas to modality • Image content (specifies the type of transformation) • Anatomical part concerned (head, thorax, abdomen…) • Changes expected • intrasubject / intersubject / atlas • Smooth evolution / pathology AGIR - Sophia

36. Medical applications evaluation P-Y Bondiau Medical Apps. Interactive volume reconstuction A. Osorio Workflow Management J. Montagnat Medical data access protocols J-M. Moureaux Medical data Management J. Montagnat Services for Interactivity C. Germain Middleware evaluation E. Jeannot Les thématiques Cardiological images Segmentation I. Magnin Humanitarian Medical Development V. Breton Image registration in oncology X. Pennec Algorithm Gridification Dissemination C. Germain Core Grid Medical Services AGIR - Sophia