Slicer3 EMSegment Tutorial

1. Slicer3 EMSegment Tutorial January 2008 NAMIC All-Hand’s Meeting Brad Davis, Yuman Yuan, Sebastien Barre, Will Schroeder, Polina Goland, Ron Kikinis, Kilian Pohl

2. Program • EM Segmenter basics • Algorithm “views” in Slicer3 • Slicer3 EM segment under the hood • Tutorial • Data • Simple Segmentation Interface • Template Builder Interface • Questions (all week…)

3. EM Segment Basics Figure: Pohl et al.

4. EM Segment Basics Figures: Pohl et al.

5. EM Segment Basics Parameters Segmentation Result Target Images Atlas Images WM GM CSF Air Skin/Skull

6. EM Segment Basics Parameters • Collection of images (input channels) that will be segmented Segmentation Result Target Images Atlas Images T1 T2

7. EM Segment Basics Parameters • Hierarchy of anatomical structures • Intensity distributions, prior probabilities • Default preprocessing • Intensity normalization, registration • EM segmentation algorithm • Smoothing, weights, # iterations • Stored as XML MRML file Segmentation Result Target Images Atlas Images Root ICC Background Air Skin/Skull GM WM CSF

8. EM Segment Basics Parameters • Atlas images (spatial priors) • Aligned in common coordinate frame Segmentation Result Target Images Atlas Images CSF GM WM Air Skin/Skull

9. EM Segment Basics Parameters • Labelmap volume • Voxel values specified in parameters Segmentation Result Target Images Atlas Images Root ICC Background Air Skin/Skull GM WM CSF

10. Algorithm Development • 12-year history of algorithm development • Wells et al. 1996 (TMI): EM framework for simultaneous estimation of bias field and label map • Kapur et al. 1999 (PhD Thesis): Model noise via Markov Random Field • Van Leemput et al. 1999 (TMI): Non-spatial tissue priors • Pohl et al. • 2002 (MICCAI): Deformable registration to align atlas • 2004 (ISBI): Hierarchical framework to model anatomical dependencies • 2007 (TMI): Up-to-date, detailed description

11. Recent Development Effort • Goals: • Easy to use • Adapts to a variety of scenarios • Is a research tool • Simplified interfaces • Minimize preprocessing requirements • Validation and testing Wizard GUI Slicer3 Segmentation Code ITK/VTK Preprocessing

12. Slicer3 EM segment “views” • Advanced interface for creating/editing segmentation parameters • Simple interface for applying parameters to new images • Command-line interface for script/batch processing EM Template Builder Module EM Simple Module Slicer3 GUI Command Line EM Command Line Executable EMSegmenter Logic / Preprocessing / Algorithm

13. Template Builder View • Runs inside Slicer3 • Purpose: Build/Adapt parameter sets • Run segmentation • Wizard guides user through interface panels

14. `One-Click’ View • Runs inside Slicer3 • Purpose: Simple interface for segmenting new data • Requires predefined parameter set

15. Command-line View • EMSegmentCommandLine executable bundled with Slicer3 • Purpose: Scripted/batch processing • Requires predefined parameter set • Specify new target/atlas images on command line Parameters Segmentation Result Target Images Atlas Images

16. Slicer3 EM Segment Workflow Specify Inputs Parameters Target Images Atlas Images

17. Slicer3 EM Segment Workflow Specify Inputs Parameters Target Images Atlas Images Default Pre- Processing Target Image Normalization Atlas-to-target Registration Target-to-target Registration

18. Slicer3 EM Segment Workflow Specify Inputs Parameters Target Images Atlas Images Default Pre- Processing Target Image Normalization Atlas-to-target Registration Target-to-target Registration Segmentation EM Segment Algorithm: Pohl et al.

19. Slicer3 EM Segment Workflow Specify Inputs Parameters Target Images Atlas Images Default Pre- Processing Target Image Normalization Atlas-to-target Registration Target-to-target Registration Segmentation EM Segment Algorithm: Pohl et al. Review Results Slicer3 Slice Views Slicer3 Model Maker External Program

20. New Development: Default Preprocessing Default Pre- Processing • Lower bar for entry • Starting point for building effective pipeline • Not intended for all scenarios • No interface for complex parameter tweaking • Easily bypassed by advanced users Target Image Normalization Atlas-to-target Registration Target-to-target Registration

21. New Development: Default Preprocessing Default Pre- Processing • VTK filter by Kilian Pohl • Scale intensities to achieve predefined mean • Heuristics determine histogram cutoffs • Default settings for T1, T2 images Target Image Normalization Atlas-to-target Registration Target-to-target Registration

22. New Development: Default Preprocessing Default Pre- Processing • Target image 1 is fixed image • Perform registration for each additional target image • Align image centers • Rigid registration, mutual information, gradient descent optimization (ITK) • Coarse-to-fine Target Image Normalization Atlas-to-target Registration Target-to-target Registration

23. New Development: Default Preprocessing Default Pre- Processing • Target image 1 is fixed image • Perform registration with selected atlas image • Align image centers, Rigid registration, B-Spline registration • Mutual information, (normalized cross-correlation, sse, not yet tested) • Rigid: gradient descent, B-Spline: ITK LBFGSB optimizer, 5x5x5 knot points • Coarse-to-fine • Fast/Standard/Slow: sample points, max iterations • Resample all atlas images via transform; guess background value Target Image Normalization Atlas-to-target Registration Target-to-target Registration

24. Default Registration Results

25. Tutorial Data Walkthrough 1/3 • README: extra release notes • Predefined parameter file • MR T1 & T2 brain segmentation: White matter, gray matter, CSF • Target Images • ImageData_Input sub directory • Target: T1 & T2 images used in parameter file • Target_NewPatient1, Target_NewPatient2: New data to try with the parameter file

26. Tutorial Data Walkthrough 2/3 • Atlas Images • ImageData_Input sub directory • Atlas: atlas images used in parameter file • Atlas_NewAtlas: Extra atlas data for demonstration purposes (currently a copy of first atlas)

27. Tutorial Data Walkthrough 3/3 • Low resolution data for testing • Full sized data: 30 minutes • Low resolution data: 1 minute • Data, parameters have “_small” appended to filename

28. `One-Click’ Module Tutorial 1/3 • EMSegment tutorial data is required • Within Slicer3, open EMSegment Simple Module

29. `One-Click’ Module Tutorial 2/3 • MRML Scene: Select the tutorial parameter set • Result Labelmap: Select a new image filename • Target Volumes: Select input target volumes from one of the target directories (e.g., Target_NewPatient1) • Note: target volume order is important • Click `Apply’, wait (~30 minutes for full size images)…

30. `One-Click’ Module Tutorial 3/3 • Review the results in Slicer3 • Load target images • Load result labelmap • Use slice views to review segmentation results • Apply Model Maker to generate surfaces

31. EMSegment Command-line Examples • Script examples available in tutorial directory • New images • New atlas • New image and atlas • See README

32. Template Builder Tutorial • EMSegment tutorial data is required • Within Slicer3, open EMSegment Module

33. Template Builder Tutorial • Load tutorial parameters • File->Load Scene • Select ‘mrml’ file included with tutorial • Explore volumes included with scene • Target volumes, atlas volumes • An old segmentation result is also included

34. Template Builder Tutorial • Step 1: Choose EMSegmenter parameter set • For now, use tutorial set • Later use this interface to create a new parameter set

35. Template Builder Tutorial • Define anatomical structures • Right click to add or delete nodes • Label corresponds to eventual voxel values in segmentation result

36. Template Builder Tutorial • Choose an atlas volume for each leaf node • To change or add new atlas volumes • Load new volumes into Slicer3 • Select new volumes at this step

37. Template Builder Tutorial • Choose target images • To change or add target images • Load new volumes into Slicer3 • Select new volumes at this step • You can reorder target images; order is important

38. Template Builder Tutorial • You can choose to align target images • First target is fixed image • Rigid, mutual information registration

39. Template Builder Tutorial • You can choose to normalize target images • Simple, default strategy • Default parameter sets available from pulldown

40. Template Builder Tutorial • Intensity distributions define appearance of each leaf structure • Gaussian • Dimensionality equal to number of target images • Two methods • Sample voxels from images • Specify mean and covariance manually • Tip: sample first, then fine tune manually

41. Template Builder Tutorial • Sample voxels from images • Load first target image into Slicer3 slicer view • Choose anatomical structure • Choose `manual sampling’ • Ctrl-left-click on image to add voxels

42. Template Builder Tutorial • Sample voxels from images (continued) • To remove an unwanted sample right-click on it and choose “remove” • Change back to “manual” mode to tweak distribution

43. Template Builder Tutorial • Segmentation parameters for every tree node • Influence • Prior weight relative to other structures • Atlas • Input channels • Smoothing (parent nodes only) • Stopping Conditions

44. Template Builder Tutorial • Atlas-to-target registration parameters • Moving image registered to first target image • You can choose any image loaded into Slicer3 • Same transformation applied to all atlas images before segmentation begins

45. Template Builder Tutorial • Select output and run registration • You can troubleshoot preprocessing by saving intermediate results • Choose an output labelmap • ROI governs segmentation processing (one-based, not zero-based)

46. Acknowledgements • Wendy Plesniak, SPL BWH • Steve Pieper, Isomics • Luis Ibáñez, Kitware • Sylvain Bouix, PNL BWH • William Wells, BWH • Funding provided by NAMIC

47. Questions…

48. Slicer3 EM Segment Workflow Specify Inputs Parameters Target Images Atlas Images Default Pre- Processing Target Image Normalization Atlas-to-target Registration Target-to-target Registration Segmentation EM Segment Algorithm: Pohl et al. Review Results Slicer3 Slice Views Slicer3 Model Maker External Program

49. Parameters Segmentation Result Target Images Atlas Images Specify Inputs Parameters Target Images Atlas Images Default Pre- Processing Target Image Normalization Atlas-to-target Registration Target-to-target Registration Segmentation EM Segment Algorithm: Pohl et al. Review Results Slicer3 Slice Views Slicer3 Model Maker External Program