Slicer Architecture

1. Slicer Architecture Steve Pieper, PhD

2. Overall Goals • Execution Environment • Command Line Args • Flow of Control, Startup Sequence • Data Model • Mrml Tree • Mrml Nodes • Mrml Data • Module Architecture • Standard Developer Widgets • GUI Modules • Editor Modules • I/O Modules

3. Execution Environment • Slicer runs inside a VTK Tcl/Tk shell • Uses the vtk executable that gets built when Tcl wrapping is enabled for the VTK build or the wish84.exe shell on Windows • vtkSlicerBase is a tcl package that gets pulled in at run time • Each Module is a tcl package • Note that vtkSlicerBase and the Modules can be wrapped for other languages if desired (python, java)

4. Tcl Packages • Includes Tcl code to be interpreted • Includes compiled C++ code as a shared library (optional) • Can reference and subclass C++ classes from other packages • Package path that can be used to find module-specific logos, data, etc

5. Command Line Arguments usage: slicer2-<arch> \[options\] \[MRML file name .xml | dir with MRML file\] <arch> is one of win32.exe, solaris-sparc, or linux-x86 \[options\] is one of the following: --help : prints this message and exits --verbose : turns on extra debugging output --no-threads : disables multi threading --no-tkcon : disables tk console --load-dicom <dir> : read dicom files from <dir> --load-analyze <file.hdr> : read analyze file from <file.hdr> --load-freesurfer-volume <COR-.info> : read freesurfer files --load-freesurfer-label-volume <COR-.info> : read freesurfer label files --load-freesurfer-model <file> : read freesurfer model file --load-bxh <file.bxh> : read bxh file from <file.bxh> --script <file.tcl> : script to execute after slicer loads --exec <tcl code> : some code to execute after slicer loads --all-info : print out all of the version info and continue --enable-stereo : set the flag to allow use of frame sequential stereo --old-voxel-shift : start slicer with voxel coords in corner not center of image pixel

6. Command Line Examples • slicer2-win32.exe mydata.xml • reads Mrml scene file after slicer starts • slicer2-win32.exe –no-threads • good for debugging ThreadedExecute methods • slicer2-win32.exe –load-dicom <dir> • loads all the dicom volumes from a study directory • slicer2-win32.exe –exec set a 100 • executes tcl code on command line after slicer loads • slicer2-win32.exe –script testscript.tcl • runs the tests script after slicer loads • slicer2-win32.exe testscript.tcl • runs the script instead of booting slicer GUI

7. Bootstrapping • slicer2-win32.exe • essentially the same as calling • slicer2-win32.exe launch.tcl • The launch.tcl script creates a subshell with correct environment variables • Then execs vtk or wish84.exe to start main slicer process with Go.tcl as a startup script

8. Flow of Control Outline • Base/tcl/Go.tcl is main entry point • Parse command line args • Starts tkcon console • Load vtkSlicerBase and all module packages • Uses tcl ‘package require’ command • Run MainBoot

9. MainBoot Startup Sequence • Init Slicer Base and Modules • MainInit – sets up base framework, appearance, etc • Create Main Viewer Windows • Call All <Module>Init Procs – set callback procs and defaults • Set up VTK • MainBuildVTK – makes the cube and letters • Call All Module procVTK – module-specific VTK instances • Build GUI • Build Main GUI • Call All Module procGUI – each module is given it’s tab frame • Read Options.xml – per-installation defaults • Read Command Line MRML scene if specified • MainSetup – set view parameter defaults, etc • Enter Main Tcl Event Event Loop

10. Global Tcl Arrays • Module() • Module(*) – state about modules • Module($m,*) – common state about module $m • <ModuleName>() • an array with the same name as the Module • Internal state of the module, including the variables that are tied to the GUI controls • Volume() and Model() • Two important modules • Volume($id,*), Model($id,*) • SLICER() • Command line argument values and version info • Gui() • Window handles and appearance settings • Anno() • Viewer annotations state and settings

11. Key VTK Class Instances • Slicer • Instance of vtkMrmlSlicer is “heart” of the volume display process • Mrml(dataTree) – note, this is a class instance of vtkMrmlTree, not an array reference • Hook to the scene data • viewRen • The vtkOpenGLRenderer in the Viewer

12. Module VTK Class Instance Naming Convention • Name of the Module with variable name in parentheses is used as name of VTK class instance • For Example: catch “SiemensMosaicReader(imageAppend) Delete” vtkImageAppend SiemensMosaicReader(imageAppend) SiemensMosaicReader(imageAppend) SetAppendAxis 2 • Note: delete class instances after use, but it is also good practice to do the catch “<inst> Delete” to make your module re-entrant • Note: the is Not standard Tcl practice – but this is used to avoid namespace conflicts between modules • This convention is widely used in the slicer Tcl code, but it can lead to confusion because the class instances look like Tcl array references. • Be careful!

13. Mrml Data Model • Tree-structured scene description • Accessible as Mrml(dataTree) • Serialized to XML file • Compiled to VTK class instances in memory for display and manipulation

14. Mrml http://www.slicer.org/mrml

15. Mrml XML <?xml version="1.0" standalone='no'?> <!DOCTYPE MRML SYSTEM "mrml20.dtd"> <MRML> <Transform> <Matrix name='manual' matrix='1 0 0 -3 0 1 0 -12 0 0 1 28 0 0 0 1'></Matrix> <Volume name='SPGR' filePattern='%s.%03d' filePrefix='spgr/I' rasToIjkMatrix='0 -1.06667 0 142.187 0 0 -1.06667 98.1333 0.666667 0 0 59.9667 0 0 0 1' rasToVtkMatrix='0 -1.06667 0 142.187 0 0 1.06667 157.867 0.666667 0 0 59.9667 0 0 0 1' positionMatrix='0 0 1 -89.95 -1 0 0 133.3 0 1 0 -148 0 0 0 1' description='LR' colorLUT='0' window='146' level='76' applyThreshold='yes' lowerThreshold='17' upperThreshold='355' imageRange='1 124'></Volume> </Transform> <Volume name='all' filePattern='%s.%03d' filePrefix='labels/all' rasToIjkMatrix='0 -1.06667 0 128 0 0 -1.06667 128 0.666667 0 0 62 0 0 0 1' rasToVtkMatrix='0 -1.06667 0 128 0 0 1.06667 128 0.666667 0 0 62 0 0 0 1' positionMatrix='0 0 1 -93 -1 0 0 120 0 1 0 -120 0 0 0 1' description='LR' colorLUT='-1' labelMap='yes' interpolate='no' window='4' level='1' lowerThreshold='0' upperThreshold='0' imageRange='1 124'></Volume> <Model name='Skin' fileName='models/Skin.vtk' color='Skin' visibility='yes'></Model> <Model name='Ventricles' fileName='models/Ventricles.vtk' color='Ventricles' visibility='yes'></Model> <Model name='Vessels' fileName='models/Vessels.vtk' color='Vessels' visibility='yes'></Model> </MRML>

16. Mrml XML • Describes the data as it exists on disk • Composes the scene by positioning data elements • Retains visualization and other state parameters

17. Data Model – the Mrml Tree • Mrml Tree is an in-memory data description corresponding to the mrml xml file • vtkMrmlNode subclasses contain the metadata • Mapping of metadata to run-time state is managed mainly in Tcl • vtkMrmlData subclasses are helpers for classes that need C++ help to efficiently manage runtime state

18. vtkMrmlNode

19. Mrml Nodes • vtkMrmlNode abstract base class • int ID; • char *Description; • char *Options; • char *Name; • char *Title; • virtual void Write(ofstream& of, int indent); • IsA vtkObject • Can be subclassed in run time loadable Modules

20. vtkMrmlModelNode • vtkMrmlModelNode Example Instance Variables // Strings char *ModelID; char *FileName; char *FullFileName; char *Color; // Numbers float Opacity; // Booleans int Visibility; int Clipping; int BackfaceCulling; int ScalarVisibility;

21. vtkMrmlModelNode - methods vtkMrmlModelNode -- Example Methods SetModelID with 1 arg GetModelID SetFileName with 1 arg GetFileName SetFullFileName with 1 arg GetFullFileName SetColor with 1 arg GetColor SetOpacity with 1 arg GetOpacity VisibilityOn VisibilityOff GetVisibility SetVisibility with 1 arg ClippingOn ClippingOff GetClipping SetClipping with 1 arg BackfaceCullingOn BackfaceCullingOff GetBackfaceCulling SetBackfaceCulling with 1 arg ScalarVisibilityOn ScalarVisibilityOff GetScalarVisibility SetScalarVisibility with 1 arg

22. vtkMrmlModelNode DTD • XML Representation Example: <Model name='Skin' fileName='models/Skin.vtk' color='Skin' visibility='yes'></Model> • DTD entry from Base/tcl/mrml20.dtd: <!ELEMENT Model (#PCDATA | Fiducials)> <!ATTLIST Model name NMTOKEN "" fileName CDATA #REQUIRED color NMTOKEN "" description CDATA "" opacity NMTOKEN "1.0" visibility (yes | no) "yes" clipping (yes | no) "no" backfaceCulling (yes | no) "yes" scalarVisibility (yes | no) "no" ignore (yes | no) "no" scalarRange NMTOKENS "0 100">

23. How Mrml Files are Read • MainMrmlReadVersion2.x • In Base/tcl/tcl-main/Parse.tcl • Reads xml file and returns tcl list of nodes and values • MainMrmlBuildTreesVersion2.0 • In Base/tcl/tcl-main/MainMrml.tcl • Converts list into vtkMrmlTree and vtkMrmlNode instances • Modules can define Module($m,procMRMLLoad) to extend Mrml

24. Compiling the Mrml Tree • MainUpdateMRML • In Base/tcl/tcl-main/Main.tcl • Is called after read or after changes to the Mrml Tree • Invokes Main*UpdateMRML • Models, Volumes, etc • Instances VTK classes, reads data, changes render properties, etc to make scene match the MRML Tree • Keeps Dev* GUI elements in sync with Tree and Selections • Invokes Modules($m,procMRML) for each Module

25. Rendering • To make the newly updated data visible to the user: • Render3D • Redraw just the 3D view • RenderSlice <s> • Redraw slice view 0, 1, or 2 • RenderSlices • Redraw all slice views • RenderAll • Redraw 3D and Slices

26. Editing the MrmlTree Examples • MainMrmlAddNode <nodeType> • Creates a new vtkMrmlNode instance and adds it to Mrml(dataTree) and returns nodeId • Use vtkMrmlNode methods to set data • Mrml(dataTree) RemoveItem $node • Remove a node • Mrml(dataTree) InsertAfterItem $nodeBefore $node • Add your new node to specific spot in the hierarchy • DevCreateNewCopiedVolume <OrigId> {Description ""} { VolName ""} • Create a copy node of existing volume with optional new name and description

27. Iterating Through the Mrml Tree set nitems [Mrml(dataTree) GetNumberOfItems] for {set widx 0} {$widx < $nitems} {incr widx} { set item [Mrml(dataTree) GetNthItem $widx] puts “$item is a [$item GetClassName]” }

28. Anatomy of A Module • See Modules/vtkCustomModule/tcl/@ModuleName@.tcl.in • For example, Custom • CustomInit • Analogous to a Constructor • CustomBuildGUI • CustomEnter / CustomExit • When user module pane is visible/hidden • Custom* • Any other “Methods” for the Module • These procs Edit the MrmlTree and call MainUpdateMRML and Render

29. The Shared Dev* Procs • Base/tcl/tcl-shared/Developer.tcl • Interact with MrmlTree • Use standard look and feel conventions • DevAddSelectButton • Automatically updating popup menu for Volumes, Models, etc • Adds entries when objects are created • DevUpdateNodeSelectButton • Updates to currently selected • DevAddButton • DevAddLabel • DevAddFileBrowse • DevInfoWindow, DevErrorWindow, DevWarningWindow… • Etc…

30. Event Management • The <Module>Enter and <Module>Exit procs can be used to set up custom mouse/keyboard bindings • Base/tcl/tcl-shared/Events.tcl • pushEventManager / popEventManager • Arguments are list of: • Widget, event, callback

31. Event Example In EMSegmentInit: foreach s $Slice(idList) { set widget $Gui(fSl${s}Win) append EMSegment(eventManager) " \ {$widget <Control-Button-1> {EMSegmentBindingCallback Sample %x %y}}” } In EMSegmentEnter: pushEventManager $EMSegment(eventManager) In EMSegmentExit: pushEventManager $EMSegment(eventManager)

32. Event Example (continued) EMSegmentBindingCallback calls EMSegmentReadGreyValue $x $y 1 In EMSegmentReadGreyValue: set s $Interactor(s) ;# get the slice window where user clicked scan [MainInteractorXY $s $x $y] "%d %d %d %d" xs ys x y ;# convert to pixels Slicer SetReformatPoint $s $x $y scan [Slicer GetIjkPoint] "%g %g %g" xIjk yIjk zIjk ;# convert to array coords set xIjk [expr int($xIjk)] set yIjk [expr int($yIjk)] set zIjk [expr int($zIjk)] # get the pixel data from the volume set ImageData [Volume($v,vol) GetOutput] set pixel [$ImageData GetScalarComponentAsDouble $xIjk $yIjk $zIjk 0]

33. Progress Bar • Progress Bar on Lower part of slicer control window • For a vtkProcessObject instance X x AddObserver StartEvent MainStartProgress x AddObserver ProgressEvent "MainShowProgress x" x AddObserver EndEvent MainEndProgress

34. Fiducials Example proc intensities {vol} { # Note: does not account for any transforms applied to Volume catch "m Delete" vtkMatrix4x4 m eval m DeepCopy [Volume($vol,node) GetRasToVtkMatrix] set ids [FiducialsGetPointIdListFromName default] foreach id $ids { set pt [eval m MultiplyPoint [FiducialsGetPointCoordinates $id] 1] set i [expr int([lindex $pt 0])] set j [expr int([lindex $pt 1])] set k [expr int([lindex $pt 2])] set imageData [Volume($vol,vol) GetOutput] set pixel [$imageData GetScalarComponentAsDouble $i $j $k 0] puts "fiducial $id value $pixel at $i $j $k" } }

35. Reader Modules Motivation • vtkMrmlVolumeNodes can be created by • User Interface • Reading in XML File • Programmatically • Once the Node exists, the reader proc provides the vtkImageData

36. Reader Modules • Managed by Base/tcl/tcl-modules/Volumes.tcl • ReaderModule needs an entry point proc named Vol<type>Init • For example: VolGenericInit • This will be invoked by VolumesInit • ReaderModule sets • Volume(readerModules,<type>,procGUI) • to fill in type-specific controls as sub-pane of Volumes Modules • Module(Volumes,readerProc,<type>) • so that MainUpdateMRML has a hook to call the reader when <type> if found in the FileType attribute of a vtkMrmlVolumeNode

37. Editor Modules Motivation • The Editor manages manually guided segmentation of a • source volume into a working labelmap • Editor Modules add to the palette of tools • Ed(editor) is an instance of vtkImageEditorEffects • Allows you to apply a filtering pipeline to modify the labelmap

38. Editor Modules • Similar to Readers, these are managed by Base/tcl/tcl-modules/Editor.tcl • Editor Module needs an entry point proc named Ed<type>Init • For example: EdWatershedInit • This will be invoked by EditorInit • EditorModule sets • Ed(<type>,procGUI) • to fill in type-specific controls on the this editor type’s own subframe of the editor • Segmentation is done In the UI control callbacks • Create a vtk (or vtkITK-based) processing pipeline of vtkImageToImageFilter subclasses • Set parameters from UI elements or Slicer data such as Fiducial locations • call Ed(editor) Apply <pipeline start> <pipeline end> • Ed(editor) manages progress bar, updating label map, undo (single)

39. Conclusions • Slicer provides a usable and extensible application framework for medical image computing research • A significant base code provides a reference for many common coding tasks and user interface paradigms

40. Resources • www.slicer.org • www.na-mic.org/Wiki • www.na-mic.org/Bug • www.na-mic.org/Testing • slicer-devel@bwh.harvard.edu • slicer-users@bwh.harvard.edu

41. Slicer Architecture SupplementAdding A New View Option Nicole Aucoin

42. Adding a new option example To add a new View option, X, to the Scene Options node so can save and restore it via Mrml file: • cxx/vtkMrmlSceneOptionsNode.h • add vtkGet/SetStringMacros(ViewX) • declare char *ViewX • cxx/vtkMrmlSceneOptionsNode.cxx • Add ViewX in constructor, Write, Copy, PrintSelf

43. Adding a new option, con’t • tcl/tcl-main/MainView.tcl • MainViewInit: add to Module(View,presets) list, set View(X) to same default value. • Add a new proc MainViewSetX, with default arg set to default value • MainViewStorePresets: add setting X's Preset from current value • MainViewRecallPresets: add setting X from preset value

44. Adding a new option, con’t • tcl/tcl-main/MainOptions.tcl • MainOptionsRetrievePresetValues: set Preset(View,n,X) from the node • MainOptionsUnparsePresets: set the node's X from the Preset

45. Adding a new option, con’t • tcl/tcl-main/MainMrml.tcl: • MainMrmlBuildTreesVersion2.0: add to parsing of SceneOptions node, refer to vtkMrmlSceneOptionsNode C++ code