OpenMAF 2

1. OpenMAF 2 Paolo Quadrani – p.quadrani@cineca.it Roberto Mucci – r.mucci@cineca.it Sistems and Technologies Department

2. WHAT’S MAF The Multimod Application Framework (MAF) is: “An open source freely available framework for rapid development of applications, based on theVisualization Toolkitand other specialized libraries”. It provides high level components that can be easily combined to develop a vertical application in different areas of scientific visualization.

3. OPEN MAF HISTORY • FP5 Multimod Project => MAF v1.0 • Only biomedical • Monolithic framework • Not open-source project • CINECA MAF v1.0 => MAF v2.0 • Generalisation to other domains • Massive refactoring • Reorganisation in MafCore, MafMedical, Application • Launch of openmaf.org Open Source project • LHDL Project MAF v2.0 => MAF v2.2 • Data-remote VME • Execution-remote VME • Support for LHDL WS and SWS

4. MAF FEATURES • C++ based code • Cross platform using CMake Tool: Windows VS .NET 2003/2005/2008, Linux makefiles • Integrates many specialized libraries (VTK, wxWidget, ITK, Crypto++, XercesC, Curl) • Straightforward support for different kind of data inside a single data space (support for 4D hierarchical data) • Support for multiple visualization of the data tree. • Support for non conventional interaction devices: trackers, haptics. • Extended support for collaboration on top of Access Grid. • http://www.openmaf.org

5. MEDICAL APPLICATIONS Registration between skeletalanatomy extracted form CT data, with movement analysis information (kinematic + force plates) LHDL Intellectual property right. Data IOR and ULB.

6. MEDICAL APPLICATIONS (2) • Registration of data coming from different sources: • Skeleton from TC • 3Dsurfaces of muscles from magnetic resonance • Markers: insertions and muscle’s origins from anatomical dissection of a cadaver LHDL Intellectual property right. Data IOR and ULB.

7. GEOPHYSICS APPLICATIONS Volcanic eruption simulation

8. Input Data Views VME Vertical application Interaction Operations Input Devices MAF ARCHITECTURE

9. MAF STRUCTURE App Logic View Manager Operation Manager VME Manager Interaction Manager Views Operations VMEs Devices + Interactors

10. App Logic View Manager Operation Manager VME Manager Interaction Manager Views Operations VMEs Devices THE ‘APP’ CLASS • Create the parent frame of the application • Make an instance of the logic • Plug the views and the operations used inside the application • Hide/Show the interface elements for the vertical application • Terminate the application deleting the logic

11. App Logic View Manager Operation Manager VME Manager Interaction Manager Views Operations VMEs Devices LOGIC: APPLICATION’S BEHAVIOR • Create and instance of each manager: mafViewManager, mafOpManager, mafVmeManager, mafInteractionManager • Create the menu’ • Create the toolbar • Create the sidebar • Create the timebar • Create the log area • Listen all the events raised from the managers and from the interface elements • Call (virtual) methods to mange the events

12. App Logic View Manager Operation Manager VME Manager Interaction Manager Views Operations VMEs Devices mafViewManager • Create/Delete plugged views • Manage view selection • Add/Remove VMEs to/from the views • Has knowledge about the selected view and selected VME

13. MAF VIEW • All VMEs are visible inside each view • View ask to VME for its own visual pipe, and use it to render the VME • This association can be customized by plugging inside the view a custom visual pipe for a specific VME

14. Surface DRR RX-CT VISUAL REPRESENTATION IN MAF OrthoSlice Volume Rendering

15. VME-VIEW ASSOCIATION • View may be compound and have interactive elements • The User associate VME to Views through the Icons in the Tree which reflect the status of the selected view • Views may support or not the various VME types non supported supported Supported (mutual exclusive)

16. App Logic View Manager Operation Manager VME Manager Interaction Manager Views Operations VMEs Devices mafOpManager • Manage the operation execution • Manage the undo stack • Enable/Disable menu items and toolbar • buttons according to the VME selected and to the operation execution

17. SOME OPERATIONS INSIDE MAF • mafOpImporterMSF • mafOpImporterVTK  • mafOpImporterMesh • mafOpExporterMSF • mafOpExporterBmp • mafOpExporterRAW • mafOpCreateGroup • mafOpAddLandmark • mafOpCreateSurfaceParametric • mafOp2DMeasure • mafOpBooleanSurface   • mafOpClipSurface • mafOpReparentTo • mafOpConnectivitySurface • mafOpCopy  • mafOpExtractIsosurface • mafOpDecimateSurface    • mafOpDecomposeTimeVarVME • mafOpFilterSurface • mafOpMAFTransform • mafOpOpenExternalFile   • mafOpEditMetada • mafOpEditNormals  

18. App Logic View Manager Operation Manager VME Manager Interaction Manager Views Operations VMEs Devices vector analog mafVMEManager • Add/Remove a VME to/from the tree • Manage the file save, open, new • Manage the file history • Set the time to the VME tree DataTree Type of VME currently supported

19. VME INSIDE MAF • mafVMESurfaceParametric • mafVMEVector   • mafVMEExternalData    • mafVMEGizmo    • mafVMEGroup    • mafVMEInfoText    • mafVMEMesh    • mafVMEPolyline    • mafVMEPolylineSpline    • mafVMERefSys • mafVMEVolume • mafVMEVolumeGray • mafVMEVolumeRGB • mafVMEImage • mafVMELandmark • mafVMELandmarkCloud • mafVMEMeter • mafVMEPointSet • mafVMERefSys • mafVMEScalar • mafVMESlicer • mafVMEProber • mafVMESurface

20. Parent VME VME VME 1 MAF View Link 4x4 pose matrix Matrix Matrix Array PIPE Render VME 2 Visual Link 3D Data Array Window PIPE dataset Data PIPE VME 3 Attributes Link Current MAF Time Operation VME GUI VME 4D DATA STRUCTURE VME represents 4D data, positioned in space with a hierarchical organization

21. Parent VME VME VME 1 MAF View Link 4x4 pose matrix Matrix Matrix Array PIPE Render VME 2 Visual Link 3D Data Array Window PIPE dataset Data PIPE VME 3 Attributes Link Current MAF Time Operation VME GUI VME 4D DATA STRUCTURE (2) VME stores array of pose matrixes (trajectories), datasets, and metadata attributes

22. XML Hierarchy VME Tree <MSF Name="legs"> <VME Name="L-Femur"> ... <VME Name="L-Tibia"> ... Storage </VME> </VME> <VME Name="L-Femur"> ... </VME> </MSF> VME STORAGE The VME tree is stored inside a folderor zip-file as a single XML file (MSF file) storing the hierarchy, pose and metadata, plus data files of single nodes (VTK datasets or zip of datasets array).

23. Parent VME VME VME 1 MAF View Link 4x4 pose matrix Matrix Matrix Array PIPE Render VME 2 Visual Link 3D Data Array Window PIPE dataset Data PIPE VME 3 Attributes Link Current MAF Time Operation VME GUI VME OUTPUT VME Output: 4x4 matrix and VTK dataset

24. Parent VME VME VME 1 MAF View Link 4x4 pose matrix Matrix Matrix Array PIPE Render VME 2 Visual Link 3D Data Array Window PIPE dataset Data PIPE VME 3 Attributes Link Current MAF Time Operation VME GUI VME OUTPUT CONSUMERS Consumers: Visual Pipes and Views

25. Parent VME VME VME 1 MAF View Link 4x4 pose matrix Matrix Matrix Array PIPE Render VME 2 Visual Link 3D Data Array Window PIPE dataset Data PIPE VME 3 Attributes Link Current MAF Time Operation VME GUI VME INPUTS VME output can change because of GUI parameters or current time changes.

26. Parent VME VME VME 1 MAF View Link 4x4 pose matrix Matrix Matrix Array PIPE Render VME 2 Visual Link 3D Data Array Window PIPE dataset Data PIPE VME 3 Attributes Link Current MAF Time Operation VME GUI VME LINKS Links to other VME can be added to extend the tree into a Graph.

27. PROCEDURAL VME • Procedural VMEs are sort of persistent operations which are saved in the data tree. • An example of procedural VME is the VME Meter: it’s a VME continuously computing the distance in between linked VMEs. • Procedural VMEs has not binary data associated, but produce data according to its algorithm or the one defined into a data-pipe • All VME in MAF are indeed procedural, data VMEs have a “standard” data pipe for reading data from the storage.

28. App Logic View Manager Operation Manager VME Manager Interaction Manager Views Operations VMEs Devices mafInteractionManager • Make a default instance of mouse device • Manage interaction devices through DeviceManager • Make an instance of the Positional Event Router (PER), used to interact dynamically with a device and the selected VME through VME’s behavior • Make an instance of the Static Event Router (SER), used to interact statically with a device and a fixed VME

29. MAF INTERACTION MODEL • The typical MAF interaction model is composed by: • an event source (the device), • an interpreter (the interactor) • and a logic module (operations, logic or others) Device Interactor Operation

30. Inter. Manager Speech Mouse Device Manager S.E.R. Tracker Haptic Sync Events Async Events EVENTS FROM DEVICE Events coming from input devices are synchronized with the main application thread by Device Manager. • Device Manager is responsible for devices: • Creation • Storing/Restoring of device setup • Input events synchronization

31. Logic Inter. Manager Op Manager Running Op S.E.R. Static Event Routing StaticInteractor Tracker Action INTERACTION WITH MAF Static Event Routing Interaction with devices is called “Static Event Routing”, where devices are assigned directly to “actions” by user, and interactors are connected to “actions”by application:

32. Current View S.E.R. Camera Interactor Head Track Action INTERACTION WITH MAF (2) SER Example Tracking device assigned to changing the point of view in the scene.

33. INTERACTION WITH MAF (3) • Positional Event Routing • P.E.R. is an interactor statically assigned to “PointAndManipulate” action. • Picking in the scene occurs only in response to “button down” events, therefore to be used with the P.E.R. an input device must be able to rise a button event. • P.E.R. routes events toward interactors attached to single VMEs (“behavior”). • P.E.R. and “behaviors” allow to define how interaction should occur when we click on a object in the scene.

34. Thank You!!