Karabo: The European XFEL software framework

1. Karabo: The European XFEL software framework Burkhard Heisen 2nd Collaboration Meeting of the European XFEL April 24, 2013

2. What I always start with… A typical use case: Control drive hardware and complex experiments monitor variables & trigger alarms DAQ data readout online processing quality monitoring (vetoing) Sample Injector allow some control & show hardware status Undulator Accelerator Beam Transport show online data whilst running setup computation & show scientific results DM storage of experiment & control data data access, authentication authorization etc. Control DAQ SC processing pipelines distributed and GPU computing specific algorithms (e.g. reconstruction) DM SC Tight integration of applications Easy, efficient and comfortable usage of those Burkhard Heisen (WP76)

3. Outline – Follow the requirements Identify needs, Accept/Reject/Postpone several month before Proposalphase Prepare configurations, custom developments several month – few weeks before Preparationphase Setupphase Verify setup, find signal, alignments few days – few hours before Phase Independent Data storage, sample switching/delivery Data takingphase Remove custom h/w, bring s/w back to safe Clean upphase Data analysis workflows, data management Data analysisphase Data cleanup Publicationphase Burkhard Heisen (WP76)

4. Phase independent GUI username password currentTime location userId accessRole session Authorizes Computes context based access role Central DB Requirement • Distinction between different users with associated access restrictions Solution • Karabo is user centered, i.e. any access to control, (meta-)data, or computing infrastructure requires login. • Karabo interacts with a central database managing all user data. • Access is controlled using a role-based system. • Roles are assigned based on login context. The context involves username, current time and current location. • The same user for example may have more rights during his beamtime and at a specific computer than at other places and during different times. Burkhard Heisen (WP76)

5. Preparation phase Burkhard Heisen (WP76) Requirement • Setup of experiment configuration, scan macros, etc. Solution • XFEL system operators can prepare configurations and scan macros according to information of the proposal. • Configurations and macros are managed within Karabo’s central database. • The control system is fully scriptable. The default language is Python. • Control commands can be issued sequentially, or concurrently (e.g. move several axes at the same time) • Scripting can be done procedurally (python program) or interactively (IPython) • Being Python any complicated constructs can be coded (i.e. n-D scans with conditionals, etc.) • Scripting can be done within the GUI • Main API: get, set, monitor, execute, instantiate, kill

6. Preparation phase (special needs) Burkhard Heisen (WP76) Requirement • Implementation of new control devices and/or integration of new algorithms to analysis pipeline Solution • Karabo is split into a core framework and extension plugins (devices). • XFEL developers (or even the beamline users) can create new devices for Karabo that can be injected into the runtime and will enable new control or analysis capabilities. • Several code templates exist for different categories of devices. • Devices can be written in either C++ or pure Python (transparent to the Karabo runtime). • Devices impose a standard for self-description (properties, commands, state-machine). GUI widgets and behavior such as auto-completion and interactive scripting is automatically generated using this information. • Devices can be deployed at any computer and are automatically made available and ready to use for any control-clients.

7. Setup phase Burkhard Heisen (WP76) Requirement • Verification of h/w and s/w functionality, alignments, signal search, etc. Solution • Sanity functionality within Karabo ensures correct technical setup (hosts available, correct plugins loaded etc.) • XFEL beamline scientist can use the GUI for inspecting parameters, trendlines, histograms and images in conjunction with prepared macros to verify correct behavior • The GUI system uses the Device self-description to provide basic access to all features. • Custom panel creation needs no coding but is a process that is similar to the way you create a PowerPoint slide. • Panels can be loaded and saved per user. Sharing with other users is also possible.

8. Multi-purpose GUI system Configuration Custom composition area Navigation Logging / Scripting console Documentation Notifications Kerstin Weger (WP76)

9. Property/Command composition Display widget (Trend-Line) Editable widget drag & drop Display widget Kerstin Weger (WP76)

10. Property/Command composition Display widget (Image View) Display widget (Histogram) drag & drop Kerstin Weger (WP76)

11. Custompanels – PowerPoint like Open/Save panel view Cut, copy, paste, remove item Group items Change between “Design/Live” mode Bring to front/back Insert text, line, rectangle, … Rotate, scale item Kerstin Weger (WP76)

12. Data taking phase Burkhard Heisen (WP76) Requirements • Data collection, modification of variables, switching samples, online data analysis Solutions • Karabo is tightly integrated into the data management system. Within the GUI, experiment runs can be started and stopped. • Whilst running, Karabo monitors and saves all (slow-)control data which can later be associated to the experiment raw data. • Raw data is streamed to the PC Layer, formatted and is saved in the online storage as HDF5 files. • Karabo scientific workflow system can directly be fed with the raw data allowing highly parallel, pipelined processing whilst data taking. • The GUI allows monitoring experiment statistics and data-analysis runs. • Results of online data analysis can be used for tuning the experiment and vetoing bad data (those won’t even travel to the offline storage)

13. Device (workflow) composition Workflow node (device) drag & drop Draw connection Kerstin Weger (WP76)

14. Data analysis phase Burkhard Heisen (WP76) Requirements • Pre-analysis on-site up to preliminary results, full analysis off-site Solutions • The same workflow system as already available during data taking can be used in offline mode. Workflows are now fed by data as available from the offline storage (or - if still available - from the online storage). • Data analysis for the user starts with already calibrated data • Data calibration is also implemented as a workflow (but maintained by XFEL and accessing calibration parameters from Karabo’s central resources) • The analysis system will provide common image processing algorithms making also use of GPU technology • Community algorithms can easily be integrated to Karabo and shared amongst scientists. XFEL.EU plans to build up a central resource for state of the art analysis tools.

15. Processing workflows CPU IO whilst computing Pixel parallel processing (one GPU thread per pixel) Notification about new data possible to obtain GPU Burkhard Heisen (WP76)

16. Yes, it’s there! Burkhard Heisen (WP76)

17. Conclusions and timelines Challenges are important to learn about detailed requirements • pnCCD (combines control, DAQ, calibration and analysis) • Beckhoff test stands • Slice test (gives ideas about data throughput) • 2-tile LPD, later ¼ MpxLPD (imaging detector) • CrystFEL integration (data analysis workflows) First release of Karabo on 30thJuly • Will cover what was mentioned in this talk • Need more time to place the available technical solutions in larger contexts and workflows Burkhard Heisen (WP76)

18. Thank you for your kind attention. Burkhard Heisen (WP76)