TELL1 - CONTROL Guidelines for Users & Developers

1. TELL1 - CONTROL Guidelines for Users & Developers Tutorial – User Guide

2. Contents: • framework components (fwCcpc, fwHw, fwTell1) • configuration of Tell1 boards • finite state machines • design issues - special requirements • user specific - what you still have to do Tutorial – User Guide

3. fwCcpc – in a nutshell: • interface between PVSS and CCPC using DIM and a generic server • basic test panels & PVSS library doxygen documentation and tutorial under: http://lhcb-online.web.cern.ch/lhcb-online/ecs/PVSS_CCPC/default.html Tutorial – User Guide

4. fwCcpc – in a nutshell: • communication via Ccpc DP • each Tell1 has its own DP and each bus has its own DP element • Note: use fwHw for accessing registers and not this DP! (model your hardware) Tutorial – User Guide

5. fwHw – in a nutshell: • a tool (panels & library) for modeling hardware as PVSS DPs and creating recipes. generic: SPECS & CCPC Tutorial – User Guide

6. fwHw – in a nutshell: • each register has its own DP and DIM srvcs/cmnds associated • subscription: - register settings transferred to ccpc and stored there in list. - create DIM server cmnds/srvcs. (establishes communication between server & PVSS) doxygen documentation and tutorial under: http://lhcb-online.web.cern.ch/lhcb-online/ecs/FwHw/default.html Tutorial – User Guide

7. Recipes – in a nutshell: • a recipe is a collection of registers, whose values can be stored in the configuration DB - from where they can be applied to hardware at any time. • Panels and framework functions to create and operate on recipes are provided. documentation under: http://lhcb-online.web.cern.ch/lhcb-online/ecs/FwHw/default.html Tutorial – User Guide

8. fwTell1 – in a nutshell: • datapoint representations of Tell1 types (including detector specific parts) • standard recipes of all recipe types (including detector specific part) To be imported from hardware: myTell1_’type’ • panels for device units (stopped counting ) – adopting automatically to Tell1 type • control unit with user interface • finite state machines Tutorial – User Guide

9. fwTell1 – in a nutshell: • use fwHw tool to instantiate your Tell1 boards (instantiate datapoints) • use DEN (device editor/navigator) to associate device unit with datapoint • create your FSM tree screenshot of Device Editor/Navigator Tutorial – User Guide

10. fwTell1: Tell1 control unit: pof – file: stores the firmware of FPGAs each board has its own ccpc cfg – file: server can access EPFL C-code to configure the boards Tutorial – User Guide

11. fwTell1: Tell1 control unit: Send down commands to enabled boards. (configure uses recipes e.g. “PHYSICS\Configure” Commonly modified configurations can be sent to enabled boards Tutorial – User Guide

12. A tab register is provided that allows you to access the common part of each Tell1 board. A tab for specific part is foreseen. The panel configures automatically according to board type. Some checks are performed. Watch the info buttons and tooltips for documentation. And many registers to be monitored … (a few hundred) fwTell1: Tell1 device unit: Tutorial – User Guide

13. fwTell1: Tell1 device unit: in fact a lot of panels … We provide tool-tips and info buttons for help! And perhaps a dedicated tutorial on device unit panels. Tutorial – User Guide

14. fwTell1: Tell1 device unit: This is where you can put your own panels! Tutorial – User Guide

15. fwTell1: We are not providing the monitoring of each sub-detector specific parameters. (but a few…) Tell1 device unit: Tutorial – User Guide

16. fwTell1: Tell1 device unit: We allow for spying on the data of the MEP buffer. The structure of the MEP buffer differs – we do not provide parsers for each sub-detector. Tutorial – User Guide

17. fwTell1: Tell1 device unit: Some examples for parsing the MEP buffer… Tutorial – User Guide

18. fwTell1: Tell1 device unit: We divide between 3 sub-recipes. Each sub-recipe has its own panel. You can export/import recipes or simply read them directly from hardware. Tutorial – User Guide

19. fwTell1: specific recipes: Define your specific recipe type. Create the appropriate panels. Tutorial – User Guide

20. fwTell1 – conclusion: • for commissioning you can just use the common part • BUT: start developing the specific part NOW! • NOTE: the specific part is related to your firmware. Get in touch with EPFL Lausanne before you start! You might profit from already existing things. doxygen documentation and tutorial under: http://lhcb-online.web.cern.ch/lhcb-online/ecs/PVSS_TELL1/default.html Tutorial – User Guide

21. Ctrl managers – in a nutshell: • fwCcpcScript.ctl: connects to running servers, automatic register subscription (crash/restart) • fwTell1FSM.ctl: needed to stop monitoring of registers when switching panels in DUs • fwTell1DisableArchives: disables the archive bit of registers  gain speed for configuration • fwFSMServer Tutorial – User Guide

22. Configuration of many boards: • trials and errors • benchmarking results Tutorial – User Guide

23. a snapshot from the talk of June 2007: Configuration of many boards: A first impression of scaling: 50 boards  ~ 6 minutes Tutorial – User Guide

24. first steps … black: old - no changes applied yellow/yellow: disabling archiving (but 35% CPU load from panels) l pink: disabling archiving & 1 ctrl manager per DU (still 35% CPU load from panels) green: disabling archiving (no CPU load from panels) red: disabling archiving & 1 ctrl manager per DU (no CPU load from panels) blue: reference when applying just specific part of recipes (disabled & 1 ctrl/DU) • disabling archiving significant • additional ctrl managers in FSM significant under high load • panels consuming much CPU? 50 boards  ~ 3.5 minutes Tutorial – User Guide

25. same as before – two (lower) lines added … black: checks on serverside - comparison of applied values with hardware values read back cyan:common part of recipes (16 registers of 32 bit) • complete recipe of type ST: • 236 registers • ~80 kB • grouping and packing together memory blocks would help, but not so easy to implement in FPGAs • changes on server significant but still too small 50 boards  ~ 2.5 minutes Tutorial – User Guide

26. so where does the time go … • most of the time consumed by DIM, Event and Data manager • in current design each register is sending a DIM command Tutorial – User Guide

27. parallelizing helps … … but not yet enough! Tutorial – User Guide

28. new approach – without changing the hardware representation … • getApplyRecipes() for fast Tell1 configuration: • retrieve recipes from configuration DB • pack them together in one buffer • send them as one buffer/DIM command • decode it on server Tutorial – User Guide

29. The buffer - array of unsigned char: register-names size of registers # registers … … \0 \0 & data checks buffer corruption … … & C C & command character Tutorial – User Guide

30. on the server side: … \0 & & C & parsing through buffer • get number of registers • get strings, compare them with list • fill array of pointers to list of registers (all hardware info stored) note: subscription is necessary! • check for corruption • fill array of register sizes • check for corruption • execute commands with appropriate data • final check Tutorial – User Guide

31. the results for getApplyRecipes(): red: one DIM command per register (old design) – but improved framework blue: sending one single buffer containing all registers (new design) grey: applying .cfg file 1.5 sec/board 1 sec/board no slope 50 boards  < 1 minute • most time gets lost when retrieving the recipes from DB • storing the whole buffer? I am optimistic that we could come closer to the .cfg file! Tutorial – User Guide

32. how to create recipes without typing several thousand parameters: • ‘get image from current board settings’ reads registers from hardware and saves them as a recipe • (you can upload the .cfg file before) • ‘import recipe from other hardware’ takes the recipe previously defined for another hardware. • (you can use ‘myTell1_type’ as default recipe) Tutorial – User Guide

33. Conlcusion: • creating recipes is not a big deal • (tools are available) • downloading recipes to hardware • is already fast & sufficient • we still support the interface • to the .cfg files • (original C-code) • Note: • some parameters are provided • by Gaudi jobs. • interface under development Tutorial – User Guide

34. Finite State Machines: towards a fully automatized expert system … Tutorial – User Guide

35. FSM – in a nutshell: Tell1 boards belong to the DAQ domain. Predefined transitions and states: Recipes are applied under ‘Configure’ – transition from NOT_READY to READY. Checks are done to verify the status of the board. If they fail status moves to ERROR. If connection to ccpc is lost, it moves to UNKNOWN. Tutorial – User Guide

36. FSM – in a nutshell: • functions to be called by FSM are provided in fwTell1.ctl • if needed you can provide a specific function to be called • by the FSM (e.g. for calibration) • fwTell1_Muon_Cmnd(string domain, string device, string • command, string parameter) Please read Integration Guidelines: https://edms.cern.ch/file/732486/3/LHCb_ECS_Guidelines.pdf and FSM Guidelines: https://edms.cern.ch/file/655828/3/LHCb_ECS_FSM_Guidelines.pdf Tutorial – User Guide

37. ECS FSM – from a higher point of view: Templates available: Please use them as they are Contact Clara for integration issues! INF DCS HV DAI DAQ L0 TFC HLT Storage LHC OT OTA OTC OTADCS OTAHV OTADAI OTADAQ OTACOOLING OTALV OTATEMP OTACRATES OTAFEE OTATELL1 Tutorial – User Guide

38. FSM – Run Control: • Run Control is like a Matrix, can exclude/include: • by SubDetector part • Or by Domain • “Standalone” run, need to allocate: • TFC • HLT • Storage • Monitoring • Allocate uses the Partition Settings • TFC, HLT, Storage, etc. will be ready soon. Tutorial – User Guide

39. N Steps FSM – Run Sequences: OTA OTA_DAQ TFC OTA 1 1’ 2 3 4 5 OTADAQ OTATFC 6 7 8 OTAFEE 9 OTATELL1 10 OTADev1 OTADev1 OTADev1 1 - Configure/RUN_TYPE=“CALIB_TIMMING” 1’ - Configure/RUN_TYPE=“CALIB_TIMMING”/N_STEPS=10 2 - READY 3 – Start 4 – RUNNING 5 - Start/N_TRIGGERS=10000 6 - RUNNING 7 - TFC Paused (after N_TRIGGERS) 8 - Step/RUN_TYPE =“CALIB_TIMMING”/STEP_NR=1 9 - RUNNING 10 - Continue/N_TRIGGERS=10000 OTADev1 OTADev1 OTADev1 OTADevN OTATell1N Tutorial – User Guide

40. FSM – Run Sequences: • The Activity can be saved, contains: • Nr Triggers • Nr Steps • Nr Sub-Farms, etc. • It corresponds to the RUN_TYPE that gets propagated down and used to form a recipe name by all devices. Tutorial – User Guide

41. FSM – Run Sequences: • Configure Command • As usual propagates down so that each device can apply a recipe (or more) based on: • RUN_TYPE (and N_STEPS) • Step Command • Again propagates down, should be accepted in state RUNNING (and return RUNNING) • Each device should change only the necessary values based on: • RUN_TYPE and STEP_NR Tutorial – User Guide

42. FSM – Conventions: • Recipe Names: • Guideline: <Run Type>/<FSM Command>, ex.: • PHYSICS/Configure, PHYSICS/Start • CALIBRATION/Configure, etc. • Necessary because a Configurator Object will preload recipes from DB to cache Tutorial – User Guide

43. few comments on design: • started with a generic/altruistic server • more and more intelligence moved to server side Tutorial – User Guide

44. clear separation between hardware and control issues – a ‘smarter’ server: • server keeps track of all the information about the register (e.g. type, address,…) and stores it in a list • server treats each register in the appropriate manner • (e.g. Fifo vs. standard blockwrite or different I2C derivates) • Once the register is defined the control system (PVSS) knows nothing about the hidden details and just writes to or reads from a register •  less and more readable code & better performance (interpreted language) server-side PVSS-side Intelligence related to hardware Intelligence related to ECS Tutorial – User Guide

45. how to keep the design open: • a ‘commonOperation’ part of the Ccpc DP allows to send a string to the server. Anything can be done. • Identifier and parameters are separated by comas. • a ‘user specific’ register type which offers many parameters to be filled in inside the fwHw tool. Any type of register can be modeled. • the new recipe block executed via ‘configurationOperation’ on the Ccpc DP is in fact a set of instructions, • which in addition can be stored in the Configuration DB. … \0 & & C & commands Tutorial – User Guide

46. What is left for the sub-detectors? in fact not much … • Specific part has to be inserted into datapoint type • The panels related to the specific part: • - MEP buffer spy • - Monitoring of specific registers/values • - Recipe related things • specific functions to be called by FSM (e.g. for calibration) • preferably packed as a library (obey to naming conventions) • Scripts inside device units might have to be extended … but still a bit! Tutorial – User Guide