The Software Framework available at the ATLAS ROD Crate

1. The Software Frameworkavailable at theATLAS ROD Crate DISCLAIMER: Focus on common software Might be biased by Level-1 Central Trigger THANKS TO: M. Joos, G. Lehmann Possible Replacement of VME for the Upgrades of ATLAS, CERN, 12-JUL-2012

2. Outline • Introduction • The Architecture Model • Ethernet – Single-Board Computer – VME • The Software Framework • A layered framework • Outlook Possible Replacement of VME for the Upgrades of ATLAS, CERN, 12-JUL-2012

3. Trigger/DAQ System Level-1 Trigger: Electronics + Firmware Readout Link (SLink) Level-2 Trigger + Event Filter: Computers + Networks + Software Level-1 Trigger VME RODs +Trigger Processors Level-2 Trigger PC Except ROIBuilder(VME) and ROBIN (SLink-PCI ) Event Filter Possible Replacement of VME for the Upgrades of ATLAS, CERN, 12-JUL-2012

4. The Architecture Model VME User (ssh) config SBC ROD TRIG PROC ROD_BUSY Ethernet Sub-detector specific Diagnostics status & monitoring ATLAS Run Control (TCP/IP) main data flow Readout Driver Crate: • Can also be a trigger processor or other front-end crate; can also contain modules other than RODs, e.g. LTP, TTCvi, ROD_BUSY, etc. • Provides interactive use, detector-specific diagnostic tools and ATLAS run controlthrough a single-board computer (SBC) • VME is used for configuration, status, and monitoring;the event or trigger data use other links (e.g. SLink or dedicated backplanes). • Transfer protocol (parallel memory bus): • single register READ/WRITE • memory/FIFO block transfer • interrupts Possible Replacement of VME for the Upgrades of ATLAS, CERN, 12-JUL-2012

5. The Software:A Layered Framework ROD Crate DAQ Low-level Software* Drivers and Libraries Operating System Hardware * Mainly sub-detector specific Possible Replacement of VME for the Upgrades of ATLAS, CERN, 12-JUL-2012

6. Framework: Hardware Provides common hardware infrastructure • VME: • Choice of crates, power supplies, and fans • Common guidelines for the use of VME64x • Common purchase & repair service (PH/ESE) • Common “slow control”: ATLAS DCS over CANbus • Single-board computers: • A family of three generations of single-board computers:Intel-based central processing unit and PCI/X-VME bridge • Additional Modules: • ROD_Busy Module • TTC Modules: TTCvi, TTCex, etc. • LTP, LTPI • Other modules available from the Electronics Pool which are usedin a ROD Crate framework, e.g. at test beam and in lab tests,and for which common low-level software exists Possible Replacement of VME for the Upgrades of ATLAS, CERN, 12-JUL-2012

7. Framework: Operating System Provides common operating system and environment • Common Service: • Provided by CERN/IT & ATLAS TDAQ System Administrators • Operating System  Scientific Linux CERN • Boot configuration (in particular loading DAQ drivers) • Security (at Point 1) • System/network monitoring (at Point 1) Possible Replacement of VME for the Upgrades of ATLAS, CERN, 12-JUL-2012

8. Framework: Drivers&Libraries - 1 Provides common drivers and software infrastructure • Common VME Driver (pure CERN product): • Dynamically loaded driver with static mappings (vmetab file) • Single READ and WRITE • FAST: uses mapped I/O (ignore VME bus errors,requires SBC with h/w for byte swapping) • SAFE: uses programmed I/O under driver control (catch VME bus errors) • Slave mappings (SBC memory as slave on VME) • To my knowledge not used • Block transfer • Allows several pipelined chains of block transfers. • Interrupt handling • Wait synchronously on semaphore or register POSIX signal with driver • To my knowledge not often used • Other features: • CR/CSR access, SYSRST generation, SYSFAIL and bus error signalling, and debugging features • Provides C library and C++ wrapper Possible Replacement of VME for the Upgrades of ATLAS, CERN, 12-JUL-2012

9. Framework: Drivers&Libraries - 2 Provides common drivers and software infrastructure • Other drivers and libraries: • PCI: • Read/write memory I/O • Link/unlink PCI devices • Could be reused in an xTCA environment. • Contiguous memory • Kernel pages (or previously BIG PHYS area) • Other libraries: • “Helpers”: • Errors codes, time stamps, bit strings, JTAG chains, simple menu programs,XML module templates, etc. • And their tools for testing and debugging Possible Replacement of VME for the Upgrades of ATLAS, CERN, 12-JUL-2012

10. Framework: Low-level Software Provides sub-detector specific software • Sub-detector specific diagnostic tools • for early development or debugging • Sub-detector specific calibration tasks • detailed knowledge of the calibration and front-end system • Libraries (C++) which “encapsulate” the specificitiesso that the ROD (or other modules) can be used from the next higher level This is probably the greater part of the software base. Common low-level software • Common libraries for common modules,e.g. LTP, TTCvi, ROD_BUSY, etc. Possible Replacement of VME for the Upgrades of ATLAS, CERN, 12-JUL-2012

11. Framework: ROD Crate DAQ Provides Integration into ATLAS DAQ: • Based on a skeleton program* (C++) which needs to befilled in with sub-detector specific calls+ a set of common modules ready to be used.* A ReadoutApplication originally developed for the ROS DAQ • Run Control: • Distributed system of finite-state machines (controllers) • Configuration Databases: • Run parameters, calibration data, trigger configuration, etc. • Monitoring: • Status/error messages, status values, histograms, event monitoring, data quality,persistent information, etc. • Ready-to-use Modules: • Controller, configuration, and monitoring are readily available for common modules(e.g. ROD_BUSY) and other DAQ tasks Is based on the same software as the rest of the ATLAS TDAQ (ROS, L2, and EF). Have to see how that one evolves! Possible Replacement of VME for the Upgrades of ATLAS, CERN, 12-JUL-2012

12. History From my recollection: • Detector Interface Group (DIG): • Define interface between sub-detectors and DAQ • Define DAQ functionality required by sub-detectors • TDAQ Interfaces with Front-end Systems Requirements Document+ Readout Link (SLink) and Raw Event Format • ROD Working Group (sub-group of DIG): • Exchange of information on ROD hardware • Define common solutions, e.g. VME hardware/driver & common modules • ROD Crate DAQ Task Force (sub-group of DIG): • Define common DAQ for the ROD crate → RCD Common solutions have proven to be useful in terms of effort for development and maintenance (in particular for smaller groups). Possible Replacement of VME for the Upgrades of ATLAS, CERN, 12-JUL-2012

13. Outlook If we intend to replace VME: • What will replace the existing services/infrastructure? • Common solutions in terms of common hardware/software? • Common service for purchase & repair? • Common DCS? • What architecture/protocol to use? • Architecture: Based on SBC which distributes commands to modules? Common driver and library? • Protocol: Single or grouped READ/WRITE? Block transfers? Interrupts? • Performance: What latency for control and throughput for monitoring are required? • Common ROD Crate DAQ (same for all sub-detectors? same as ROS DAQ)? • Common modules? • How to arrive at common solution? • Do we need another ROD Working Group? • What is the manpower required to develop and maintain common framework?And who can provide that manpower? Must not forget the VME legacy. It will be around until phase 2 and probably beyond. Possible Replacement of VME for the Upgrades of ATLAS, CERN, 12-JUL-2012