CMS Software Architecture

1. CMS Software Architecture Software framework, services and persistency in high level trigger, reconstruction and analysis Vincenzo Innocente CERN/EP

2. CMS (offline) Software Quasi-online Reconstruction Environmental data Slow Control Online Monitoring store Request part of event Store rec-Obj Request part of event Event Filter Objectivity Formatter Request part of event store Persistent Object Store Manager Object Database Management System Store rec-Obj and calibrations store Request part of event Data Quality Calibrations Group Analysis Simulation G3 and or G4 User Analysis on demand Software Architecture

3. Requirements (from the CTP, dec.`96) • Multiple Environments: • Various software modules must be able to run in a variety of environments from level 3 triggering, to individual analysis • Migration between environments: • Physics modules should move easily from one environment to another (from individual analysis to level 3 triggering) • Migration to new technologies: • Should not affect physics software module Software Architecture

4. Requirements (from the CTP) • Dispersed code development: • The software will be developed by organizationally and geographically dispersed groups of part-time non-professional programmers • Flexibility: • Not all software requirements will be fully known in advance • Not only performance • Also modularity, flexibility, maintainability, quality assurance and documentation. Software Architecture

5. Use Cases • Simulated Hits Formatting • Digitization of Piled-up Events • Test-Beam DAQ & Analysis • L1 Trigger Simulation • Track Reconstruction • Calorimeter Reconstruction • Global Reconstruction • Physics Analysis Software Architecture

6. Track Reconstruction Local measurements belongs to detector element and are affected by the detector element state (calibrations, alignments) Pattern recognition navigates in the detector to associate local measurements into a track Subject of T.Todorov & A.Vitelli talks Software Architecture

7. Global Reconstruction • Global reconstruction is performed in an absolute reference frame • 4-vector-like objects are built out of trajectories and localized energy deposits • A wide range of particle identification, jet, vertex, etc algorithms can be applied to produce others 4-vector-like objects • Access to the original detector data maybe required Software Architecture

8. Reconstruction Scenario • Reproduce Detector Status at the moment of the interaction: • front-end electronics signals (digis) • calibrations • alignments • Perform local reconstruction as a continuation of the front-end data reduction until objects detachable from the detectors are obtained • Use these objects to perform global reconstruction and physics analysis of the Event • Store & Retrieve results of computing intensive processes Software Architecture

9. Components • Reconstruction Algorithms • Event Objects • Physics Analysis modules • Other services (detector objects, environmental data, parameters, etc) • Legacy not-OO data (GEANT3) • The instances of these components require to be properly orchestrated to produce the results as specified by the user Software Architecture

10. ODBMS C++ standard library Extension toolkit Geant4 CLHEP PAW Successor LHC++ CARFCMS Analysis & Reconstruction Framework Application Framework Physics modules Reconstruction Algorithms Event Filter Physics Analysis Data Monitoring Calibration Objects Visualization Objects Event Objects Utility Toolkit Software Architecture

11. Architecture structure • An application framework CARF (CMS Analysis & Reconstruction Framework), customisable for each of the computing environments • Physics software modules with clearly defined interfaces that can be plugged into the framework • A service and utility Toolkit that can be used by any of the physics modules • Nothing terribly new, but... • Traditional architecture can not cope with • LHC-collaboration complexity Software Architecture

12. Problems with traditional architectures • Traditional Framework schedules a-priori the sequence of operations required to bring a given task to completion • Major management problems are produced by changes in the dependencies among the various operations • Example 1: • Reconstruction of track type T1 requires only tracker hits • Reconstruction of track type T2 use calorimetric clusters as seeds • If a user switches from T1 to T2 the framework should determine that calorimeter reconstruction should run first now • Example2: • The global initialization sequence should be changed because, for one detector, some condition changes often than foreseen Software Architecture

13. Framework Basic Dynamics • Avoid monolithic structure • Collection of loosely coupled mechanisms which implements • in abstract the tasks of a HEP reconstruction and analysis software. • Implicit Invocation Architecture • No central ordering of actions, no explicit control of data flow: only implicit dependencies • External dependencies managed through an Event Driven Notification to subscribers • Internal dependencies through an Action on Demand mechanism Software Architecture

14. Event Driven Notification Observers are instantiated by static factories residing in shared libraries. These are loaded on demand during application configuration Dispatcher Detector elements observe physics events Factories observe user requests Obs1 Obs2 Obs3 Obs4 Observers clients or providers Software Architecture

15. Action on Demand Compare the results of two different track reconstruction algorithms Rec Hits Rec Hits Rec Hits Detector Element Hits Event Rec T1 T1 CaloCl Rec T2 Analysis Rec CaloCl T2 Software Architecture

16. Reconstruction Object Model All persistent objects are managed by CARF. Physics Modules access them through standard C++ pointers Software Architecture

17. Framework Main Services • Define the events to be dispatched and link them to the their actual source • Allow the selection among available resources (user plug-in’s) • Integration with the DBMS Transparent use of persistent objects in physics modules • Manage the “not yet removed” sequential components (coming from legacy code & data) • The combination of Implicit Invocation and Run-Time Dynamic Loading allows a CARF application to configure and build by itself Software Architecture

18. Framework middle layer • A given application specializes a certain set of generic CARF mechanism to provide specific services • dispatch given events, loads specific libraries • A middle layer implements generic clients to such specific services • simplified API • uniform detailed design • uniform use of ancillary services • shield developers and users from CARF technical implementations • Requires synergy with detectors’ sub-systems Software Architecture

19. CARF layered Structure Core mechanisms and “data structures” Generic Application Simulation TestBeam G3 H2 T9/X5 Raw Data Raw Data Raw Data Generic Clients Software Architecture

20. CARF in Production • CMS Analysis&Reconstruction Framework is used in the present ORCA “functional prototype” • supports both detector and event reconstruction • provides a fully integrated persistency services based on a commercial ODBMS (Objectivity/DB) • is being validated by several applications ranging from test-beam (Daq and analysis) to high level trigger studies (digitization, reconstruction, event selection) Software Architecture

21. Conclusions • Traditional software architectures (main&subroutines, pipes&filters) have been found not to be adequate to CMS (multiple environments, evolving requirements, a long time-scale) • An “implicit invocation” architecture is a flexible software solution which can scale with the complexity of the CMS project. • ODBMS, integrated into the framework, • provides a coherent management of persistent objects • coupled withrun-time dynamic-loading, allows to automatically configure an application • The framework can effectively shield physics modules from the underlying technology without penalizing performances Software Architecture