1 / 29

CMS Core Software for Physics Applications

CMS Core Software for Physics Applications. Software services for Simulation, High Level Trigger, Reconstruction and Analysis. Vincenzo Innocente CERN/PH/SFT. CMS ( offline ) Software. Quasi-online Reconstruction. Environmental data. Slow Control. Online Monitoring. store.

dhaas
Download Presentation

CMS Core Software for Physics Applications

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. CMS Core Softwarefor Physics Applications Software services for Simulation, High Level Trigger, Reconstruction and Analysis Vincenzo Innocente CERN/PH/SFT

  2. CMS (offline) Software Quasi-online Reconstruction Environmental data Slow Control Online Monitoring store Request part of event Store rec-Obj Request part of event Event Filter HLT Request part of event store Distributed Persistent Data Store Manager Store rec-Obj and calibrations store Request part of event Data Quality Calibrations Group Analysis Simulation G3 and or G4 User Analysis on demand Core Application Software

  3. Requirements (from the CTP, dec.`96) • Multiple Environments: • Various software modules must be able to run in a variety of environments from level 3 triggering, to individual analysis • Migration between environments: • Physics modules should move easily from one environment to another (from individual analysis to level 3 triggering) • Migration to new technologies: • Should not affect physics software module Core Application Software

  4. Requirements (from the CTP) • Dispersed code development: • The software will be developed by organizationally and geographically dispersed groups of part-time non-professional programmers • Flexibility: • Not all software requirements will be fully known in advance • Not only performance • Also modularity, flexibility, maintainability, quality assurance and documentation. Core Application Software

  5. Architecture Overview Data Browser Generic analysis Tools GRID Distributed Data Store & Computing Infrastructure Analysis job wizards LCG tools COBRA ORCA OSCAR IGUANA FAMOS Detector/Event Display CMS tools Federation wizards Coherent set of basic tools and mechanisms Software development and installation Consistent User Interface Core Application Software

  6. ReconstructionAlgorithms EventFilter PhysicsAnalysis DataMonitoring (Grid-aware) Data-Products LCG Component ArchitectureFramework Layering Specific Frameworks Grid-Uploadable Physics modules Calibration Objects Generic Application Framework Configuration Objects Event Objects Adapters and Extensions Basic Services (O/R) DBMS GEANT4 Interactive Analysis Tool HEP Foundation Libraries C++ Standard Library + Extension Toolkits Core Application Software

  7. Software Development Process • Baseline • Contribute to the design and implementation of common software as part of the LCG project • Adopt common component and integrate them into our framework and applications • Milestones (timescale and features) must be negotiated with LCG and the other LHC experiments • Large use-case coverage, great flexibility, more complexity, some overhead • Fallback • Do it ourselves • Full control and ownership of design, implementation and maintenance • Complete freedom in defining milestones • Streamlined implementations of limited use-cases relevant to us NOW Core Application Software

  8. Contribution to LCG Application Area • SPI : • OVAL • SCRAM • SEAL: • Contribution to general design • Implementation of Foundation classes (re-engineering of iguana classlib) • Implementation of core framework services (plug-in mechanism) • Mathlib • POOL: • Responsibility of FileCatalog • Implementation of Collections based on root-tree • Contribution in the Relational storage service • Debug and testing • PI: • Project Leadership • Interface to POOL • Simulation: • Leadership of and contribution to the Generator task • Contribution to Simulation validation • 3D (Distributed DataBase Deployment) : • Responsibility in the Proxy system (Frontier project) Core Application Software

  9. Simulation, Reconstruction, Analysis Software Structure FAMOS Fast Simulation IguanaCMS Detector&Event Visualization ORCA Detector Reconstruction HLT Physics Analysis OSCAR Detector Simulation G3 OO Interface Mantis G4 Simulation Framework { CARF Reconstruction Framework DDD Detector Description Framework COBRA Core Framework(s) Core Services Iguana Core Visualization GUI and Scripting Services Persistency Layer Profound PRS Foundation Application Infrastructure Generator Interface Core Application Software

  10. Core Software components • Foundation • Persistency • Event processing Framework • Book-keeping, Configuration & Conditions • In collaboration with Data Management • Frameworks and tools specific for • Simulation • In collaboration with SPROM • Reconstruction • In collaboration with RPROM • Analysis • In collaboration with APROM • Visualization • In Collaboration with APROM Core Application Software

  11. OSCAR overview • Object Oriented Simulation for CMS Analysis andReconstruction • Full CMS simulation based on the Geant4 toolkit • Geant4: physics processes describing in detail electro-magnetic and hadronic interactions; tools for the CMS detector geometry implementation; interfaces for tuning and monitoring particle tracking • CMS framework:application control, persistency, common services and tools (magnetic field, generator interfaces and support for MC truth, infrastructure for hits and readout units,…), “action on demand” to selectively load desired modules, configure, tune application • CMS changed from CMSIM/GEANT3 to OSCAR/GEANT4 end 2003; • OSCAR used for substantial fraction of DC04 production; will be used for physics TDR production • CPU:OSCAR  1.5 x CMSIM- with lower production cuts! • Memory: ~110 Mb/evt for pp in OSCAR  100 Mb in CMSIM • Robustness: ~1/10000 crashes in pp events (mostlyin hadronicphysics) in DC04 production to 0 crashes in latest stress test(800K single particles, 300K full QCD events) Core Application Software

  12. G4FLASH Parameterized Simulations Geant46.2 - fullvsfast Implementation of fast EM shower simulation in Geant4/OSCAR, using GFLASH parameterized showers (spot density) - tuning in progress… 50 GeV electrons Timing studies Core Application Software

  13. Heavy Ion Simulation 55K generated particles, with 97K tracks from 80K vertices kept at the end of event performance optimization with a twist… G3/CMSIM: chop event in slices of 100 tracks each and run them separately; needed due to limitations from ZEBRA OSCAR/Geant4: run full HI events Factor 5 performance improvement by improved calorimeter track selection and hit processing Event cut in slices of 100 particles Full event time/evt in given machine (*) 2.3 CPU hrs on P4 3.2 GHz …effect entirely negligible in pp events! Core Application Software

  14. IGUANA Applications and Frameworks Large-scale reuse of object-oriented libraries requires frameworks Run-time environment Uniform user view of the CMS visualisation applications for ORCA, OSCAR, COBRA, Geant4, Mantis, DDD, and test-beams. Distribution Integration The framework provides a context for the components in the library to be reused Uniform transparent data access Coherent user view Non-intrusive, loosely-connected frameworks Wrapped loosely coupled components

  15. CMS Data • Event-Collection Meta-Data • Environmental data • Detector and Accelerator status • Calibrations, Alignments (luminosity, selection criteria, …) • … • Event Data, User Data User Tag (N-tuple) Tracker Alignment Event Collection Ecal calibration Tracks Collection BookKeeping Event DataSet DataSet BookKeeping Electrons Event Navigation is essential for an effective physics analysis Complexity requires coherent access mechanisms Core Application Software

  16. Hybrid Store • CMS vision of an Hybrid Store is of a Data Management System whose components do not come all from a single “vendor” • Use best technology for each “domain” • ROOT trees for event data • XML for simple descriptions and as light exchange format • DBMS for robust management of configuration and conditions • “Grid” technologies to distribute data • It is User/CMS/LCG/HEP responsibility to glue all together • Challenge: produce a Data Management System • POOL as access layer • Coherent Application Interface • Flexible data “transfer” protocols (rfio/xrootd/frontier) • Common CMS Book-Keeping system • Support of native tools for experts/browse/debug Core Application Software

  17. Technical items on Critical Path • Access to RDBMS from physics-applications • Required to access Conditions/calibrations, configuration and Metadata • Proposal is to follow Frontier model • Interactive access to ORCA objects • Root is currently the most popular interactive analysis tool in HEP • We must provide easy access to ORCA objects from the Root prompt • Two solutions (could coexist) • ORCA writes a data structure that Root can understand in native mode • ORCA writes in root-tree using “simple” data structures • Useful for simple data inspection • We develop a root-plugin to use ORCA from Root (or Root in ORCA) • LCG/AA provides plugin for pool and mechanisms to populate CINT dictionary • We provide a root-binding from transient COBRA/ORCA classes • Useful for analysis of “medium” complexity • Requires changes to CMS Event Model Core Application Software

  18. Database types • We will have 4 kinds of data classes in CMS: • Construction data • All information about the sub detector construction up to the start of integration • Equipment management data • Mainly hardware description and location history • Configuration data • All information required to bring the detector in any running mode • Conditions data • All information about detector conditions Core Application Software

  19. Construction data • Sub detector specific • Sub detector responsibility • Already existing for most sub detectors. • Have to be available for the lifetime of CMS • Data: construction info, initial calibration data, work flow info, etc. • Data will partially be copied into the 3 other data classes and then frozen in the construction DBs. Core Application Software

  20. Equipment management data • Common for all CMS • Set up by CMS integration, data entered and maintained by sub detectors • Data: detector geometry and electronics parts, cables, racks, crates, location history of all items, etc… • User interface existing (rack wizard) to enter required information. • Needs time stamping. Core Application Software

  21. Configuration data • Sub detector specific parts on front end electronics configuration, combined DAQ, Trigger and DCS info. • Set up by corresponding groups • Interface to online software framework XDAQ existing • First prototype of interface to DCS existing • Needs time stamping, versioning and tagging Core Application Software

  22. The conditions data • All parameters describing the run conditions and logging. • Most of the conditions data stays at the experiment and is NOT used for offline reconstruction • Set up by corresponding groups • Conditions data is the logging of the online system • Conditions data will be used for error tracking in the online system • Conditions data will be related to • Other conditions data • Event quality monitoring • Configuration data • … Core Application Software

  23. Data flow Online Master Data Storage Offline Reconstruction Conditions DB ONline subset Equipment Configuration Conditions Create rec conDB data set Calibration Conditions Conditions Configuration Master copy Offline Reconstruction Conditions DB OFfline subset Bat 513 Core Application Software

  24. DataBase Work packages • OMDS • Data maintenance tools • Data visualization tools • “Export” to ORCON • ORCON • Transfer to/from ORCOF • ORCOF • Data distribution to/from TierX • ORACLE replication • FroNtier Core Application Software

  25. FroNtier Overview C++ Header and Stubs Client FroNtier Client API Library HTTP Caching Squid Proxy/Caching Server HTTP CDF Persistent Object Templates (Java) XML Server Descriptors FroNtier Server FroNtier Servlet running under Tomcat JDBC Database (or other persistency service) DDL for Table Descriptions Database FroNtier components in yellow Core Application Software

  26. A Multi-tier Architecture • Database (Oracle, MySQL, etc.) • Database Access Layer • Tomcat: servlet management engine • JDBC: Database connection • Database Connection pool management • Caching and proxy layer: Squid • widely used, highly configurable, caching proxy server • Client • Client API provides simple interface packaged in a compact client library Core Application Software

  27. General Deployment Overview Cache Layer local to each CAF Gbit ethernit CAF 1 Processor Farm CAF 2 Processor Farm CAF N Processor Farm Dedicated Squid Node Dedicated Squid Node Dedicated Squid Node WAN Launchpad Load Balancing and Failover Intermediate Cache Layer Squid Node 1 Squid Node 2 Squid Node N DB Access Layer Tomcat Node 1 Tomcat Node 2 Tomcat Node N Database

  28. Schedule • Nov/Dec 2004 • Release production software for ORCA (dst production) • Review Data Model (Event, Conditions, Book-Keeping) and propose a new one • Jan/Feb 2005 • Prototype&test Conditions Database • Prototype Event Data Model and DataSet Book-Keeping system • March/April 2005 • Release Analysis software • First version of new data model • First examples of full chain analysis for the Integration test • July 2005 • Deploy first version of software for integration test for ALL detectors • Oct 2005 • Start Integration (Magnet) test Core Application Software

  29. Summary • CMS has developed and deployed a Core Software System based on LCG components • Supports Simulation, Reconstruction, Analysis and Visualization • Successfully used in production and user analysis • A process to review our Data Model & Framework in light of successful experiences at Tevatron and to account for the new Computing Model is underway • Proposal in Dec, release in April • Mayor technical challenges in near future • Database access and deployment • Data access from root • Dataset book-keeping • Next year milestones driven by • TDR preparation • Integration (magnet) tests Core Application Software

More Related