Persistency Framework Introduction & Overview

1. Persistency FrameworkIntroduction & Overview Dirk Düllmann, CERN IT Applications Area Internal Review 18-20 September 2006, CERN

2. Persistency Framework Parts Three Software Packages • CORAL - Ioannis (presented by Radovan) • Relational Database Access (and abstraction) • POOL - Giacomo • Object-Persistency (via Root or Databases) • Object-Navigation (via catalogs and collections), • COOL - Andrea • Management of versioned conditions time series • Layered on top of each other • Eg COOL uses CORAL to access databases • Experiments (eg ATLAS) utilize conditions objects stored in POOL • even though COOL does not depend on POOL directly Persistency Framework Introduction & Overview - 2

3. Main developments • CORAL • Area of key importance for database • DB lookup, monitoring, authorization, connection pool • Encourage efficient db apps by API design (bind var’s session reuse) • New package replaced database implementation dependent code in POOL/COOL • Savings -> one ConditionsDB instead of a Oracle and MySQL flavour • Picked up by experiments (offline and online) • Integration with LCG services • Database service at T0 and T1s • Replica management via LFC Persistency Framework Introduction & Overview - 3

4. Changes in PF Underpinnings • POOL • Consolidation and maintenance phase • Few larger functionality request • Significant work went into absorbing the changes of the infrastructure below • Components moving from SEAL into ROOT - POOL support Dec 05 • Interface changes because of different coding conventions between SEAL and ROOT • Absorbing some dependency changes: eg change of exception base class in ROOT Reflex • Goal - POOL user code has stayed largely stayed unaffected • According to savannah we have largely achieved this goal Persistency Framework Introduction & Overview - 4

5. PF and SEAL • Preparing for the next round of core changes • All PF components uses the SEAL component model • To be prepared for an eventual plug-in mgr change in case that is requested by experiments or the ROOT project • Actively involved in SEAL bug fixing (eg threading issues) • Need a firm AA strategy in this area and insure maintenance for base packages • Just one more standard compromise between • Removing dependencies by introducing copied code • Removing code duplications by introducing dependencies • May need to accept minor copies or minor dependencies • Better than going back and force between the two extremes • PF can provide a short list of used SEAL components • Can we setup a table of SEAL components and plan evolution Persistency Framework Introduction & Overview - 5

6. PF and AA Releases • Now largely driven by ROOT bug fix releases, which are driven by experiment requests • Release frequency increased and latency decreased • Often just rebuild releases for POOL and SEAL • Also the warning time in the planning cycle decreased • CORAL and COOL still adding functionality • Could further streamline the process if AA components would stabilize according to dependency structure • Trade-off between all parallel release work and consistency • “All parallel” testing is best suited for adiabatic changes (eg bug-fixing rather than interface or semantic changes) • Not always the case yet… • We can work with any proposed build-system, but expect the support of this system to be handled not by PF Persistency Framework Introduction & Overview - 6

7. PF Manpower • POOL/CORAL - Moved to maintenance and improved service integration • Increasingly IT based - experiment fraction lowered • Profile is compatible with 3D and local database service improvements work • COOL - Move to full production, distribution to LCG Tiers • Two very effective ATLAS contributors left • Little manpower left also in other areas • Collections • Main ATLAS person (Kristo) left - part of Marcin’s tasks now • Documentation • could still improve but little effort left • Middle tier for DB access • Suggested in last review as possible solution for scalability and security problems • Do we have the development effort needed? • Manpower reduction did and will affect work plan!! Persistency Framework Introduction & Overview - 7

8. PF and Grid Components • File Catalogs • Good news: consolidation on LFC • Still some discussions on their individual merits / problems • Typically grid deployment discussions • POOL did not have to invest in new catalog developments • Rather some internal restructuring • Joint configuration management between apps an deployment side is still an issue • Versions numbers & compatibility matrix • Need a process to make sure both sides stay in sync Persistency Framework Introduction & Overview - 8

9. Some Observations • Excellent cooperation with SPI and ROOT teams • Preparation of new LCG configurations needs backup for Andreas to cope with vacations periods • Manpower in SPI could help build/release infrastructure consolidation between projects • Stability of manpower and keeping existing expertise is a concern • Testing early (low down in the s/w stack) could help to further improve TTU • Moved some tests from COOL into CORAL • Propose to move some additional file format regression tests down into ROOT Persistency Framework Introduction & Overview - 9