80 likes | 93 Views
Detailed plan for 2003 releases, including feature sets, performance metrics, scalability targets, and proposed improvements for LCG-1 components.
E N D
POOL Release Plan for 2003 Dirk Düllmann LCG Application Area Meeting, 5th March 2003 LCG - POOL Project
Proposed Release Schedule for 2003 • V0.4 - February `03 (today) • Automated Dictionary generation, SEAL integration, SPI compliance • V1.0 - April ‘ 03 (wrongly tagged V0.5 im my last AAM talk) • Functional complete LCG-1 set (see next slides) • V1.1 - June `03 • First performance & bug fix release • mid July – Start LCG-1 deployment of V1.1 • Expect bug fix releases in the 1.x series as required by experiments using LCG-1 in pre-production activities LCG - POOL Project
LCG-1 (=V1.1) Feature Set • File Catalog - Assume hierarchical chain of catalogs to minimise coupling • a few jobs populate XML based local catalog • after quality check those get merged into a db hosted production catalog • after additional checks entries are published to a grid wide catalog • Start with rather weak consistency checks on the catalog • Won’t check existence of files, pfn or lfn syntax • But we will check for duplicated pfn entries (with different FileID) • Need to tighten checks as filename syntax and semantics get better defined • Grid enabled EDG catalog requires a service to be run • Only client side interface is part of POOL (external package) • Service run by IT-DB, IT-GD and others • Proposed Scalability/Performance Metrics for Catalog Components • 1M entries for DB catalogs, 20k entries XML catalogs • 300 concurrent clients for DB served catalogs • Target access time for insert/lookup better than 30ms LCG - POOL Project
LCG-1 Feature Set • Storage Manager • Sufficient to implement current production schema set • may still require slight changes to code on the experiment side • Aim at stable (1 year) file format for bulk data classes • Some data migration/reproduction of meta data may be unavoidable to add missing functionality after June • Rely on underlying (ROOT I/O) format to be stable for the expected period • Proposed Metrics • Support for an agreed experiment test suite • eg ATLAS Guinea Events, CMS examples, …. • to be established and shared with SEAL LCG - POOL Project
LCG-1 Feature Set • Data Service (Object Cache) • Significant Redesign • Support reference counting to allow automatic cache memory management • user reads objects (automatic allocation in the cache) • all references to the same object refer to the same cache copy • once the last reference is destroyed the cache space is freed by POOL • Now support two working models • eg LHCb : all objects in a POOL cache will be deleted explicitly by the experiment framework • no Refs on the application side exist after a transaction commit • eg CMS : cache object may have different lifetimes and come and go as required by the application • Refs may continue to exist (and stay valid) across transaction boundaries • Need to understand longer term strategy • after first feedback from experiments -> relation to SEAL Object Whiteboard • Proposed Performance Metrics (Full stack: POOL -> Disk) • less than 30% bandwidth degradation wrt. ROOT I/O • less than 30% CPU mark-up wrt. ROOT I/O LCG - POOL Project
POOL needs for SEAL deliverables • Dictionary and Dictionary Generator • Pre-release of SEAL component is used now • Improvements for persistency support of more complex types, dictionary import export will likely be required • Message Service and Exception Base • Centrally configurable diagnostic output • SEAL component used now, but very likely to change significantly • Component Infrastructure • Component library loading and un/re-loading • Pre-release in time for LCG-1 ? • Scripting Infrastructure • POOL component interface from scripting languages • Pre-release in time for LCG-1 ? • Object Whiteboard • Relation to object cache to be clarified • To be addressed after LCG-1 ? LCG - POOL Project
POOL needs for SPI deliverables • Maintenance of LCG APPs external libraries for platform/compiler combination foreseen for regression test • also applies to SEAL releases • Automated (“Nightly”) build system • Regression testing • subsystem & integration tests • platform/compiler regression • file format regression test • Can we get this in time for POOL V1.0 ? LCG - POOL Project
Proposed Schedule for 2003 cont. • Major new developments for the second phase of LCG-1 • V1.5 - September `03 • RDBMS independence layer (beta) • for vendor neutral Catalog, Collection reimplementation • RDBMS based Storage Manager • ConditionsDB re-implementation (outside of scope of POOL) • Fall - II. Persistency Workshop • Focus: First LCG-1 production experience – new developments • (??) POOL tutorial to framework integrators • To be defined after feedback on our V1.x user documentation • V1.6 - November `03 • Feature set still depends very much on experiment feedback • POOL Light • Is there a need for re-packaged/feature constrained subset? • RDBMS Storage Manager production • Is there a need for a storage manager implementation based on an RDBMS (secure in place update with transactional consistency) LCG - POOL Project