1 / 8

POOL Release Plan for 2003

Detailed plan for 2003 releases, including feature sets, performance metrics, scalability targets, and proposed improvements for LCG-1 components.

jewellg
Download Presentation

POOL Release Plan for 2003

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. POOL Release Plan for 2003 Dirk Düllmann LCG Application Area Meeting, 5th March 2003 LCG - POOL Project

  2. Proposed Release Schedule for 2003 • V0.4 - February `03 (today) • Automated Dictionary generation, SEAL integration, SPI compliance • V1.0 - April ‘ 03 (wrongly tagged V0.5 im my last AAM talk) • Functional complete LCG-1 set (see next slides) • V1.1 - June `03 • First performance & bug fix release • mid July – Start LCG-1 deployment of V1.1 • Expect bug fix releases in the 1.x series as required by experiments using LCG-1 in pre-production activities LCG - POOL Project

  3. LCG-1 (=V1.1) Feature Set • File Catalog - Assume hierarchical chain of catalogs to minimise coupling • a few jobs populate XML based local catalog • after quality check those get merged into a db hosted production catalog • after additional checks entries are published to a grid wide catalog • Start with rather weak consistency checks on the catalog • Won’t check existence of files, pfn or lfn syntax • But we will check for duplicated pfn entries (with different FileID) • Need to tighten checks as filename syntax and semantics get better defined • Grid enabled EDG catalog requires a service to be run • Only client side interface is part of POOL (external package) • Service run by IT-DB, IT-GD and others • Proposed Scalability/Performance Metrics for Catalog Components • 1M entries for DB catalogs, 20k entries XML catalogs • 300 concurrent clients for DB served catalogs • Target access time for insert/lookup better than 30ms LCG - POOL Project

  4. LCG-1 Feature Set • Storage Manager • Sufficient to implement current production schema set • may still require slight changes to code on the experiment side • Aim at stable (1 year) file format for bulk data classes • Some data migration/reproduction of meta data may be unavoidable to add missing functionality after June • Rely on underlying (ROOT I/O) format to be stable for the expected period • Proposed Metrics • Support for an agreed experiment test suite • eg ATLAS Guinea Events, CMS examples, …. • to be established and shared with SEAL LCG - POOL Project

  5. LCG-1 Feature Set • Data Service (Object Cache) • Significant Redesign • Support reference counting to allow automatic cache memory management • user reads objects (automatic allocation in the cache) • all references to the same object refer to the same cache copy • once the last reference is destroyed the cache space is freed by POOL • Now support two working models • eg LHCb : all objects in a POOL cache will be deleted explicitly by the experiment framework • no Refs on the application side exist after a transaction commit • eg CMS : cache object may have different lifetimes and come and go as required by the application • Refs may continue to exist (and stay valid) across transaction boundaries • Need to understand longer term strategy • after first feedback from experiments -> relation to SEAL Object Whiteboard • Proposed Performance Metrics (Full stack: POOL -> Disk) • less than 30% bandwidth degradation wrt. ROOT I/O • less than 30% CPU mark-up wrt. ROOT I/O LCG - POOL Project

  6. POOL needs for SEAL deliverables • Dictionary and Dictionary Generator • Pre-release of SEAL component is used now • Improvements for persistency support of more complex types, dictionary import export will likely be required • Message Service and Exception Base • Centrally configurable diagnostic output • SEAL component used now, but very likely to change significantly • Component Infrastructure • Component library loading and un/re-loading • Pre-release in time for LCG-1 ? • Scripting Infrastructure • POOL component interface from scripting languages • Pre-release in time for LCG-1 ? • Object Whiteboard • Relation to object cache to be clarified • To be addressed after LCG-1 ? LCG - POOL Project

  7. POOL needs for SPI deliverables • Maintenance of LCG APPs external libraries for platform/compiler combination foreseen for regression test • also applies to SEAL releases • Automated (“Nightly”) build system • Regression testing • subsystem & integration tests • platform/compiler regression • file format regression test • Can we get this in time for POOL V1.0 ? LCG - POOL Project

  8. Proposed Schedule for 2003 cont. • Major new developments for the second phase of LCG-1 • V1.5 - September `03 • RDBMS independence layer (beta) • for vendor neutral Catalog, Collection reimplementation • RDBMS based Storage Manager • ConditionsDB re-implementation (outside of scope of POOL) • Fall - II. Persistency Workshop • Focus: First LCG-1 production experience – new developments • (??) POOL tutorial to framework integrators • To be defined after feedback on our V1.x user documentation • V1.6 - November `03 • Feature set still depends very much on experiment feedback • POOL Light • Is there a need for re-packaged/feature constrained subset? • RDBMS Storage Manager production • Is there a need for a storage manager implementation based on an RDBMS (secure in place update with transactional consistency) LCG - POOL Project

More Related