1 / 15

Storage Management

Storage Management. Owen Synge Developer, Packager, and first line support to System Administrators. Talks Scope GridPP for the year ahead. Role RAL employee, RAL data management Team. Working within GridPP2. Mass Storage and Local Data Management. Project Members

hong
Download Presentation

Storage Management

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Storage Management • Owen Synge • Developer, Packager, and first line support to System Administrators. • Talks Scope • GridPP for the year ahead. • Role • RAL employee, RAL data management Team. • Working within GridPP2. • Mass Storage and Local Data Management. • Project Members • Owen Synge, Jens Jensen, Tara Shah, Glen Johnson Title of Talk

  2. Summary • Background • History • Current State • Mass Storage • Mass Storage Future • Mass Storage Future : Upcoming Releases • Mass Storage Future : SRM Release • Mass Storage Future : SRM News • Worker Node Disks • Local Storage Future : Time Scales • Local Storage Future : Priorities/Features • Storage Future • Conclusion • Issues Title of Talk

  3. Background • ADS Service • Peta-byte level Tape Storage Solution • Grid Summary • Distributed Computing • Commodity Hardware • European Data Grid Project (EDG) • Europe wide Grid Research Project • Provided much of LCG infrastructure • SRM Collaboration • Collaboration between Fermilab, Jefferson Lab, Lawrence Berkeley, RAL, CERN • Contributed to the design of the SRM version 2 protocol Title of Talk

  4. History • EDG SE History • V1 • HTTP based Storage interface • V2 • Java Web services Storage Interface • V3 (Not History as Still on V2.2) • SRM Standard Compliant Interface Title of Talk

  5. Current State • EDG SE Status • Stable Version • Stable since November. • One Bug Fix (File Truncation) • One Administration Script created. • Atlas Data Transfer • About 1500 files; 2 TB in total transferred to CERN • Going into LCG Production (Security bug on Client Side). • ADS Support • Not using Disk, Castor or HPSS interfaces • Current Development Version • Released this month. • Configuration Upgrade (Layered Template System). • Metadata Upgrade (See Later). Title of Talk

  6. Mass Storage Future • Addressing issues in EDG-SE • Metadata/Configuration (Due in near future) • Scalability (Number of Files) • Performance (Time taken and resources used) • SRM Compliance • Addressing issues in GridPP • Disk Resources as MSS system • Checksums (Atlas request) • Addressing issues for LCG (UK) • Clustering for Bandwidth • Testing frameworks • Addressing issues (Generic) • Access Control • Job/User Namespaces Title of Talk

  7. Mass Storage Future : Upcoming Release Time Line • Release 2.2 • Renaming of LCG release • Release 2.3 (May/June 2004) • Layered Configuration • Metadata upgrade • Release 2.4 (Before August 2004) • LCG Release of 2.3 when Stable and bugs squashed • Release 3.0 (Before August 2004) • SRM compliance • Release 3.1 • To be decided Title of Talk

  8. Mass Storage Future : SRM Release • Extra Features • Multiple Files acted upon with a single operation • Fully Asynchronous (srmStatusOfGetRequest) • Benefits • Interoperability with major Storage solutions in Grid community. • GFAL and a large number of other Client tools available. Title of Talk

  9. Mass Storage Future : SRM News • SRM2 • Finalised now working on 2.1 Revisions • SRM and the GGF • SRM is going to be a GGF standard (Honolulu) • Specification of Basic/Advanced SRM • We will provide a service somewhere between these specifications for ADS on the first SRM release • SRM and Other Storage solutions • SRB • On going work to support SRM API • DCache • Currently supports SRM v1 API Title of Talk

  10. Worker Node Disks • EDG/EGEE Local Data Management • Missing from EDG • Clean Down Worker Nodes • Reservation • Remote File Access • RFIO, SlashGRID, Replica Manager/EDG SE Client tools, GFAL. • DCache • Aggregate Worker node space into storage • Mature system Title of Talk

  11. Local Storage Future : Time Scales • PM03 LD1: UK Site Coordinated Deployment and Support Plan for Local Storage Management (LSM) system. • PM05 LD2: Release of LSM integrating with LCG3 at UK sites. LCG 3 expected in PM05. • PM07 SD3, LD3: Software prototype release integrating with EGEE DJRA1.3 • PM16 SD4, LD4: software prototype release integrating with EGEE DJRA1.6 • PM26 SD5, LD5: Release of MS and LSM integrating with EGEE follow-on. Title of Talk

  12. Local Storage Future : Priorities/Features • Requirements • Disk Clean down? • Using Worker Node disk space? • Space management? • Specification • Need to establish which requirements are highest priority. • Implementation • Need to get the plan signed off. • Need to employ a new member of the team. Title of Talk

  13. Storage Future • SRM (De facto Grid Standard) • SRM1 moving to SRM2 • GSM Basic moving to GSM Advanced • Worker Nodes • Clean down after Jobs • DCache • Must not forget Tier 2+3 have more storage space than Tier 0+1 Title of Talk

  14. Conclusion • SRM • On going evolution of a standards management API • Not yet clear where access control is going exist in this area. • No SRM2/GSM Advanced implementations yet exist • LSM • Local Storage management solutions not yet clear Title of Talk

  15. Issues • Tracking SRM scope and model • Local Storage Priorities • Recruiting and Training new member of the team • Testing environments for LCG/EGEE • Representation of files (Metadata Group?) • Trees/UID • Namespaces • Job/Service/User/VO/host based Title of Talk

More Related