1 / 32

Implementation of GLUE 2.0 support in the EMI Data Area

Implementation of GLUE 2.0 support in the EMI Data Area. Elisabetta Ronchieri on behalf of JRA1’s GLUE 2.0 Working Group INFN-CNAF 13 April 2011, EGI User Forum, Vilnius. Overview. GLUE details Working Group Implementation State Future Work Conclusions. Overview. GLUE details

amills
Download Presentation

Implementation of GLUE 2.0 support in the EMI Data Area

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Implementation of GLUE 2.0 support in the EMI Data Area Elisabetta Ronchieri on behalf of JRA1’s GLUE 2.0 Working Group INFN-CNAF 13 April 2011, EGI User Forum, Vilnius

  2. Overview • GLUE details • Working Group • Implementation State • Future Work • Conclusions

  3. Overview • GLUE details • Working Group • Implementation State • Future Work • Conclusions

  4. GLUE goals • GLUE describes the Grid infrastructure • GLUE: Grid Laboratory Uniform Environment • Why is it used? • Service discovery • Monitoring • Brokering • Accounting • …

  5. GLUE 1.3 • GLUE 1.3 is currently widely deployed • In addition to Nordugrid schema, UNICORE • What’s wrong with GLUE 1.3? • Protocol architecture: gridftp, gram… • comparing with service architecture • Numerous details • e.g., SE/CE relationships • e.g., SRM is published as two objects (GlueSEControlProtocol and GlueService) • Not easily extensible

  6. GLUE 1.3 • GLUE 1.3 is currently widely deployed • In addition to Nodugrid schema from UNICORE • What’s wrong with GLUE 1.3? • Protocol architecture: gridftp, gram… • comparing with service architecture • Numerous details • e.g., SE/CE relationships • e.g., SRM is published as two objects (GlueSEControlProtocol and GlueService) • Not easily extensible GLUE 1.3 is no longer able to provide an accurate description of Grid resources

  7. Top BDII Top BDII Site BDII Site BDII Resource BDII Resource BDII Resource BDII Resource BDII Provider Provider Provider Provider How is it used? • How is it implemented now? • BDIIs • Info Providers • Clients, LDAP

  8. GLUE 2.0 • It is an OGF standard (GFD-R-P.147) • Ratified March 2009 • It is described as an abstract schema • A separate document describes LDAP rendering

  9. GLUE 2.0 Storage Schema 7 1 Storage Access Protocol Has Storage Capacity 3 8 Storage Service Storage Manager Offers Has Offers Manages 5 4 Storage End Point Storage Share Data Store 2 Maps User to Defined on Has 6 Storage Share Capacity S. Burke, L. Field, and D. Horat, “Migration to the GLUE 2.0 information schema in the LCG/EGEE/EGI production Grid”, CHEP 2010, Taipei

  10. Roll out Plan • The plan was presented at CHEP 2010 (by S. Burke): • Define the abstract schema • Define the LDAP rendering • Implement the schema in the BDII and roll out • Write and deploy information providers • Update client tools to understand GLUE 2.0 • Retire GLUE 1.3 • Both GLUE versions will be published but with different bases: • “o=grid” for GLUE 1.3 • “o=glue” for GLUE 2.0

  11. Roll out Plan • The plan was presented at CHEP 2010 (by S. Burke): • Define the abstract schema • Define the LDAP rendering • Implement the schema in the BDII and roll out • Write and deploy information providers • Update client tools to understand GLUE 2.0 • Retire GLUE 1.3 • Both GLUE versions will be published but with different bases: • “o=grid” for GLUE 1.3 • “o=glue” for GLUE 2.0 Completed during EGEE Ready for EMI 1 Ready for EMI 2 Far in the future

  12. Overview • GLUE details • Working Group • Implementation State • Future work • Conclusions

  13. The Working Group • Purpose is • To ensure the consistent implementation of GLUE 2.0 across the Data Area • Representatives are • SEs: dCache, DPM, StoRM • Clients: dCache, gLite, ARC, UNICORE • Timetable is as follows (from DJRA1.2.1) • Common agreement on the interpretation of the GLUE 2.0 schema [PM6] • Publishing GLUE 1.3 data with GLUE 2.0 schema [PM 12] • EMI data components fully GLUE 2.0 compatible [PM 24] • Useful links • https://twiki.cern.ch/twiki/bin/view/EMI/EmiJra1GLUE20 • https://twiki.cern.ch/twiki/bin/view/EMI/EmiJRA1GLUE20FAQ

  14. Working Group Decisions • We are only considering current LDAP Information System • We will validate published data using gstat 2.0 • Via “glue validator” provided by EMI infrastructure area • https://tomtools.cern.ch/confluence/display/IS/GLUEValidator • It performs generic checks for schema compliance • This can be extended to encapsulate the decisions of the Working Group, e.g., to enforce the presence of certain attributes that are optional in the LDAP schema

  15. Working Group Decisions (examples) • Optional attributes are optional • Exception list: • *size attributes • How do we ensure unique IDs? • Implementer should pick a hostname which is associated with the service and reasonably persistent, and append the string ‘/data’. They can then append further information as they see fit. Note that this is not intended to be parsed. • e.g., srm.example.org/data/opaque1 • Which Type is my service? • servicetype_t is an open enumeration. Use a type of org.emi-eu., e.g., org.emi-eu.dCache. • What do the *Size attributes mean? • Where possible, the “Usage of Glue Schema v1.3 for WLCG Installed Capacity Information” should be considered, being the current authoritative position. This may require further discussion.

  16. Overview • GLUE Details • Working Group • Implementation State • Future Work • Conclusions

  17. Work Completed • GLUE 2.0 info providers implemented for all three Storage Elements (dCache, DPM and StoRM) • Publishing the same info as for GLUE 1.3 • DPM was installed on the SA2 testbed at CERN • StoRM is under installation on the SA2 testbed at INFN-CNAF • dCache is under installation on the SA2 testbed at DESY • FTS updated to publish to GLUE 2.0

  18. DPM • The dpm-listspaces utility (maintained at LAL) operates as the DPM info provider • It now has a “--glue2” option to output all relevant info in the GLUE 2.0 schema • Main issue was understanding how to incorporate GLUE 1.3’s GlueVOInfo into GLUE 2.0 • The answer was multiple Storage Shares with a common SharingID

  19. dCache GLUE info-provider • The info-provider has two parts: • A generic framework for converting xml into ldif called xylophone • A dCache specific configuration for xylophone • Xylophone is platform independent • Can run on any platform with XSLT or Java. • The configuration is a domain-specific language: • Changing what is published is easy.

  20. StoRM GLUE dynamic info-provider • The dynamic info-provider is written as a bash shell script and configured by using yaim.

  21. Storage Space Concepts • Total Size (TS) is the total amount of storage of a given Type (e.g., online, nearline, and so on)

  22. Storage Space Concepts • Used Size (US) is the amount of storage occupied by stored data.

  23. Storage Space Concepts • Reserved Size (RS) is the amount of storage reserved for use by a specific user.

  24. Storage Space Concepts • Free Size (FS)=TS-US-RS

  25. Storage Space Concepts • FS contains Available Size and Unavailable Size

  26. All info-providers in EMI-1 • Support for GLUE 2.0 comes with EMI-1. • Info-providers can publish GLUE 1.3, 2.0 or both in parallel: • GLUE 1.3 and GLUE 2.0 are on separate bases; • The default is to publish both. • GLUE 2.0 base has more or less the same information as GLUE 1.3.

  27. Overview • GLUE Details • Data GLUE 2.0 Working Group Work Plan • Implementation State • Future Work • Conclusions

  28. Future work • Update clients • Gfal, FTS • Extend existing information to make a better use of GLUE 2.0 • Fix problems discovered when validating production data services

  29. Overview • GLUE Details • Data GLUE 2.0 Working Group Work Plan • Implementation State • Future Work • Conclusions

  30. Conclusions • Data PTs have delivered info-providers for GLUE 2.0 on time • They keep backward compatibility by publishing both GLUE 1.3 and 2.0 • Updating clients to query GLUE 2.0 is the next step • The EMI project is on track to deliver GLUE 2.0

  31. Acknowledgements GLUE 2.0 Working Group emi-jra1-data@eu-emi.eu dCache Product Team DPM Product Team StoRM Product Team

  32. Thank you!

More Related