1 / 32

WLCG-RUS

WLCG-RUS. An Extensible Solution to Grid Accounting & Usage Monitoring. EGEE 3 rd User Forum X. Chen, A. Khan Brunel University. Who am I?. PhD Candidate Current Research Scope Grid Computing, Accounting; Middleware Solutions; OGF Co-chair of OGF UR working group;

ialt
Download Presentation

WLCG-RUS

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. WLCG-RUS An Extensible Solution to Grid Accounting & Usage Monitoring EGEE 3rd User Forum X. Chen, A. Khan Brunel University

  2. Who am I? • PhD Candidate • Current Research Scope • Grid Computing, Accounting; • Middleware Solutions; • OGF • Co-chair of OGF UR working group; • Member of OGF RUS working group;

  3. Outline • WLCG-RUS Overview • Component Architecture • Design Status • Conclusion

  4. Overview • RUS • Resource Usage Service • Resource Management in the context of OGSA; • Aims at providing Grid resource logging and tracking interfaces; • OGF RUS and OGF UR spec. • Mainly used to enable Grid accounting & Usage Monitoring

  5. Grid Accounting • Many Production Grids provide resource usage logging and tracking facilities • e.g. NorduGrid (SGAS being accepted as one of key service in Globus), EGEE/WLCG (DGAS), and OSG (Gratia); • Issues • Heterogeneous and Grid-specific solution; • XML:DB persistence only • Gaps between Custom usage schema and OGF UR; • Lacking of interoperability in multi-Grid environment

  6. Accounting In WLCG • Multi-Grid Environment • Resources from three Grid peers (OSG, NorduGrid, and EGEE) • Most of EGEE services are reused; • WLCG Resource Broker; • Interoperability • Get usage data from individual accounting service; • Get usage data from heterogeneous usage storages (mostly relational DB);

  7. WLCG-RUS • Standardization • Full OGF RUS compatible • OGF UR version 1.0 as uniform data representation as message level; • Customization • at persistence level • Heterogeneous data persistence (Relational, file and XML); • Flexibly data provision (DGAS, Gratia, SGAS & UR generators); • Advanced Features • Summarization: aggregate UR in summary format (e.g. Total usage of a specific VO)

  8. Framework • Based on JISC funded Project: • “Review of Grid Accounting and Usage Monitoring” (Manchester & Brunel) • http://www.jisc.ac.uk/media/documents/programmes/einfrastructure/jisc_aum_final_report_wth.pdf • Proposed Framework Client Side RUS Service Access Control UR Generator RUS Operation Logics UR Mapping Session Management RUS Client SOAP Configuration Manager

  9. WLCG-RUS Architecture Enable flexible search, update dialects (e.g. SQL, Xupdate, and Xquery) Aggregator interface for custom algorithms for summarisation • Layered Architecture Property file-based configuration information for individual components RUS operation interfaces allowing custom implementations Data Access Object that allows various usage persistence access and Mapping to Usage Records Provide Access Control interface and default XACML implementation Delegate client-side requests to appropriate command Providing RUS-compatible SOAP Message and Aggregation Extensions

  10. Messaging • RUS Core WS-I Rendering (draft) • https://forge.gridforum.org/sf/go/artf6015 • http://forge.ogf.org/sf/go/artf6090 • Extensions • Grouping • Allowing summarisation by grouping criteria; • An alternative and easier search criteria; • Sorting • Sorting usage retrieval results

  11. Example: Extract Request Message • Extract and summarise Disk Usage & CPU Duration where “VO=cms” and “VO=altas” on this month, sorted by Disk Usage;

  12. Example: Extract Request Message (Cont.) • Alternative, usage selection can be specified with RUS request body as defined within RUS core spec.

  13. Example: Extract Response Message • Response message should return a single usage record that represented as a summary usage record • An extension, <urf:Resource description=“NumberOfRecordAggregated” /> is used to indicated total amount of records taken effect on this aggregation.

  14. UR Modelling • UR Modelling • Derived From OGF-UR 1.0 for those use custom usage record representation; • Entity Type • Usage properties are shared by reference; • Nine entities • Value Type • Usage properties are shared by value; • All other usage properties defined in OGF-UR 1.0

  15. Entity Relationship

  16. Default Mapping Strategies • To bring relational usage storage into RUS • XRM • XML-Relational Mapping • Based on JAXB and Hibernate • Automatic creation of default relational representation of OGF-UR schema; • Hibernate file-based configuration for custom relational representation

  17. An Example • Disk Usage Table • Association to a single record; • i.e. One-to-Many record-disk association • The disk_usage table is generated as following:

  18. Data Access Framework • Generic Data Access Interface; • Extensive access to relational DB, XML:DB, file system, other service (OGSA-DAI, DGAS, Gratia, and etc.) • Default DAO implementation provides access through Hibernate engine; • DAO pattern • Abstraction while customization, flexibility and polymorphism.

  19. Security • Authentication • TSL and mutual-authentication • Access Control • Coarse-grain Access Control on invocation of RUS operations (insert, update, extract, delete and auditing) • Fine-grain Access control on per usage record basis.

  20. Access Control • Role Based Access Control • Roles are defined according to URF entities (see session1: WLCG-RUS XOM) • Four default Roles derived from RUS Entities • User (UserEntity/urf:UserIdentity/ds:KeyInfo/ds:X509Data/ds:X509SubjectName); • Machine Manager(MachineEntity/urf:MachineName); • Host Manager(HostEntity/urf:Host); • Project Manager (ProjectEntity/urf:ProjectName); • One Role for administration • RUS administrator (for all permissions) • Roles derived from URF extension framework • e.g. Site Manager (urf:Resource/@description)

  21. Access Control Rules • RULE 1: A user is only allowed to “extract” usage records on his/her name • either /urf:Usage/urf:UserIdentity/urf:GlobalUserName/text() • or /urf:Usage/urf:UserIdentity/ds:KeyInfo/ds:X509Data/ds:X509SubjectName/text() • depending on deployment environment

  22. Access Control Rules (cont..) • RULE 2: Administrator has full permission on ALL RUS operations upon ALL persistent usage records • RULE 3: Other roles (project manager, host manager, machine manager, and extensive managers) have permissions on “insert”, “extract” and “audit” operations ONLY upon relevant usage records; • e.g. A CMS project manager can only execute “insert”, “extract” and “audit” operations on usage records where “//urf:ProjectName/text()” value is equal to “CMS”; • e.g. The Brunel Tier 2 site manager can only execute “insert”, “extract” and “audit” operations on usage records where“//urf:Resource/@description” value is equal to “Brunel-Tier2”;

  23. Role Configuration • WLCG-RUS schema; • An example

  24. Authorization Framework • Composed of two abstract components: • Authorizer • The component that performs access control check; • For both coarse- and fine-grain access control; • Authorizer Factory • the factory component that creates an instance of authorizer implementation;

  25. Default Implementation • Default Authorizer • Based on XACML RBAC 1.0 profile; • Configurable for custom role definition according to URF extension framework; • Auto-generation of XACML policy sets; • Supporting policy persistence; • Simplified configuration (ease writing verbose XACML policy files) • High performance (through XACML policy evaluation ONLY when fine-grained access control); • Default Authorizer Factory • Creation of default authorizer;

  26. Configuration • Runtime Configuration • Property-based configuration; • Parameters includes DAO factory, Aggregator factory, Filter factory; • UR Mapping Configuration • Default UR mapping is based on hibernate-mapping configuration file (XML-based) • Deployment Configuration • WS-Management configuration (see implementation slide)

  27. Runtime: Insertion Example

  28. Accounting with WLCG-RUS Interoperability

  29. Implementation • Based on Sun WS-Man platform • Web Service for Management; • Usage Record as a management resource; • Enabling enumeration for large number of query results; • Extensible through WS-Management interfaces; • Hibernate • For Default UR Mapping to relational usage representation;

  30. Development Status • Version 1.0 • Requirement Analysis (done) • System Design (done) • Implementation (debugging) • First alpha release (expected at Feb. Or March) • Test (Planned to deployed at Tier-2 site, Brunel-Tier2?)

  31. Conclusion & Future Work • WLCG-RUS • Provides extensible solution to RUS; • A development framework; • Default implementation for relational usage persistence (milestone); • Dynamic XML-Relational mapping; • Aggregation; • Future work • Version 2 with evolvement of UR 2.0

  32. The End • Thanks 

More Related