1 / 27

MDS-2.1 and Futures

MDS-2.1 and Futures. Karl Czajkowski Information Sciences Institute University of Southern California. Talk Outline. Introduction Problem, history, etc. MDS-2 Architecture Protocols Features, services MDS-2.1 Software Technology map Information Model Additional background.

axl
Download Presentation

MDS-2.1 and Futures

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. MDS-2.1 and Futures Karl Czajkowski Information Sciences Institute University of Southern California

  2. Talk Outline • Introduction • Problem, history, etc. • MDS-2 Architecture • Protocols • Features, services • MDS-2.1 Software • Technology map • Information Model • Additional background

  3. Resource Discovery/Monitoring R ? R R ? R R R R R R partioned network dispersed users R ? ? R R R R R R R R R R • Distributed users and resources • Variable resource status • Variable grouping and connectivity VO-A VO-B

  4. Basic Grid Acquisition Phases • Resource Discovery • “What resources are relevant?” • Bootstraps planner state • Resource Status Query • “How do resources compare (now)?” • Refines planner knowledge • Resource Control • “Did I acquire the resources?” • Not an information service task!

  5. MDS History • MDS-1 (classic) • Centralized database • Globus 1.1.2 and earlier • Did not scale • MDS-2 • Distributed services • MDS 2.0 in Globus 1.1.3 • New MDS 2.1 development in alpha release

  6. Base Features • Virtual Organizations (Vos) • Group together resources • Support community-specific “discovery” • Specialized “views” • Scalability • Many resources • Many Vos • Graceful degradation of service

  7. Virtual Organizations • Collaborating individuals and institutions • Shared goals • Enable sharing of resources • Non-locality of participants • Dynamic in nature • VOs come and go • Resources joing and leave Vos • Resource change status and fail • Community-wide goals

  8. Scalability • Large numbers • Many resources • Many users • Independence • Resources shouldn’t affect one another • Vos shouldn’t affect one another • Graceful degradation of service • “As much function as possible” • Tolerate partitions, prune failures

  9. New MDS-2.1 Features • Security • GSI mutual-authentication • Fine-grained access control by GSI name • Performance • Better query speeds • Less stale information • Extensibility • Convenience

  10. Service Hierarchy VO-specific AggDirs discovery (GRIP?) ? A A lookup (GRIP) registration (GRRP) R R R R standard ResDesc services • Resource Description via Info. Protocol (GRIP) • Co-located with resource on network • Aggregate Directories (via GRIP or other) • Can be made hierarchical • Dynamic Registration via Reg. Protocol (GRRP)

  11. R R R R R R R R R R R R R R R registration R fault-partition messages R R R R R R R R R R R R R R R R R R replicated directories divergent directories VO-A VO-B Distributed Services D D D D • Service scales with Grid growth • Loose consistency model tolerates failures • Interoperability by protocols

  12. Soft-state Registration • Periodic notification • Service/resource is available • Expected-frequency metadata • Automatic extension • Add new resources to directories • Invite resource to join new directory • Self-cleaning • Reduce occurrence of “dead” references

  13. MDS-2 Implementation • Grid Resource Information Service (GRIS) • Provides resource description • Modular content gateway • Grid Index Information Service (GIIS) • Provides aggregate directory • Hierarchical groups of resources • Lightweight Dir. Access Protocol (LDAP) • Standard with many client implementations • Used for GRIP (and GRRP currently)

  14. MDS-2.1 Development Activities • Incorporating external advances • New OpenLDAP 2.0.x code-base • Cyrus-SASL/GSI security integration • Leveraging new Globus packaging model • Improving internal components • Better query servicing • New configuration/policy support • Invitation (reverse registration)

  15. MDS-2.1 External Software Stack • OpenLDAP 2.0.x (.11) • Implements LDAPv3 protocol • Client and server components • Cyrus-SASL • Generic security • We provide loadable SASL/GSI plugin • Globus GSI (repackaged) • Provides GSS-API interface to PKI • Loadable module works with SASL plugin

  16. MDS 2.1 Security • PKI authentication • Static authorization • Class, attribute, object name rules • “Self” authorization • Semi-dynamic rule • Requires “owner” attribute on objects • Dynamic authorization • Directory-based group lists (or future CAS) • Per-object access rule attributes

  17. MDS-2.1 Internal Software • Wrappers/tools • Simplify typical idioms • Modular GRIS providers • Probe/query resource status • Generates LDIF-format data • LDAP server “backend” modules • GRIS provider dispatch/caching • GIIS implementation(s)

  18. MDS-2.1alpha GRIS Providers • globus-software reports Globus packages • grid-info-host reports host OS info • grid-info-host-interfaces reports NICs • grid-info-host-load reports host load • grid-info-host-filesystem reports disks • globus-gram-reporter reports jobs

  19. GRIS Dispatch Tests • Concurrent dispatch for each provider: • Could search intersect provider? No, then stop. • Is provider cache stale? Yes, then refill. • Apply search filter to cache data. • Combine all providers’ results

  20. MDS-2.1 GRIS Configuration dn: sw=Globus, hn=${GLOBUS_HOSTNAME}, ${GRID_INFO_ ORGANIZATION_DN} objectclass: GlobusTop objectclass: GlobusActiveObject objectclass: GlobusActiveSearch type: exec path: /opt/globus-mds/bin base: globus-version args: -ldif cachetime: 950400 timelimit: 10 sizelimit: 1 …

  21. GRIS Configuration cont’d dn: hn=${GLOBUS_HOSTNAME},{GRID_INFO_ORGANIZ ATION_DN} objectclass: GlobusTop objectclass: GlobusActiveObject objectclass: GlobusActiveSearch type: exec path: /opt/globus-mds/libexec base: globus-gram-reporter args: -f /opt/globus-mds/etc/globus-gram-rep orter.conf -onetime cachetime: 30 timelimit: 10 sizelimit: 20 …

  22. Hierarchical GIIS • Maintain set of remote services • Track incoming live registrations • GRIS or GIIS registrants • Cached proxy results (now), or • Same cache logic as GRIS • Refill cache with “chaining” queries • LDAPv3 referral results (planned) • Do not maintain any local info cache • Redirect clients to active registrants

  23. Extensible GIIS Framework • Modular registration actions • Re-use registration protocol decoding • Specialize directory update • e.g. prefetch indexable data • Modular query actions • Re-use query protocol decoding • Specialize query handler algorithm • e.g. utilize precomputed indices

  24. MDS-LDAP Data Model name dn: hn=hostX types objectclass: computer values system: mips irix dn: queue=default, hn=hostX dn: perf=load5, hn=hostX dn: store=scratch, hn=hostX objectclass: service objectclass: perf objectclass: storage objectclass: queue objectclass: loadaverage objectclass: raidstore • Info named within service • Info tagged with content type name(s) • Values associated with typed attributes url: gram://hostname/default period: 10 free: 33515 MB dispatchtype: immediate load5: 3.2 raidmode: stripe

  25. MDS-LDAP Query Model • Search scoping • Search rooted in namespace • Search depth of “root,” “root’s children,” or “root’s subtree” • Search filter • Value or type comparison • Logical combinations of filters • Namespace represents concept space

  26. Namespace Management host: hn=R1, O=O1 AggDir host: hn=R2, O=O1 host: hn=R3, O=O1 host: hn=R1, O=O2 host: hn=R2, O=O2 host: hn=R1 O1 O2 R1 host: hn=R1 host: hn=R1 AggDir AggDir host host: hn=R2 host: hn=R2 host: hn=R3 R1 R2 R3 R1 R2 ResDesc • Info is named uniquely within a service • Append “source name” to disambiguate locally, or use URLs to refer to remote info host host host host host

  27. More Information • Questions? • HPDC-10 Paper (to appear August 2001) • “Grid Information Services for Distributed Resource Sharing” • MDS-2.1 Alpha Website • http://www.globus.org/mds2-alpha • Early access to development code

More Related