1 / 114

The Emergence of Open Grid Standards

Managed shared virtual systems. Computer science research. Open Grid Services Arch. Web services, etc. Real standards Multiple implementations. Globus Toolkit. Internet standards. Defacto standard Single implementation. The Emergence of Open Grid Standards. Increased functionality,

Gabriel
Download Presentation

The Emergence of Open Grid Standards

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Managed shared virtual systems Computer science research Open Grid Services Arch Web services, etc. Real standards Multiple implementations Globus Toolkit Internet standards Defacto standard Single implementation The Emergence ofOpen Grid Standards Increased functionality, standardization Custom solutions 1990 1995 2000 2005 2010

  2. Open Grid Services Architecture • Service orientation to virtualize resources • Everything is a service • From Web services • Standard interface definition mechanisms • Evolving set of other standards: security, etc. • From Grids (Globus Toolkit) • Service semantics, reliability & security models • Lifecycle management, discovery, other services • A framework for the definition & management of composable, interoperable services “The Physiology of the Grid: An Open Grid Services Architecture for Distributed Systems Integration”, Foster, Kesselman, Nick, Tuecke, 2002

  3. The Open Grid Services Architecture

  4. Web Services • XML-based distributed computing technology • Web service = a server process that exposes typed ports to the network • Described by the Web Services Description Language, an XML document that contains • Type of message(s) the service understands & types of responses & exceptions it returns • “Methods” bound together as “port types” • Port types bound to protocols as “ports” • A WSDL document completely defines a service and how to access it

  5. WSDL Example <wsdl:definitions targetNamespace=“…”> <wsdl:types> <schema> <xsd:element name=“fooInput” …/> <xsd:element name=“fooOutput” …/> </schema> </wsdl:types> <wsdl:message name=“fooInputMessage”> <part name=“parameters” element=“fooInput”/> </wsdl:message> <wsdl:message name=“fooOutputMessage”> <part name=“parameters” element=“fooOutput”/> </wsdl:message> <wsdl:portType name=“fooInterface”> <wsdl:operation name=“foo”> <input message=“fooInput”/> <output message = “fooOutput”/> </wsdl:operation> </wsdl:portType> </wsdl:definitions>

  6. Transient Service Instances • “Web services” address discovery & invocation of persistent services • Interface to persistent state of entire enterprise • In Grids, must also support transient services, created/destroyed dynamically • Interfaces to the states of distributed activities • E.g. workflow, video conf., dist. data analysis • Significant implications for how services are managed, named, discovered, and used • In fact, much of our work is concerned with the management of services

  7. OGSA Structure • A standard substrate: the Grid service • Standard interfaces and behaviors that address key distributed system issues: naming, service state, lifetime, notification • A Grid service is a Web service • … supports standard service specifications • Agreement, data access & integration, workflow, security, policy, diagnostics, etc. • Target of current & planned GGF efforts • … and arbitrary application-specific services based on these & other definitions

  8. Overview • Grid background • Open Grid Services Architecture • Open Grid Services Infrastructure • Beyond OGSI: other OGSA services • Globus Toolkit v3 implementation • Early GT3 performance results • Scientific and commercial perspectives • Summary

  9. OGSI Specification • Defines WSDL conventions and extensions • For describing and naming services • Working with W3C WSDL working group to drive OGSI extensions into WSDL 1.2 • Defines fundamental interfaces (using extended WSDL) and behaviors that define a Grid Service • A unifying framework for interoperability & establishment of total system properties • http://www.ggf.org/ogsi-wg

  10. FundamentalInterfaces & Behaviors • OGSI defines basic patterns of interaction, which can be combined with each other and with custom patterns in a myriad of ways • OGSI Specification focuses on: • Atomic, composable patterns in the form of portTypes/interfaces • Define operations & associated service data elements • A model for how these are composed • Compatible with WSDL 1.2 • Complete service descriptions are left to other groups that are defining real services

  11. OGSI: Standard Web Services Interfaces & Behaviors • Naming and bindings (basis for virtualization) • Every service instance has a unique name, from which can discover supported bindings • Lifecycle (basis for fault resilient state management) • Service instances created by factories • Destroyed explicitly or via soft state • Information model (basis for monitoring & discovery) • Service data (attributes) associated with GS instances • Operations for querying and setting this info • Asynchronous notification of changes to service date • Service Groups (basis for registries & collective svcs) • Group membership rules & membership management • Base Fault type

  12. Resource allocation Create Service Grid Service Handle Service data Keep-alives Notifications Service invocation Service discovery Register Service Service instances Open Grid ServicesInfrastructure (OGSI) Authentication & authorization are applied to all requests Service factory Service requestor (e.g. user application) Service registry Interactions standardized using WSDL

  13. Client • Introspection: • What port types? • What policy? • What state? GridService (required) Other standard interfaces: factory, notification, collections Grid Service Handle Service data element Service data element Service data element handle resolution Grid Service Reference Open Grid Services Infrastructure • Lifetime management • Explicit destruction • Soft-state lifetime Data access Implementation Hosting environment/runtime (“C”, J2EE, .NET, …)

  14. GWSDL • OGSI requires interface extension/composition • We worked within W3C WSDL working group to define standard interface extension in WSDL 1.2 that meets OGSI requirements • But could not wait for WSDL 1.2 • So defined gwsdl:portType that extends WSDL 1.1 portType with: • WSDL 1.2 portType extension • WSDL 1.2 open content model • Define GWSDL  WSDL 1.1 & 1.2 mappings

  15. GWSDL Example <wsdl:definitions> <wsdl:types>…</wsdl:types> <wsdl:message>…</wsdl:message> … <gwsdl:portType name=“foo”extends=“ns:bar ogsi:GridService”> <wsdl:operation name=“op1”>…</wsdl:operation> <wsdl:operation name=“op2”>…</wsdl:operation> <ogsi:serviceData … /> </gwsdl:portType> … </wsdl:definitions>

  16. OGSI portTypes

  17. GridService portType Operations

  18. OGSI Service Data • Attributes: Publicly visible state of the service • Want to bring full power of XML to attributes • getXXX/setXXX is too limiting • How to get/set multiple? • Want richer queries across attributes (e.g., join) • Use XML Schema, XPath, XQuery, XSLT, etc. • OGSI service data: • Attributes defined using XML Schema • Attributes combined into a single (logical) document within the service • Rich pull/push/set operations against service data document • Should declare attributes in WSDL interface

  19. Standard Service Data Elements

  20. Handles and Resolvers

  21. Service Groups

  22. Client Client Client Request and manage file transfer operations Notf’n Source Policy Grid Service Fault Monitor Pending interfaces Query &/or subscribe to service data Performance service data elements Policy Perf. Monitor Faults Example:Reliable File Transfer Service File Transfer Internal State Data transfer operations

  23. File Transfer Example

  24. File Transfer Factories

  25. Making a Reservation

  26. Establishing Delivery Service

  27. Monitoring Progress

  28. OGSI Implementations • Globus Toolkit version 3.0 (Java, C client) • U Virginia OGSI.NET (.NET) • LBNL pyGlobus (Python) • U Edinburgh (.NET) • U Manchester (PERL) • Fujitsu Unicore (Java)

  29. Overview • Grid background • Open Grid Services Architecture • Open Grid Services Infrastructure • Beyond OGSI: other OGSA services • Globus Toolkit v3 implementation • Early GT3 performance results • Scientific and commercial perspectives • Summary

  30. Structured Data Integration Job Submission Brokering Workflow Registry Banking Authorisation Transformation Structured Data Access Data Transport Resource Usage Web Services: Basic Functionality Structured Data Relational XML Semi-structured Open Grid Services Architecture Users in Problem Domain X Applications in Problem Domain X Application & Integration Technology for Problem Domain X Generic Virtual Service Access and Integration Layer OGSA OGSI: Interface to Grid Infrastructure Compute, Data & Storage Resources - Distributed Virtual Integration Architecture

  31. OGSA Standardization & Implementation • OGSI defines core interfaces and behaviors for manageable services • Supported by strong open source technology & major commercial vendors • Efforts are underway within GGF, OASIS, and other bodies to define standards for • Agreement negotiation • Common management model • Data access and integration • Security and policy • Etc., etc., etc.

  32. Transactions & Contexts • WS-Coordination & WS-Transaction • IBM/MS (not in standards org) • WS-CAF (Coordinated Application Framework) • Sun/Oracle/Arjuna/Fujitsu (not in standards org) • WS-CTX (Context) • WS-CF (Coordination Framework) • WS-TXM (Transaction Management) • Both take a “contextualization” approach • Context (id) threaded through SOAP header • OGSI for context creation, naming & lifecycle???

  33. Security Standards • Many core security standards are from IETF • X.509, Kerberos, etc. • X.509 Proxy Certificates (RFC soon hopefully) • Used by Globus Toolkit GSI • OASIS appears to be leader in Web services security standards • WS-Security: SOAP message security • SAML: signed assertions using XML • XACML: access control lists using XML • GGF OGSA Security WG evaluating security specifications for applicability to OGSA

  34. IBM/MicrosoftWS Security Architecture • Large set of specifications for doing Web services security, most of which should be appropriate for OGSA • Announced April 2002 • Initial spec in July 2002 (WS-Security) • Submitted to OASIS • New crops of specs arrive periodically • WS-Policy*, WS-Trust, WS-Federation, etc. • But… Not yet in any standards organization

  35. WS SecurityCurrent/Proposed WSS-specs WS-Secure Conversation WS-Federation WS-Authorization WS-Policy WS-Trust WS-Privacy WS-Security In progress SOAP Foundation proposed promised

  36. OASIS SAML & XACML • SAML: Security Assertion Markup Language • Good for asserting properties such as group membership, etc • XACML: eXtensible Access Control Markup Language • For defining access control policies • These are gaining considerable momentum, but WS-Policy* leaves them in question

  37. Project Liberty Alliance • V1.x specifications for identity federation • Allows cross-organization identification • Privacy preserving model • Based on SAML

  38. WS-Agreement • Recall key criteria of a Grid: • Coordinates resources that are not subject to centralized control … • using standard, open, general-purpose protocols and interfaces … • to deliver non-trivial qualities of service. • Implies need to express and negotiate agreements that govern the delivery of services to clients • Agreement = what will be done, QoS, billing, compliance monitoring

  39. WS-Agreement Contents • Standard agreement language • A composition of a set of terms that govern a service’s behavior with respect to clients • Agreement language uses WS-Policy (currently) • Standard attributes for terms that express current state of negotiation • Other groups define specific terms • Standard agreement negotiation protocol • Establish, monitor, re-negotiate agreement • Expressed using OGSI GWSDL interfaces • Each agreement represented by a service

  40. WS-Agreement Applicability • All interesting Web/Grid services interactions will be governed by agreements! • WS-Agreement (language and interfaces) should be used by specifications that define domain-specific services • Data services • Job submission • Specialized services • Etc.

  41. Platform’s Community Scheduling Framework & WS-Agreement GT3.0 GT3.0 LSF Queuing Service RM Adapter Internet Job Service Site B Reservation Service GT3.0 PBS Site A RM Adapter Site C Reservation Agreement Exchange GT3.0 SGE RM Adaptor

  42. WSDM / WSMF / CMM • OASIS Web Services Distributed Management (WSDM) technical committee • Management using/of Web Services • HP submitted its Web Services Management Framework (WSMF) to WSDM in July 2003 • WS-Events: event schema, subscription, message queues • WSMF-Foundation: management using Web services • WSM: management of Web services • GGF Common Management Model (CMM) WG • IBM submission overlaps WSMF-Foundation • Working to bring WSDM & CMM together, based on OGSI foundation

  43. Data as Service:OGSA Data Access & Integration • Service-oriented treatment of data appears to have significant advantages • Leverage OGSI introspection, lifetime, etc. • Compatibility with Web services • Standard service interfaces being defined • Service data: e.g., schema • Derive new data services from old (views) • Externalize to e.g. file/database format • Perform queries or other operations

  44. Data Services • GGF Data Access and Integration Svcs (DAIS) • OGSI-compliant interfaces to access relational and XML databases • Needs to be generalized to encompass other data sources (see next slide…) • Generalized DAIS becomes the foundation for: • Replication: Data located in multiple locations • Federation: Composition of multiple sources • Provenance: How was data generated?

  45. “OGSA Data Services”(Foster, Tuecke, Unger, eds.) • Describes conceptual model for representing all manner of data sources as Web services • Database, filesystems, devices, programs, … • Integrates WS-Agreement • Data service is an OGSI-compliant Web service that implements one or more of base data interfaces: • DataDescription, DataAccess, DataFactory, DataManagement • These would be extended and combined for specific domains (including DAIS)

  46. 1a. Request to Registry for sources of data about “x” SOAP/HTTP service creation API interactions Registry 1b. Registry responds with Factory handle 2a. Request to Factory for access to database Factory Client 2c. Factory returns handle of GDS to client 2b. Factory creates GridDataService to manage access 3a. Client queries GDS with XPath, SQL, etc XML / Relational database Grid Data Service 3c. Results of query returned to client as XML 3b. GDS interacts with database Data Access & Integration Services Slide Courtesy Malcolm Atkinson, UK eScience Center

  47. Overview • Grid background • Open Grid Services Architecture • Open Grid Services Infrastructure • Beyond OGSI: other OGSA services • Globus Toolkit v3 implementation • Early GT3 performance results • Scientific and commercial perspectives • Summary

  48. The Globus Alliance • A group of people with a common mission: • “Make Grid computing an everyday reality.” • Argonne/U.Chicago, USC-ISI, EPCC, & KTH-PDC • Led by Ian Foster (Argonne), Carl Kesselman (ISI), Malcolm Atkinson (EPCC), Lennart Johnsson (PDC) • Includes researchers, software developers, software architects & designers, systems engineers, & others • Collaborations (or at least acquaintances) with most Grid activities in the world • All activities contribute to our common mission • Research • Software development (prototypes, reference implns) • Application consulting • Infrastructure consulting

  49. Globus Toolkit:A Story of Evolution • Definition of Grid problem has been stable since original Globus Project proposal in 1995 • Though we’ve gotten better at articulating it • But our approach to its solution has evolved: • From APIs and custom protocols… • to standard protocols… • to Grid services (OGSA) • Driven by experience implementing and deploying the Globus Toolkit, and building real applications with it

  50. Globus Toolkit® v3.0 • All of the GT v2.4 services and clients • Complete Java implementation of OGSI v1.0 • Rich, container-based implementation • Built on Apache Axis • Globus “proprietary” services built on OGSI • Managed Jobs (akin to GT2 GRAM) • Reliable File Transfer (RFT) • Index Services (akin to GT2 GIIS) • Some services not yet OGSI-fied: • GridFTP, Replica Location Services (RLS)

More Related