An Open Grid Service Architecture (OGSA)

An Open Grid Service Architecture(OGSA) Tiziana.Ferrari@cnaf.infn.it INFN – CNAF Corso di Laurea specialistica in Informatica Anno Acc. 2004/2005 Slide sources: 1 Globus Toolkit Futures: An Open Grid Services Architecture 2. Introduction to Web Services, Andrea Ceccanti 3. Web Services Survey, Giuseppe Avellino An Open Grid Service Architecture

Outline • PART I: ripasso • PART II: Web Services technology in brief • PART III: OGSA (An Open Grid Services Architecture) • PART IV: OGSA interfaces • References An Open Grid Service Architecture

PART IRipasso An Open Grid Service Architecture

Grid Computing (1) • Purpose of Grid Computing: flexible, secure, coordinated resource sharing among dynamic collections of individuals, institutions, and resources From “The Anatomy of the Grid: Enabling Scalable Virtual Organizations” • Virtual organization (VO): a set of individuals and/or institutions identified by the same set of rules, which define: • the resources shared (what); • the individual users allowed to share (who); • conditions under which sharing occurs (how). An Open Grid Service Architecture

Grid Computing (2) • “A computational Grid is a hardware and software infrastructure that provides dependable, consistent, pervasive and inexpensive access to high-end computational capabilities” • Infrastructure: the Grid pools a large number of heterogeneous resources, and consequently requires a complex a hardware infrastructure for the necessary interconnections, and a software infrastructure to monitor and control the resulting compound system. • Dependable (“affidabile”): Grid computing needs to deliver predictable, continuous (and often high-level) performance from different components. Examples of performance guarantees: network bandwidth, latency, jitter, computer power, security, reliability, service response time, etc. • Consistent: services need to be accessed through standard interfaces. • Pervasive (“diffuso”): users need to count on universal access within the confines of the environment the Grid is designed to support – like in the case of the World Wide Web – thanks to the large scale of a grid computing infrastructure. • Inexpensive access to high-end computational capabilities An Open Grid Service Architecture

Grid Computing Basic Properties (3) • The three fundamental properties of Grid computing: • Large-scale coordinated management of resources belonging to different administrative domains (multi-domain vs single domain)  Grid computing involves multiple management systems • Standard, open, multi-purpose protocols and interfaces that provide a range of services (standard vs proprietary) • Grid computing supports heterogeneous user applications • Delivery of complex Quality of Service (QoS): Grid computing allows its contituent resources to be used in a coodinated fashion to deliver various types of QoS, such as resposed time, throughput, avaiability, reliability, security, etc. An Open Grid Service Architecture

Grid Architecture: Definition • The Grid Architecture: • identifies the fundamental system components: • services; • protocols; • users, organized in VOs. • specifies purpose and function of these components; • indicates how these components interact with each other. • Grid architecture protocols define the basic communication mechanisms by which: • Virtual Organization users and resources negotiate, establish, manage and exploit sharing relationships; • Different services interact. • Grid architecture services are the components whose standardization facilitates extensibility, interoperability, portability and code sharing. The term service includes the Grid resources, as a Grid resource instance offers specific capabilities within the Grid infrastructure. An Open Grid Service Architecture

Grid Archticture: Definition (cont) U7 U8 U6 VO2 1, VOs and their users 2. Services 3. Protocols: - between services - between services and VOs protocol protocol S2 U1 S3 S1 protocol U2 protocol protocol protocol S4 U5 VO1 U3 U4 An Open Grid Service Architecture

Application Application Internet Protocol Architecture “Coordinating multiple resources”: ubiquitous infrastructure services, app-specific distributed services Collective “Sharing single resources”: negotiating access, controlling use Resource “Talking to Grid elements”: communication (Internet protocols) & security Connectivity Transport Internet “Controlling elements locally”: Access to, & control of, resources Fabric Link Layered Grid Architecture(by Analogy to the Internet Architecture) Grid Architecture Internet Architecture An Open Grid Service Architecture

Part II Web Services technology in brief An Open Grid Service Architecture

Problem Today most of the Grid prototypes suffer from: • protocol deficiencies, e.g. • heterogeneous set of protocols: HTTP (Hyper Text Transfer Protocol), LDAP (Lightweight Directory Access Protocol ), FTP (File Transfer Protocol), etc. • no standard means of invocation, notification, error propagation, authorization, termination, … • significant missing functionality, e.g. • modeling of resources such as: databases, sensors, instruments, etc. • Handling of complex workflows (such as the execution of a set of jobs with cross-dependences), … • Independence from the type of the end-systems running the Grid software (i.e. the hosting environments) • little work on total system properties, e.g. • Dependability, end-to-end Quality of Service, etc. An Open Grid Service Architecture

The Web Services technology in brief: • The Web Services are an increasingly popular standards-based framework for accessing applications through the network. Standardization is in progress at the World Wide Web Consortium (W3c, http://www.w3c.org/), which has been active since 10 years ago. The list of companies involved includes: Microsoft, IBM, Sun, others. • The W3C organization defines a Web Service as: “a Web Service is a software system identified by a URI whose public interfaces and bindings are defined and described using XML”. And: “its definition can be discovered by other software systems. These systems may interact with the Web Services in a manner prescribed by its definition, using XML-based messages conveyed by Internet protocols”. • Web services provide a way to offer application-to-application communication running on different platforms (interoperability). • WSDL: Web Services Description Language: the language used to describe the interface to a give service (see following slide for more information). An Open Grid Service Architecture

WSDL • A “Web Service Description” (WSD) is a machine-processable document that describes the expectations and functionality of a particular Web Service instance, so that a potential client can read the description and understand how to correctly interact with the service. The service functionality is described in terms of messages exchanged between service consumers and service providers. • WSDs are intended to be used both by clients and services. • Client and service providers need to agree on the service semantics. This is achieved by adding a pointer to the WSD, where the pointer refers to a document describing the service semantics. • Web Service descriptions are in XML format. • Message: it is used to exchange data structures, whose types are defined in the XML schema datatype system. • Operation: message exchange patterns supported by the Web Service, which define for example, the input, output and fault messages used for providing a given WS function. • PortTypes: also known as “interfaces” in WSDL 1.2, they are named collections of “operations”. An Open Grid Service Architecture

Web Service Technologies • SOAP: Simple Object Access Protocol: XML-based RPC protocol used to exchange structured and typed information between peers in a decentralized, distributed environment. HTTP is the transport protocol. • WS-Inspection: its specification defines • an XML format for assisting in the inspection of a site for available services; • a set of rules for how inspection related information should be made available for consumption. • UDDI: Universal Desc., Discovery, & Integration: a directory for Web services. In particular, it defines a set of services for: • the discovery of Web service providers; • the Web services made available; • the interfaces with may be used to access those services. An Open Grid Service Architecture

Web Service Technologies (cont) UDDI Registry Request Service Info (WSDL) WSDL Web Service Client Download WSDL Web Service Send SOAP messages with XML document An Open Grid Service Architecture

PART IIIAn Open Grid Services Architecture An Open Grid Service Architecture

The Need to SupportTransient Service Instances in Grids • The “Web services” technology traditionally addresses discovery & invocation of persistent services. • However, Grids, must also support transient service instances, created/destroyed dynamically: • Interfaces to the states of distributed activities • E.g. workflow, video conf., dist. data analysis • Significant implications for how services are managed, named, discovered, and used • In fact, much of our work is concerned with the management of service instances An Open Grid Service Architecture

An Open Grid Service Architecture • The OGSA service model: • It models the Grid architecture following a service-oriented approach, i.e. “Everything is represented by a Service: a network enabled entity that provides some capabilities through the exchange of messages.” • Resources such as computational resources, storage, networks, programs, databases, etc. are allservices. • OGSA defines a set of basic service interfaces (i.e. WSDL portTypes) for manipulating abstractions that follow the service model. It does not defines specific set of services, which would result in a specific rendering of the OGSA service model. All services comply to specified Grid service interfaces and behaviors • OGSA interfaces can be composed arbitrarily to produce a range of high-level services. • System comprises (a typically few) persistent services and (potentially many) transient (with limited lifetime) services. In OGSA all components of the environment are virtual An Open Grid Service Architecture

Grid Architecture and OGSA • While the Grid Architecture models the entire Grid infrastructure by defining components and functionalities of broad services classes, OGSA models the interface of a generic Grid service, by assuming that any capability in the Grid can be virtualized and represented by the “Grid Service abstraction”. An Open Grid Service Architecture

The Grid Service • Definition of Grid service: “a web service that conforms to a set of conventions and supports standard interfaces, where each interface defines a set of operations that are invoked by exchanging a defined sequence of messages. • The availability of a core set of consistent interfaces allows the creation of higher-order services that can be treated in a uniform way across layers of abstraction. • OGSA represents everything as a Grid Service. • Grid service interface: WSDL portType • Grid service instance: a particular instantiation of a Grid service. • A Grid service can maintain state for the lifetime of the service: state distinguishes different instances of the same service. • Protocol binding: defines a messagedelivery semantics associated to an interface An Open Grid Service Architecture

The Grid Service (cont) • Every Grid service is assigned a globally unique name. Grid Service stateful dynamic • OGSA service scan be created and • destroyed dynamically. Services can • Be destroyed: • Explicitly; • Implicitly, as a result of some system failure such as operting system crash or network partition. State is affected by the sequence of messages received, hence, reliability of message delivery is important to produce the expected state in a remote Grid service instance. An Open Grid Service Architecture

Grid Service Handle (GSH) vs Reference (GSR) • The Grid Service Handle (GSH) distinguishes a specific Grid service instance from all other instance that have existed, exist now, or will exist in the future. In order to be unique, it is based on the name of a home mapper service. • In case of service failure and restart, the same handle is preserved in order to keep the service state. • The Grid Service Reference (GSR): while the GSH carries not protocol- or instance-specific information (e.g. IP address, protocol binding, etc.), this information, which is required to interact with a specific service instance, is carried in the GSR. • GSH: invariant; GSR: can change over the service lifetime and has an explicit expiration time. • Holding a valid GSR does not guarantee access to a Grid service instance, as local policies control the acess through control contraints. An Open Grid Service Architecture

OGSA interfaces and their composition • The intial OGSA proposal only included a basic set of Grid service interfaces (where names were non-normative). The OGSA proposal did not include a comprehensive list of interfaces (for example, interfaces for authorization, policy management where not included). • All OGSA interfaces are optional with the only exception of the GridService interface, which must supported by all Grid services. • Interfaces can be arbitrarily composed to produce high-level services. An Open Grid Service Architecture

The four OGSA service requirements • Discovery: • Applications require mechanisms for discovering available services and for determining the characteristics of those service, so that they can decide the service instance to which requests need to be addressed. •  RegisterService/UnregisterService (Registry) operations •  FindServiceData (GridService) operations: services are described by service data elements, which are XML elements with a specific name and type. Their representation is standard. •  FindByHandle (HandleMap): handles   references • Dynamic service creation: • service instances need to be created dynamically •  Factory Interface An Open Grid Service Architecture

The four OGSA service requirements (cont) • Lifetime management • Distributed systems need to deal with the problem of system failures. In order to manage transient and stateful services, mechanisms must be provided to: • Reclaim services (in case of failure of other service instances they depend on); •  SetTerminationTime (GridService interface): for soft state lifetime management. The term “soft state” refers to the need to release resources allocated by a Grid service instance, after a timeout, i.e. when a service instance is not refreshed by a stream of keepalive messages. This allows to prevent deadlock (total allocation of the resources available). •  Destroy (GridService interface) • Reclaim state associated with failed operations An Open Grid Service Architecture

The four OGSA service requirements (cont) • Notification • Services need to notify each other asynchronously of interesting changes to their state. Subscription and delivery of such notifications is need for this. • For example, a high-level service needs to enter a failure state when at least one of the component services fails. This can be achieved through the notification of failure states from components. •  NotificationSource interface •  NotificationSink interface • But: other requirements will be defined (OGSA is an open architecture, which allows extensibility). An Open Grid Service Architecture

PART IVOGSA Interfaces An Open Grid Service Architecture

GridService PortType Operations: • FindServiceData: to query a variety of information about the Grid service instance, including: • basic introspection information (handle, reference, etc.), • per-interface information, • service-specific information (e.g. Service instances known to a registry). • Pull model: information is explecitely requested by the entity which performed the query. • SetTerminationTime: set and get termination time of a Grid service instance. • Destroy: terminate Grid service instance An Open Grid Service Architecture

Notification-Source, Notification-Sink PortTypes Notification-Source Operation: • SubscribeTo-NotificationTopic: • subscribe to notifications of service-related events, based onmessage type and interest statement. Allows for delivery via third party messaging services. • It supports the push model, i.e. information is directly distributed by the source. Notification-Sink Operation: • DeliverNotification: carry out asynchronous delivery of notification messages. An Open Grid Service Architecture

Registry, Factory and HandleMap PortTypes Registry Operations: • RegisterService: conduct soft-sate registration of Grid service handles. Handles need to be periodically refreshed (this is needed in case of failure of the Registry). • UnregisterService: deregister a Grid service handle Factory Operations: • CreateService: create nwe Grid service instance HandleMap Operations: • FindByHandle: return a Grid Service Reference (GSR) currently associated with the supplied Grid Service Handle (GSH) An Open Grid Service Architecture

Hosting environments • OGSA only defines interfaces and operations, while does not place requirements on: • functions of a Grid service; • how the functions are implemented. • A hosting environment is the execution environment associated to a Grid service instance. It defines: • the programming language (e.g. C, C++, Java, Fortran, etc.) • development tools (e.g. in case of Web Services: Webshere – IBM, .NET – Microsoft, Sun One – Sun, J2EE, etc.) • the debugging tools, • how a Grid service instance delivers the semantic functions associated to it. • Types of hosting environments: • Simple hosting environment; • Virtual hosting environment; • Collective services. An Open Grid Service Architecture

Simple hosting environment • Characteristics: • Resources are all part of the same administrative domain. • A single Grid service management facility is available. • The interface of the user to the environment comprises: • a registry; • one or more factories. When a factory receives a creation request, the creation is implemented through a list of operations. Each operation directly maps to local hosting-environment-specific capabilities: • creation of a Grid service instance; • a handle is assigned to the service instance; • the instance is register within the local registry service; • the Handle is made available to the HandleMap service. • a HandleMap service. • Example: Grid service for allocation of disk space on a storage element. create Assign handle register Handle in HandleMap An Open Grid Service Architecture

Virtual Hosting Environment • In this case resources span heterogeneous, geographically distributed simple hosting environments, and are owned by a set of Virtual Organizations. • The virtual hosting environment is always accessed by the user through the same interfaces used for the simple hosting environment. However, in this case: • high-level factories are available to delegate creation requests to lower-level factories. • a high-level registry only knows about: • the high-level factries and the service instances created; • The policies that regulate the access to the resources of VOs. • high-level registries and factories support standard Grid service interfaces. • Example: Grid service that ensures that a given computation finishes by a specified end-time ( allocation of multiple CPUs across several Grid sites, where the number of CPUs depends on the finish time) An Open Grid Service Architecture

Collective services • Collective operations are a specific example of “virtual hosting environment” whereVO participants can use complex, collective (see also the corresponding layer in the Grid Architecture) end-to-end services. • In particular, in this case “higher-level service instances are implemented by asking lower-level factories to create multiple service instances, and by composing the behaviors of those multiple lower-level service instances”. On the contrary, in the previous hosting environment example, lower-level services are independent and not composed together. • Example: service for file transfer with guaranteed bandwidth. For each network administrative domain, a given amount bandwidth on a specific interconnecting path has to be allocated. Several Grid service instances in each domain need to be invoked. The techniques (e.g. the network protocols) used to enforce the guranatees in each domain, need to be compatible. An Open Grid Service Architecture

(a) Simple Hosting Environment (b) Virtual Hosting Environment (c) Compound Services Registry Registry E2E Factory Factory E2E Reg Service Service Factory ... ... H2R H2R E2E H2R Factory Factory Mapper Mapper Mapper ... ... ... Service Service Service Service Service Service E2E S E2E S E2E S F F R R R R F F 1 2 M M M M F F S S S S S S S S S S S S Examples of different hosting environments In each case, Registry handle is effectively the unique name for the virtual organization. An Open Grid Service Architecture

Reference • The Physiology of the Grid: An Open Grid Services Architecture for Distributed Systems Integration; I.Foster, C.Kesselman, J.M.Nick, S.Tuecke. An Open Grid Service Architecture

Network protocol bindings • Protocols used to exchange messages with remote Grid service instances need to satisfy four requirements: • Reliable transport: services need to be invoked reliably; in order to do so, service invocation needs to cope with message loss. • Authentication and delegation: credentials of the user who originally issued a request to a service abstraction, are carried along every time a new service is invoked. The credentials are initially released by a trusted body. In case of a service request that triggers the request of other services, the requesting service behaves as a user proxy. • Ubiquity: an arbitrary set of services need to interact, regardless of their specific implementations. • GSR format: references take a binding-specific format, and hence, they depend on the service implementation. An Open Grid Service Architecture

An Open Grid Service Architecture (OGSA)