The Anatomy of the Grid: An Integrated View of Grid Architecture

1. The Anatomy of the Grid: An Integrated View of Grid Architecture Carl Kesselman USC/Information Sciences Institute Ian Foster, Steve Tuecke Argonne National Laboratory

2. Outline and Goals • Define what is (and isn’t) a grid in terms of a layered protocol architecture • Understand the role of protocols, services, APIs and tools in the deployment and use of Grids • Understand what role Globus plays • Understand the role of other technologies

3. What is a Grid? • Grid language has been driven by genesis from metacomputing, but… • In practice, the Grid is about resource sharing and coordinated problem solving in dynamic, multi-institutional virtual organizations • Focus on how to enable, maintain and control the sharing of resources to achieve a common goal

4. Some Useful Definitions • Network Protocol • A formal description of message formats and a set of rules for exchange of messages • Rules define sequences of message exchange, and potentially resulting behavior • Protocol may define state-change in endpoint • Network Enabled Services • Defines a set of capabilities • Protocol defines interaction with service • All services require protocols, although not all protocols are to services

5. More definitions • Resource • Entity that is to be shared • Provides some capabilities, that can be accessed via interface (API) or protocol • Application Programmer Interface (API) • Software Development Kit (SDK) • Package that enables application development, consisting of one or more APIs, and programming tools

6. Protocols Make the Grid • Protocols and APIs • Protocols enable interoperability • APIs enable portability • Sharing is about interoperability, so … • Grid architecture should be about protocols

7. Grid Services Architecture: Previous Perspective … a rich variety of applications ... Applns Appln Toolkits Remote data toolkit Async. collab. toolkit Remote sensors toolkit Remote comp. toolkit Remote viz toolkit ... Protocols, authentication, policy, resource management, instrumentation, discovery, etc., etc. Grid Services Grid Fabric Grid-enabled archives, networks, computers, display devices, etc.; associated local services

8. Characteristics of Grid Services Architecture • Identifies separation of concerns • Isolates Grids from languages and specific programming environments • Makes provisions for generic and application specific functionality • Protocols not explicit in architecture • fails to make clear distinction between language, service and networking issues

9. Layered Grid Protocol Architecture Application User Grid Resource Connectivity Fabric

10. Important Points • Being Grid-enabled requires speaking appropriate protocols • Protocol only requirement, not reachability • Protocols can be used to bridge local resources or “local Grids” • Intergrid as analog to Internet • Built on Internet protocols • Independent of language and implementation • Focus on interaction over network • Services exist at each level

11. User Service APIs and SDKs User Service Protocols User Services Grid Service APIs and SDKs Grid Service Protocols Grid Services Resource APIs and SDKs Resource Service Protocols Resource Services Protocols, services and interfaces Applications Languages/Frameworks Connectivity APIs Connectivity Protocols Local Access APIs and protocols Fabric Layer

12. How does Globus fit in? • Defines connectivity and resource protocols • Enables definition of grid and user protocols • Globus provides some of these, others defined by other groups • Defines range of APIs and SDKs that leverage Resource, Grid and User protocols

13. Fabric • Local access to logical resource • May be real component, e.g. CPU, software module, filesystem • May be logical component, e.g. Condor pool • Protocol or API mediated • Fabric elements include: • SSP, ASP, peer-to-peer, Entropia-like, and enterprise level solutions

14. Connectivity Protocols • Two classes of connectivity protocols underlie all other components • Internet communication • Application, transport and internet layer protocols • I.e., transport, routing, DNS, etc. • Security • Authentication and delegation • Discussed below

15. Security • Protocols • TLS with delegation • Services • K5ssl, Globus Authorization Service • APIs • GSS-API, GAA, SASL, gss_assist • SDKs • GlobusIO

16. Resource Protocols • Resource management, • Storage system access • Network quality of service • Data movement • Resource information

17. Resource Management • Protocols • GRAM+GARA (on HTTP) • Resource services • Gatekeeper, JobManager, SlotManager • APIs and SDKs • GRAM API, JavaCog Client, DUROC

18. Data Transport • Protocols • Grid FTP, LDAP for replica catalog • Services • FTP, LDAP replica catalog • APIs and SDKs • GridFTP client library, copy URL API, replica catalog access, replica selection

19. Resource Information • Protocol • LDAP V3, Registration/Discovery protocol • Service • GRIS • APIs & SDKs • C API; JNDI, PerlLDAP, ….

20. Grid Protocols • Grid Information Index Services • LDAP and Service registration protocol, … • GIIS service • LDAP APIs and specialized information API • Co-allocation and brokering • GRAM (HTTP+RSL) • DUROC service • DUROC client API, end-to-end reservation API

21. Grid Protocols (cont) • Online authentication, authorization services • HTTP • MyProxy, Group policy servers • Myproxy API, GAA API, • Many others (e.g.): • Resource discovery (Matchmaker) • Fault recovery

22. User Protocols • In general, there are many of these, they tend to be on off, and not well defined • Examples: • Portal toolkits (e.g. Hotpage) • Netsolve • Cactus framework

23. Summary • Grids are about sharing, not making big MPI jobs • Well defined protocol architecture is essential to understanding and progress • Provides a framework for figuring out where the pieces fit • Globus provides many basic protocols and services • Interoperability between different Grid components