gLite Overview

1. gLite Overview Christian Grunfeld, UNLP 8th EELA Tutorial, La Plata, 11/12-12/12, 2006

2. Middleware • Grid Middleware – Layer between user applications and grid resources. Segundo Taller Latino Americano de Computación GRID – Primer Taller Latino Americano de EELA – Primer Tutorial Latino Americano de EELA

3. User Interface User Interface Grid services Grid Concepts • VOs:Individuals and/or Institutions having direct access to resources. Segundo Taller Latino Americano de Computación GRID – Primer Taller Latino Americano de EELA – Primer Tutorial Latino Americano de EELA

4. Introduction • Grid Sistems & Applications aim is to: • Integrate • Virtualise • Manage Resources and services across different VOs. Segundo Taller Latino Americano de Computación GRID – Primer Taller Latino Americano de EELA – Primer Tutorial Latino Americano de EELA

5. Grid Requirements • Heterogeneous • VO Resource Sharing • Resource Utilization • Job Execution • Data Services • Security • Scalability • Availability • Specific Requeriments Segundo Taller Latino Americano de Computación GRID – Primer Taller Latino Americano de EELA – Primer Tutorial Latino Americano de EELA

6. Users join VOs Virtual organisation contributes resources & negotiates access Grid middleware runs on each resource “Storage elements” “Compute elements” Additional services (both people and grid middleware) enable the grid Effect: collaboration INTERNET Multi-VOs Segundo Taller Latino Americano de Computación GRID – Primer Taller Latino Americano de EELA – Primer Tutorial Latino Americano de EELA

7. Building on GSI • Build on Grid Security Infrastructure to create services that include: • Job submission: run a job on a remote computer • Information services: So I know which computer to use • File transfer: so large data files can be transferred • Replica management: so I can have multiple versions of a file “close” to the computers where I want to run jobs • Production grids are (currently) based on the Globus Toolkit release 2 Globus Alliance: http://www.globus.org/ Segundo Taller Latino Americano de Computación GRID – Primer Taller Latino Americano de EELA – Primer Tutorial Latino Americano de EELA

8. gLite Segundo Taller Latino Americano de Computación GRID – Primer Taller Latino Americano de EELA – Primer Tutorial Latino Americano de EELA

9. gLite • gLite is the next generation middleware for grid computing. • Born from the collaborative efforts from academic and industrial research centers as part of the EGEE Project. • The gLite Grid services follow a Service Oriented Architecture • facilitate interoperability among Grid services • allow easier compliance with upcoming standards • Architecture is not bound to specific implementations • services are expected to work together • services can be deployed and used independently • The gLite service decomposition has been largely influenced by the work performed in the LCG project Segundo Taller Latino Americano de Computación GRID – Primer Taller Latino Americano de EELA – Primer Tutorial Latino Americano de EELA

10. gLite – Service Decomposition 5 High level services + CLI & API Segundo Taller Latino Americano de Computación GRID – Primer Taller Latino Americano de EELA – Primer Tutorial Latino Americano de EELA

11. gLite – Security Services Segundo Taller Latino Americano de Computación GRID – Primer Taller Latino Americano de EELA – Primer Tutorial Latino Americano de EELA

12. gLite –Security ServicesAuthentication Identify entities (users, systems and services) when establishing context for message exchange (Who are you?). Aim - Provide a Credential having a universal value that works for many purposes across many infrastructures, communities, VOs and projects. gLite uses the PKI (X.509) infrastructure using CAs as thrusted third parties. MyProxy(http://grid.ncsa.uiuc.edu/myproxy/) Trust domain: The set of all EGEE CAs is our Trust Domain. • Revocation: Identities must be revocated timely • Credential Storage: Local or delegated credential (Services or Users) • Privacy Preservation: Use of personal data Segundo Taller Latino Americano de Computación GRID – Primer Taller Latino Americano de EELA – Primer Tutorial Latino Americano de EELA

13. gLite –Security ServicesAuthorization Allows or denies accessto services, based on policies. • Agent: The user interacts with a centralized Authorization Server • Push*: Authorization Services issue Tokens. • Pull: The resource asks to the Authorization Services. Authorization Sources: • Attribute Authority (AA): User <-> Set of Attibutes. (VOMS) Segundo Taller Latino Americano de Computación GRID – Primer Taller Latino Americano de EELA – Primer Tutorial Latino Americano de EELA

14. gLite – Grid Access Two possibilities: APIs and CLI. The use of web-services allows the automatic generation of APIs (error prone, lack of tools). Segundo Taller Latino Americano de Computación GRID – Primer Taller Latino Americano de EELA – Primer Tutorial Latino Americano de EELA

15. gLite – Information and Monitoring Services Information services are vital low level component of Grids. Segundo Taller Latino Americano de Computación GRID – Primer Taller Latino Americano de EELA – Primer Tutorial Latino Americano de EELA

16. gLite – Information and Monitoring ServicesBasic info and monitoring services (RGMA) • Information is provided by a Publish and Consume mechanism. • Appearance of a single federated database to query through the SQL. • Each VO has a VDB. • Schema - Contains tables (GLUE) • Registry – List of available sources of information (Mediation) • Producers – Source of information (Primary, Secondary, On-demand) • Consumers – Make queries against tables (Continuous, Latest, History) Segundo Taller Latino Americano de Computación GRID – Primer Taller Latino Americano de EELA – Primer Tutorial Latino Americano de EELA

17. gLite – Information and Monitoring ServicesJob Monitoring, Service Discovery, Network performance Monitoring • Job Monitoring – Java logging service, log4j, Apache/Chainsaw (for other languages). • Service Discovery – Locates suitable services to both users and services (Library!). • Network Performance Monitoring – Many network monitoring frameworks. Aim: perform a standard interface to those frameworks. Segundo Taller Latino Americano de Computación GRID – Primer Taller Latino Americano de EELA – Primer Tutorial Latino Americano de EELA

18. gLite – Job Management Services Segundo Taller Latino Americano de Computación GRID – Primer Taller Latino Americano de EELA – Primer Tutorial Latino Americano de EELA

19. gLite – Job Management ServicesAccounting Accumulates information about the resource usage done by users or groups of users (VOs). Information on Grid Services/Resources needs sensors (Resource Metering, Metering Abstraction Layer, Usage Records). Records are collected by the Accounting System (Queries: Users, Groups, Resource) Grid services should register themselves with a pricing service when accounting for billing purposes. Segundo Taller Latino Americano de Computación GRID – Primer Taller Latino Americano de EELA – Primer Tutorial Latino Americano de EELA

20. gLite – Job Management ServicesComputing Element • Service that represent the computing resource that is responsible of the job management: (submission, control, etc.) • CEs refer to a set or a cluster of computational resources (WN) managed by LRMS, to dispatch jobs matching users requests. • Two job submission models (accordingly to user requests and site policies): • PUSH (jobs pushed to CE queue), • PULL (jobs coming from WMS when CE queue is empty) • CE responsible to collect accounting information. Segundo Taller Latino Americano de Computación GRID – Primer Taller Latino Americano de EELA – Primer Tutorial Latino Americano de EELA

21. gLite – Job Management ServicesWorkload Management • WMS set of middleware components responsible of distribution and management of jobs across Grid resources. • Two core components of WMS: • WM: accept and satisfyrequests for job management.Matchmaking is the process of assigning the best available resource. • L&B: keeptrack of job execution in term of events: (Submitted, Running, Done,...) Segundo Taller Latino Americano de Computación GRID – Primer Taller Latino Americano de EELA – Primer Tutorial Latino Americano de EELA

22. gLite – Job Management ServicesJob Provenance, Package Manager Job Provenance (JP) - Keeps track of submitted jobs for long periods (months, years). Package Manager – Helper service to automate: installing, configuring, updating and removing of software components. (RPM, dpkg/APT, Portage, …) Segundo Taller Latino Americano de Computación GRID – Primer Taller Latino Americano de EELA – Primer Tutorial Latino Americano de EELA

23. gLite – Data Services Segundo Taller Latino Americano de Computación GRID – Primer Taller Latino Americano de EELA – Primer Tutorial Latino Americano de EELA

24. gLite – Data ServicesStorage Element Needed Service are at least: • Storage back-end (Drivers and Hardware) • SRM Interface (Storage Specific) • Transfer service (GridFTP) • Native POSIX like file I/O API (gLite-I/O) • Auxiliary Accounting and Logging services Segundo Taller Latino Americano de Computación GRID – Primer Taller Latino Americano de EELA – Primer Tutorial Latino Americano de EELA

25. gLite – Data ServicesCatalogs OS like file access metaphor. • LFN (Logical file name) • GUID (Grid unique identifier) • SimLinks • SURL (Site URL) • TURL (Transfer URL) Catalogs: • Authorization Base • Metadata Base • Metadata Schema • Replica Catalog • File Catalog • File Authorization • Metadata • Combined Catalog • Storage Index gLite - (FireMAN) Segundo Taller Latino Americano de Computación GRID – Primer Taller Latino Americano de EELA – Primer Tutorial Latino Americano de EELA

26. gLite – Helper Services Configuration and Instrumentation Service – Query service state. Agreement Service – Implements a communication protocol for the SLAs. Bandwidth Allocation & Reservation service (BAR) – Controlling, Balancing and Manage Network flows. Segundo Taller Latino Americano de Computación GRID – Primer Taller Latino Americano de EELA – Primer Tutorial Latino Americano de EELA

27. Components • Site: • Computing Element (CE) • Gateway to local computing resources (cluster de worker nodes) • Worker Nodes (WN) • Storage Element (SE) • Gateway to local storage (disk, tape) • A gridftp server, and SRM Interface, IO server • User Interfaces (UI) • User’s access point to the grid • Client programs using some/all grid services. Segundo Taller Latino Americano de Computación GRID – Primer Taller Latino Americano de EELA – Primer Tutorial Latino Americano de EELA

28. Components • Grid – or VO- wide • Security • Virtual Organization Server (VOMS) • MyProxy server (Proxy) • Information System • Job handling • Workload Management System (WMS) • Logging & Bookkeping (LB) • Data management • File catalog (FiReMan) • File Transfer Service (FTS) • File Placement Service (FPS) Segundo Taller Latino Americano de Computación GRID – Primer Taller Latino Americano de EELA – Primer Tutorial Latino Americano de EELA

29. VOMS • Virtual Organization Membership Service • Multiple VOs • Multiple roles in VO • Compatible X509 extensions • Signed by VOMS server • Web admin interface • Supports MyProxy • Resources providers grant access to VOs or roles • Sites map VO members/roles to local auth mechanism (unix users accounts) • Allows for local policy Segundo Taller Latino Americano de Computación GRID – Primer Taller Latino Americano de EELA – Primer Tutorial Latino Americano de EELA

30. MyProxy • MyProxy • Allows longer lived jobs / increases security • WMS renews proxy • Users should not produce long lives proxies • Allows for secure user mobility • Users does not need to copy globus-keys around Segundo Taller Latino Americano de Computación GRID – Primer Taller Latino Americano de EELA – Primer Tutorial Latino Americano de EELA

31. References • Globus Alliance: http://www.globus.org • gLite middleware: http://glite.web.cern.ch/glite • gLite 3.0 user guide: https://edms.cern.ch/file/722398//gLite-3-UserGuide.html Segundo Taller Latino Americano de Computación GRID – Primer Taller Latino Americano de EELA – Primer Tutorial Latino Americano de EELA