Comparison of LCG-2 and gLite

1. Author E.Slabospitskaya Location IHEP Comparison of LCG-2 and gLite

2. Outline • What are LCG-2 and gLite? Roles. • gLite Architecture review • What is gLite? Provenance of gLite from Alien, Grid3 et al. • Schema of gLite Services. The main subsystems of gLite. Comparison to LCG-2. • Workload management. • Data management • Information System • Security • Accounting • Alien • Current status of gLite • Mapping to physical machines

3. Future EGEE middleware http://glite.web.cern.ch/glite VDT EDG . . . AliEn LCG . . . gLite-1 gLite-2 LCG-1 LCG-2 Globus 2 based Web services based • Architecture: https://edms.cern.ch/document/476451 • Design: https://edms.cern.ch/document/487871/

4. Provenance of gLite The gLite middleware is a Service Oriented Grid middleware providing services for managing distributed computing and storage resources and the required security, auditing and information services. Target server platform is Red Hat Linux 3.0 or any binary compatible distribution, such as Scientific Linux, and Windows. AliEn is a lightweight Grid framework which is built around Open Source components using the Web Services model. It has been initially developed by the ALICE collaboration as a production environment for the simulation, reconstruction, and analysis of Physics data in a distributed way. The gLite Middleware is presently using a number of services taken from the AliEn system, namely the File Catalogue and the Task Queue (part of the AliEn Core Services), the Package Manager (part of the AliEn Site Services) and the GAS or Grid Access Service.

5. gLite and LCG-2 LCG-2focus on production, large-scale data handling Theservice for the 2004 data challenges Provides experience on operating and managing a global grid service Development programme driven by data challenge experience Data handling Strengthening the infrastructure Operation, VO management Evolves to LCG-3 as components progressively replaced with new middleware-- target is to minimise the discontinuities of migration to the new generation Aim for migration plan by end of year LCG LCG-2 (=EGEE-0) gLitefocus on analysis • Developed by EGEE project in collaboration with VDT (US) • LHC applications and users closely involved in prototyping & development (ARDA project) • Short development cycles • Co-existence with LCG-2 • Profit as far as possible from LCG-2 infrastructure, experience  Ease deployment – avoid separate hardware • As far as possible - completed components integrated in LCG-2  improved testing, easier displacement of LCG-2 2004 prototyping prototyping product 2005 product LCG-3 EGEE-1 les robertson - cern-it-5

6. gLite Services Service decomposition based on ARDA blueprint

7. GAS The Grid Access Service provides access to different Grid Services and acts as an adapter exposes a flat interface to the collection of components File Catalog Metadata catalogue Client GAS WMS API*- for appl

8. Job Management • A CE interfaces the local resource management system (e.g. LSF, PBS) to the Grid middleware. The currently released CE will be enhanced with a CE Monitor that pulls jobs from the WMS. • The Worker Nodes behind the local resource management system host all the necessary clients to interact with the Grid middleware from within a job. • The Package Manager allows the dynamic installation of application software. .

9. WMS

10. AliEn Task Queue and EDG/LCG RB L & B SUBMIT GAS MONITOR REQUEST WMS PUSH CE TASK QUEUE Submit jobs PULL User

11. CE • Works in push and pull mode • Site policy enforcement • Exploit new globus GK and CondorC (close interaction with globus and condor team) CEA … Computing Element Acceptance JC … Job Controller MON … Monitoring LRMS … Local Resource Management System

12. Data Management 3 main service groups that relate to data and file access are: Storage Element, Catalogue Services Data Transfer Scheduling . Closely related to the data services are the security-related services and the Package Manager. . • Data is stored in an SRM based storage system. • The gLite-I/O server allows posix-like access to files stored in the SRM • The local catalogue (LC) keeps track of the LFN:GUID:SURL mapping of local files The file transfer/file placement service is used for moving files.

13. Catalogs File Catalog Filesystem-like view on logical file names Keeps track of sites where data is stored Conflict resolution • Replica Catalog • Keeps information at a site • (Meta Data Catalog) • Attributes of files on the logical level • Boundary between generic middleware and application layer Metadata Catalog Metadata File Catalog GUID Site ID LFN Site ID Replica Catalog Site A Replica Catalog Site B LFN LFN GUID SURL GUID SURL SURL SURL

14. Storage Element Interfaces SRM interface Management and control SRM (with possible evolution) Posix-like File I/O File Access Open, read, write Not real posix (like rfio) Control SRM interface POSIXAPI File I/O User rfio dcap chirp aio dCache NeST Castor Disk

15. Information System TheR-GMA (Relational Grid Monitoring Architecture) servlet accepts connection from clients (producers), i.e. the services publishing and user jobs publishing information, and forwards the information to the appropriate consumers. . R-GMA is composed of the following services: a.    R-GMA server b.    R-GMA client is a set of client API in C,C++,Java and Python for the access the information and monitoring functionality of the R-GMA system c.    R-GMA site-publisher (each site) is responsible for publishing site information to the R-GMA server. d.  R-GMA service tool -regularly scans config files and updates a current Service Status R-GMA Table. CLI exists for modify these files and query the service table.

16. R-GMA R-GMA is based on the Grid Monitoring Architecture (GMA) from the Grid Global Forum (GGF), which is a simple Consumer-Producer model that models the information infrastructure of a Grid as a set of consumers (that request information), producers (that provide information) and a central registry which mediates the communication between producers and consumers. R-GMA offers a global view of the information as if each Virtual Organization had one large relational database. Producers contact the registry to announce their intention to publish data, and consumers contact the registry to identify producers, which can provide the data they require. The data itself passes directly from the producer to the consumer: it does not pass through the registry. R-GMA adds a standard query language (a subset of SQL) to the GMA model,

17. Mapping gLite Services onto physical machines

18. gLite Services and components The following high level services are part of gLite middleware --Accounting Service (DGAS) - AliEn File Catalogue (global catalogue – GC) - File & Replica Catalogue (local catalogue – LC) - Metadata Catalogue - Computing Element (CE) - File Transfer and Placement Service - gLite-I/O Server and Client - Logging and Bookkeeping Server (LB) - Package Manager - • R-GMA Servers, Client and Service Tools • VOMS and VOMS administration tools • Authorization, Authentication and Delegation Services • Standard Worker node (WN) • User Interface • Workload Manager System (WMS) • Grid Access Service (GAS) • Job Provenance