1 / 13

Overview of middleware

Overview of middleware. Alexandre Duarte CERN IT-GD-OPS UFCG LSD 1st EELA Grid School Itacuruçá, 04-15/12/2006. Input “sandbox”. DataSets info. UI JDL. Output “sandbox”. voms-proxy-init. SE & CE info. Output “sandbox”. Expanded JDL. Job Submit Event. Job Query.

rhonda-caldwell
Download Presentation

Overview of middleware

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Overview of middleware Alexandre Duarte CERN IT-GD-OPS UFCG LSD 1st EELA Grid School Itacuruçá, 04-15/12/2006

  2. Input “sandbox” DataSets info UI JDL Output “sandbox” voms-proxy-init SE & CE info Output “sandbox” Expanded JDL Job Submit Event Job Query Input “sandbox” + Broker Info Publish Job Status Storage Element Globus RSL Job Status Job Status Job Workflow in gLite LFC Catalog Information Service Resource Broker Author. &Authen. Job Submission Service Logging & Book-keeping Computing Element 1st EELA Grid School, Itacuruçá, 04-15.12.2006

  3. Input “sandbox” DataSets info UI JDL Output “sandbox” voms-proxy-init SE & CE info Output “sandbox” Expanded JDL Job Submit Event Job Query Input “sandbox” + Broker Info Publish Job Status Storage Element Globus RSL Job Status Job Status Job Workflow in gLite LFC Catalog Information Service Resource Broker Author. &Authen. Job Submission Service Logging & Book-keeping Computing Element 1st EELA Grid School, Itacuruçá, 04-15.12.2006

  4. Middleware structure • Applications have access both to Higher-level Grid Services and to Foundation Grid Middleware • Higher-Level Grid Services are supposed to help the users building their computing infrastructure but should not be mandatory • Foundation Grid Middleware will be deployed on the EGEE infrastructure • Must be complete and robust • Should allow interoperation with other major grid infrastructures • Should not assume the use of Higher-Level Grid Services 1st EELA Grid School, Itacuruçá, 04-15.12.2006

  5. Grid Foundation: Security • Authentication based on X.509 PKI infrastructure • Certificate Authorities (CA) issue (long lived) certificates identifying individuals (much like a passport) • Commonly used in web browsers to authenticate to sites • Trust between CAs and sites is established (offline) • In order to reduce vulnerability, on the Grid user identification is done by using (short lived) proxies of their certificates • Proxies can • Be delegated to a service such that it can act on the user’s behalf • Include additional attributes (like VO information via the VO Membership Service VOMS) • Be stored in an external proxy store (MyProxy) • Be renewed (in case they are about to expire) 1st EELA Grid School, Itacuruçá, 04-15.12.2006

  6. Authentication Request OK C=IT/O=INFN /L=CNAF/CN=Pinco Palla/CN=proxy Query AuthDB VOMSAC VOMSAC VOMS client Grid Foundation: VOMS Virtual Organization Membership Service: • Extends the proxy with info on VO membership, group, roles • Fully compatible with GSI • Each VO has a database containing group membership, roles and capabilities informations for each user • User contacts VOMS server requesting his authorization info • Server sends authorization info to the client, which includes it in a proxy certificate [sam@egris195 sam]$ voms-proxy-init -voms gilda Your identity: /C=IT/O=GILDA/OU=Personal Certificate/L=ITACURUCA/CN=ITACURUCA13/Email=tony.calanducci@ct.infn.it Enter GRID pass phrase: Creating temporary proxy ............................ Done Contacting egris197.eela.ufrj.br:15001 [/C=BR/O=ICPEDU/O=UFF BrGrid CA/O=UFRJ/OU=IF/CN=host/egris197.eela.ufrj.br] "gilda" Done Creating proxy .............................................. Done Your proxy is valid until Tue Dec 5 10:55:10 2006 1st EELA Grid School, Itacuruçá, 04-15.12.2006

  7. Grid foundation: Information Systems • BDII: Information system in gLite 3.0 (by LCG) • LDAP database that is updated by a process • More than one DBs is used separate read and write • A port forwarder is used internally to select the correct DB • R-GMA: provides a uniform method to access and publish distributed information and monitoring data • Used for job and infrastructure monitoring in gLite 3.0 1st EELA Grid School, Itacuruçá, 04-15.12.2006

  8. Grid foundation: Computing Element • gLite-CE: based on GSI enabled Condor-C • Coming from LCG-CE (based on GT2 GRAM) • Supported by Condor. More efficient. • Deployed for the first time in gLite 3.0 • CREAM: new lightweight CE • Not yet in gLite 3 release. Will need exposure to users on dedicated system. • WSDL interface • Will support bulk submission of jobs from WMS and optimization of input/output file transfer. • Plans are to have a CE with both Condor-C and CREAM interfaces 1st EELA Grid School, Itacuruçá, 04-15.12.2006

  9. Grid foundation: Storage Element • Storage Element • Common interface: SRMv1,migrating to SRMv2 • Various implementation from LCG and other external projects • disk-based: DPM, dCache / tape-based: Castor, dCache • Support for ACLs in DPM (in future in Castor and dCache) • Common rfio library for Castor and DPM being added • Posix-like file access: • Grid File Access Layer (GFAL) by LCG • Support for ACL in the SRM layer (currently in DPM only) • Support for SRMv2 being added 1st EELA Grid School, Itacuruçá, 04-15.12.2006

  10. High Level Services: File transfer • FTS: Reliable, scalable and customizable file transfer • Manages transfers through channels • mono-directional network pipes between two sites • Web service interface 1st EELA Grid School, Itacuruçá, 04-15.12.2006

  11. High Level Services: Workload mgmt. • WMS helps the user accessing computing resources • Resource brokering, management of job input/output, ... • gLite-RB: GT2 + Condor-G • To be replaced when the gLite WMS proves to be reliable • gLite WMS: Web service (WMProxy) + Condor-G • Management of complex workflows (DAGs) and compound jobs • bulk submission and shared input sandboxes • support for input files on different servers (scattered sandboxes) • Job File Perusal: file peeking during job execution • Supports collection of information from CEMon, BDII, R-GMA and from DLI and StorageIndex data management interfaces • Support for parallel jobs (MPI) when the home dir is not shared • Deployed for the first time in gLite 3.0 1st EELA Grid School, Itacuruçá, 04-15.12.2006

  12. Direct Acyclic Graph (DAG) is a set of jobs where the input, output, or execution of one or more jobs depends on one or more other jobs A Collection is a group of jobs with no dependencies basically a collection of JDL’s A Parametric job is a job having one or more attributes in the JDL that vary their values according to parameters Using compound jobs it is possible to have one shot submission of a (possibly very large, up to thousands) group of jobs Submission time reduction Single call to WMProxy server Single Authentication and Authorization process Sharing of files between jobs Availability of both a single Job ID to manage the group as a whole and an ID for each single job in the group nodeA nodeB nodeC nodeE nodeD High Level Services: Workflows 1st EELA Grid School, Itacuruçá, 04-15.12.2006

  13. Questions ? www.glite.org 1st EELA Grid School, Itacuruçá, 04-15.12.2006

More Related