CLRC and the European DataGrid Middleware

1. Data GRID UI JDL The Producer Consumer Model Client side Hidden components Data BaseProducer Servlet Data Base Producer Circular Buffer Producer Circular BufferProducerServlet Registry Interface Sensor Registry Servlet Consumer Servlet Consumer Schema Interface Application Archiver Servlet Schema Servlet Archiver Application Invocation Information CLRC and the European DataGrid Middleware • Middleware • Middleware provides the interface between the applications and the physical fabric • Information and Monitoring Services • The current information service is built on the hierarchical database OpenLDAP (lightweight directory access protocol) as provided by Globus’s Metacomputing Directory Service • The information published relates to the resources available at a site • This information is used by the resource broker for resource discovery and job scheduling Applications HEP Apps EO Apps Bio Apps • CLRC/PPARC and the DataGrid Project • CLRC staff, and PPARC staff based at RAL, are active in a number of DataGrid work packages • Particle Physics Applications • Information and Monitoring Services • Mass Storage Management • Testbed Integration • Network Services Middleware Workload Management • Architectural Task Force • Security Co-ordination Group • UK DataGrid Co-ordination • Testbed Certificate Authorities group Information and Monitoring Services Fabric Management Data Management Mass Storage Management Network Services computing element Globus Middleware storage element network information between this and other sites Physical Fabric Red indicates CLRC leadership/management of the work package/group Computing Fabric Mass Storage Networking status file statistics supported protocols storage elements that are close (not necessarily at the same site) CLRC/PPARC participation Job Submission • Relational Grid Monitoring Architecture • R-GMA is being developed within the Information and Monitoring Services work package as a new information and monitoring system infrastructure • R-GMA consists of Consumers, Producers and a directory service (the Registry) • Producers register with the registry and describe the type and structure of information they want to make available to the Grid • Consumers query the registry to find out what type of information is available and locate producers that provide such information • The consumer can contact the producer directly to obtain the relevant data • Virtual Organisations • A virtual organisation is a dynamic collection of individuals, institutions, and resources with a set of sharing rules that clearly and carefully define what is shared, who is allowed to share, and the conditions under which sharing occurs • A site may have resources that are allocated to one or more virtual organisation Replica Catalogue User Computing Element - B Virtual Organisation 1 (Experiment 1) Virtual Organisation 2 (Experiment 2) Virtual Organisation 3 (Experiment 3) Information and Monitoring Services Resource Broker Storage Element - B Network Monitoring - B • Network Services • One machine per testbed site is used for network monitoring • Both active and passive measurements are made • The information is published via the information service and will be used by the resource broker for job scheduling • Measurements include: packet loss one-way delay round trip time (RTT) jitter RTT variation • Storage Element • The Storage Element is an integration project between grid communication protocols and a diversity of mass storage systems • The Storage Element facilitates centralised brokering of resources via location independent interfaces • The Storage Element is divided into three layers Computing Element - A Storage Element - A Network Monitoring - A • Architecture Task Force • The ATF was created to define an architecture for the DataGrid • It is composed of one person from each of the middleware work packages and invited experts • The architecture documents define a set of services and their interdependencies • It has been an important contribution to the success of the first testbed • DataGrid Security • Grid certificates enable single sign on to the grid • Grid certificates issued by certificate authorities • Each site has a list of trusted authorities • Developing systems for grid wide authentication and authorisation Client Applications Grid FTP NFS RFIO Network API Middle Layer MSM Interface Example output from the RIPE NCC TTM box Storage Element