1 / 47

Introduction to Globus

Introduction to Globus. PHENIX experiment uses Grid to transfer 270 TB of data to Japan.

delora
Download Presentation

Introduction to Globus

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Introduction to Globus

  2. PHENIX experiment uses Grid to transfer 270 TB of data to Japan • During the polarized proton-proton run that ended in June at the Relativistic Heavy Ion Collider (RHIC) at Brookhaven, Grid tools were used by the PHENIX experiment to send recently acquired data to a regional computing centre for the experiment in Japan. Brookhaven National Laboratory, on Long Island, New York, is home to the RHIC/ATLAS Computing Facility (RCF/ACF), which is the main computing centre for experiments at RHIC and a Tier-1 computing centre for ATLAS. The PHENIX regional computing centre in Japan (CCJ) is at the RIKEN research centre on its Wako campus close to Tokyo.

  3. Daily rates of data transferred from the PHENIX experiment to the CCJ computing centre in Japan (blue), and the integrated data volume (red). Overall, 270 TB of data were transferred.

  4. Introduction • The Globus Project provides middleware services for grid environments. • There are four main components of Globus: • The Grid Security Infrastructure (GSI) provides • Authentication and authorization services using public key certificates and Kerberos authentication. • The Globus Resource Management architecture provides: • a language for specifying application requirements • mechanisms for immediate and advance reservations of one or more computational components. • interfaces for submitting jobs to remote machines.

  5. Introduction (cont.) • The Globus Information Management architecture provides: • a distributed scheme for publishing and retrieving information about resources in the wide area environment. • A distributed collection of information servers is accessed by higher-level services that perform resource discovery, configuration and scheduling. • The Globus Data Management architecture provides two fundamental components: • A universal data transfer protocol for grid computing environments called GridFTP • Replica Management infrastructure for managing multiple copies of shared data sets.

  6. The Globus Approach • The Globus toolkit provides a range of basic Grid services • Security, information, fault detection, communication, resource management, ... • These services are simple and orthogonal • Can be used independently, mix and match • Programming model independent • For each there are well-defined APIs • Standards are used extensively • E.g., LDAP, GSS-API, X.509, ...

  7. Grid Security Infrastructure • Single-sign on, run anywhere [if authorized] • Standards based (GSS, SSL, X.509) • GSS-API Interface • Identity/credential mapping at each resource • Limited delegation of rights • Integrated into wide variety of tools • Globus Resource Management • Secure shell, FTP, • Storage Resource Broker

  8. Grid Security Infrastructure (cont.) • Based on public key technology • Standard X.509 certificate, same as certificates used for the Web • Each user has: • a Grid user id (called a Subject Name) • /C=US/O=Globus/O=University of Southern California/OU=Information Sciences Institute/CN=Ann Chervenak • a private key (like a password) • a certificate signed by a Certificate Authority (CA) • A “gridmap” file at each site specifiesgrid-id to local-id mapping

  9. Authentication Model • Authentication is done on a “user” basis • Single authentication step allows access to all grid resources • No communication of plaintext passwords • Most sites will use conventional account mechanisms • You must have an account on a resource to use that resource • Sites may use “generic” Grid accounts • Not common, but Globus can deal with it

  10. Certificate Based Authentication • User has a certificate, signed by a trusted “certificate authority” (CA) • Certificate contains user’s name and public key • Globus project operates a CA • User’s private key is used to encode a challenge string • Public key is used to decode the challenge • If you can decode it, you know the user • Treat your private key carefully!! • Private key is stored in encrypted form

  11. User Proxies • Minimize exposure of user’s private key • A temporary credential for use by our computations • We call this a user proxy certificate • Allows process to act on behalf of user • User-signed user proxy certificate stored in local file • Proxy’s private key is not encrypted • Rely on file system security, proxy certificate file must be readable only by the owner

  12. Delegation • Remote creation of a user proxy • Allows remote process to act on behalf of the user • Avoids sending passwords or private keys across the network

  13. Single sign-onvia “grid-id” User User Proxy Site 1 Process Process GRAM GRAM GSI GSI Process Process Ticket Process Process Public Key Kerberos CREDENTIAL Assignment of credentials to “user proxies” Globus Credential Mutual user-resource authentication Site 2 Mapping to local ids Authenticated interprocess communication GSSAPI: multiple low-level mechanisms Certificate

  14. Resource Management • Globus Resource Allocation Manager (GRAM) • Uniform interface to resource management • Globus Arch. for Reservation and Allocation • Co-allocation of compute resources • Immediate and advance reservation of network and computers in prototype form • Fault detection service • Network measurement tools • Code management and distribution infrastructure

  15. Resource Management • Resource Specification Language (RSL) is used to communicate requirements • The Globus Resource Allocation Manager (GRAM) API allows programs to be started on remote resources, despite local heterogeneity • A layered architecture allows application-specific resource brokers and co-allocators to be defined in terms of GRAM services

  16. Broker Co-allocator Resource Management Architecture RSL specialization RSL Application Information Service Queries & Info Ground RSL Simple ground RSL Local resource managers GRAM GRAM GRAM LSF EASY-LL NQE

  17. Resource Specification Language • Common notation for exchange of information between components • RSL provides two types of information: • Resource requirements: Machine type, number of nodes, memory, etc. • Job configuration: Directory, executable, args, environment • API provided for manipulating RSL

  18. RSL Syntax • Elementary form: parenthesis clauses • (attribute op value [ value … ] ) • Operators Supported: • <, <=, =, >=, > , != • Some supported attributes: • executable, arguments, environment, stdin, stdout, stderr, resourceManagerContact,resourceManagerName • Unknown attributes are passed through • May be handled by subsequent tools

  19. Constraints: “&” • For example: & (count>=5) (count<=10) (max_time=240) (memory>=64) (executable=myprog) • “Create 5-10 instances of myprog, each on a machine with at least 64 MB memory that is available to me for 4 hours”

  20. Multirequest: “+” • A multirequest allows us to specify multiple resource needs, for example + (& (count=5)(memory>=64) (executable=p1)) (&(network=atm) (executable=p2)) • Execute 5 instances of p1 on a machine with at least 64M of memory • Execute p2 on a machine with an ATM connection • Multirequests are central to co-allocation

  21. Co-allocation • Simultaneous allocation of a resource set • Handled via optimistic co-allocation based on free nodes or queue prediction • In the future, advance reservations will also be supported

  22. Different resource managers Different counts Different executables A Co-allocation Multirequest +( & (resourceManagerContact= “flash.isi.edu:754:/C=US/…/CN=flash.isi.edu-fork”) (count=1) (label="subjob A") (executable= my_app1) ) ( & (resourceManagerContact= “sp139.sdsc.edu:8711:/C=US/…/CN=sp097.sdsc.edu-lsf") (count=2) (label="subjob B") (executable=my_app2) )

  23. Job Submission Interfaces • Globus Toolkit includes several command line programs for job submission • globus-job-run: Interactive jobs • globus-job-submit: Batch/offline jobs • globusrun: Flexible scripting infrastructure • Others are building better interfaces • General purpose • Condor-G, PBS, GRD, Hotpage, etc • Application specific • ECCE’, Cactus, Web portals

  24. Grid Information Services • Publish and retrieve information about system elements • Used for discovery, configuration, scheduling • Distributed collection of information servers and index nodes • LDAP V3 as wire protocol and API

  25. Examples of Useful Information • Characteristics of a compute resource • IP address, software available, system administrator, networks connected to, OS version, load • Characteristics of a network • Bandwidth and latency, protocols, logical topology • Characteristics of the Globus infrastructure • Hosts, resource managers

  26. Grid Information Service • Provide access to static and dynamic information regarding system components • A basis for configuration and adaptation in heterogeneous, dynamic environments • Requirements and characteristics • Uniform, flexible access to information • Scalable, efficient access to dynamic data • Access to multiple information sources • Decentralized maintenance

  27. The Globus ToolkitMetacomputing Directory Service • Store information in a distributed directory • Directory stored in collection of LDAP servers • Directory can be updated by • Information providers and tools • Applications (i.e., users) • Backend tools which generate info on demand • Information dynamically available to • Tools • Applications

  28. Directory Service Functions • White Pages • Look up the IP number, amount of memory, etc., associated with a particular machine • Yellow Pages • Find all the computers of a particular class or with a particular property • Temporary inconsistencies are often considered okay • In a distributed system, you often do not know the state of a resource until you actually use it • Information is often used as “hints” • Information itself can contain ttl, etc.

  29. MDS Approach Application • Based on LDAP • Lightweight Directory Access Protocol v3 (LDAPv3) • Standard data model • Standard query protocol • Globus specific schema • Host-centric representation • Globus specific tools • GRIS, GIIS • Data discovery, publication,… Middleware LDAP API … GIIS GRIS … SNMP NWS NIS LDAP

  30. Grid Resource Information Service • Server which runs on each resource • Given the resource DNS name, you can find the GRIS server (well known port = 2135) • Provides resource specific information • Much of this information may be dynamic • Load, process information, storage information, etc. • GRIS gathers this information on demand • “White pages” lookup of resource information • Ex: How much memory does machine have? • “Yellow pages” lookup of resource options • Ex: Which queues on machine allow large jobs?

  31. Grid Index Information Service • GIIS describes a class of servers • Gathers information from multiple GRIS servers • Each GIIS is optimized for particular queries • Ex1: Which Alliance machines are >16 process SGIs? • Ex2: Which Alliance storage servers have >100Mbps bandwidth to host X? • Akin to web search engines • Organization GIIS • The Globus Toolkit ships with one GIIS • Caches GRIS info with long update frequency • Useful for queries across an organization that rely on relatively static information

  32. Referral Service • Links together multiple GRIS and/or GIIS servers into a single LDAP namespace • Referral servers contain no actual content

  33. Data Grid Services • Access to remote data • Uniform access to diverse, remote storage management systems • Cache management • Transport services • Standards based (GSI, FTP protocol) • Client API, Extensible server, support for third party transfer • Replica Management

  34. Data Intensive Issues Include … • High-speed, reliable access to remote data • Automated discovery of “best” copy of data • Manage replication to improve performance • Co-schedule compute, storage, network • Enforce access control on data

  35. The Globus Data Grid Two major components: 1. Data Transport and Access • Common protocol • Secure, efficient, flexible, extensible data movement • Family of tools supporting this protocol 2. Replica Management Architecture • Simple scheme for managing: • multiple copies of files • collections of files

  36. Motivation for a Common Data Access Protocol • Existing distributed data storage systems • DPSS, HPSS: focus on high-performance access, utilize parallel data transfer, striping • DFS: focus on high-volume usage, dataset replication, local caching • SRB: connects heterogeneous data collections, uniform client interface, metadata queries • Problems • Incompatible protocols • Each require custom client • Partitions available data sets and storage devices • Each protocol has subset of desired functionality

  37. A Common, Secure, EfficientData Access Protocol • Common, extensible transfer protocol • Decouple low-level data transfer mechanisms from the storage service • Advantages: • New, specialized storage systems are automatically compatible with existing systems • Existing systems have richer data transfer functionality • Interface to many storage systems • HPSS, DPSS, file systems • Plan for SRB integration

  38. Replica Management • Maintain a mapping between logical names for files and collections and one or more physical locations • Important for many applications • Example: CERN HLT data • Multiple petabytes of data per year • Copy of everything at CERN (Tier 0) • Subsets at national centers (Tier 1) • Smaller regional centers (Tier 2) • Individual researchers will have copies

  39. Globus Approach to Replica Management • Identify replica cataloging and reliable replication as two fundamental services • Layer on other Grid services: GSI, transport, information service • Use LDAP as catalog format and protocol, for consistency • Use as a building block for other tools • Advantage • These services can be used in a wide variety of situations

  40. Replica Manager Components • Replica catalog definition • LDAP object classes for representing logical-to-physical mappings in an LDAP catalog • Low-level replica catalog API • globus_replica_catalog library • Manipulates replica catalog: add, delete, etc. • High-level reliable replication API • globus_replica_manager library • Combines calls to file transfer operations and calls to low-level API functions: create, destroy, etc.

  41. Replica Catalog Structure: A Climate Modeling Example Replica Catalog Logical Collection C02 measurements 1998 Logical Collection C02 measurements 1999 Filename: Jan 1998 Filename: Feb 1998 … Logical File Parent Location jupiter.isi.edu Location sprite.llnl.gov Filename: Mar 1998 Filename: Jun 1998 Filename: Oct 1998 Protocol: gsiftp UrlConstructor: gsiftp://jupiter.isi.edu/ nfs/v6/climate Filename: Jan 1998 … Filename: Dec 1998 Protocol: ftp UrlConstructor: ftp://sprite.llnl.gov/ pub/pcmdi Logical File Jan 1998 Logical File Feb 1998 Size: 1468762

  42. Replica Catalog Servicesas Building Blocks: Examples • Combine with information service to build replica selection services • E.g. “find best replica” using performance info from NWS and MDS • Use of LDAP as common protocol for info and replica services makes this easier • Combine with application managers to build data distribution services • E.g., build new replicas in response to frequent accesses

  43. Relationship to Metadata Catalogs • Metadata services describe data contents • Have defined a simple set of object classes • Must support a variety of metadata catalogs • MCAT being one important example • Others include LDAP catalogs, HDF • Community metadata catalogs • Agree on set of attributes • Produce names needed by replica catalog: • Logical collection name • Logical file name

  44. A Model Architecture for Data Grids Attribute Specification Replica Catalog Metadata Catalog Application Multiple Locations NWS Logical Collection and Logical File Name Selected Replica Replica Selection MDS gsiftp commands Performance Information and Predictions Disk Cache TapeLibrary Disk Array Disk Cache Replica Location 1 Replica Location 2 Replica Location 3

  45. Fault Detection: Globus Heartbeat Monitor • Detect and report “failure” of component of a computation • Limited by ability to distinguish between network partition and system failure • Optionally used within Globus Toolkit to monitor status of system processes • Can also be used to construct special fault monitors for applications • Example: Netsolve

  46. Fault Detection (cont.) • Goal: make low-level operations reliable • No libraries for checkpoint and restart • Can’t “checkpoint” a socket • Only application knows how to checkpoint and restart • Likewise, storage system must do logging

  47. Application Data Collector HBM Data Collector Monitored Process Monitored Process HBM Monitor HBM Monitor Monitored Process Monitored Process Monitored Process Monitored Process Heartbeat Monitor Application Level Fault Handler ! System Monitoring Tools Process and Host Heartbeat Process and Host Heartbeat Host 2 Host 1 Process Status Inquiry Process Status Inquiry Register/ Unregister Register/ Unregister

More Related