1 / 15

Globus Toolkit

Globus Toolkit. Massimo Sgaravatto INFN Padova. Introduction. Grid Services: LHC regional centres need distributed computing Analyze toolkits that integrate distributed computational resources: Globus Legion Globe …. Globus. Basic research on GRID problems Globus Toolkit

erik
Download Presentation

Globus Toolkit

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Globus Toolkit Massimo Sgaravatto INFN Padova

  2. Introduction • Grid Services: LHC regional centres need distributed computing • Analyze toolkits that integrate distributed computational resources: • Globus • Legion • Globe • …. Massimo Sgaravatto

  3. Globus • Basic research on GRID problems • Globus Toolkit • “Bag of services” for developing GRID tools and applications • Available for most Unix platforms • Limited functionalities for Win32 Massimo Sgaravatto

  4. Local Services Condor MPI TCP UDP LSF Easy NQE AIX Irix Solaris Globus Architecture Applications High-level Services and Tools GlobusView Testbed Status DUROC MPI MPI-IO CC++ Nimrod/G globusrun Core Services Nexus GRAM Metacomputing Directory Service Globus Security Interface Heartbeat Monitor Gloperf GASS Massimo Sgaravatto

  5. Core Services • Security (GSI) • Information Service (MDS) • Resource Management (GRAM) • Remote Data Access (GASS) • Communication (Nexus) • Network Performance Monitor (GLOPERF) • Fault detection (HBM) • Executable Management (GEM) (???) Massimo Sgaravatto

  6. Globus Security Infrastructure • Main focus: authentication • single authentication • not necessary to manage authentication on each GRID node • Mapping between GRID user-id and local user-id • Digital signature mechanisms Massimo Sgaravatto

  7. Metacomputing Directory Service • Static and dynamic information about GRID components (host architecture, memory, load, network bandwidth, latency. ...) • Possibility to define new attributes • Information updated by: • Globus services (i.e. GRAM Reporter) • Users/Applications • Other “providers” (i.e. SNMP) • Tools and API for discovering, publishing, and accessing information • Standard protocol and interface: LDAP • Single MDS server (located in USA) Massimo Sgaravatto

  8. Resource Management • Management of computational resources • GRAM (Globus Resource Allocation Manager) responsible for a set of “local” resources • Single interface to different resource managers (Condor, LSF, NQE, fork, ...) • Resource Specification Language (RSL) for defining: • resource requirements (machine type, memory, ...) • job definition (executable, parameters, ...) I.e.: & (ResourceManagementContact = “cond.pd.infn.it:/C=US/.../CN=cond.pd.infn.it- condor”) (count=100) (executable=myapp) Massimo Sgaravatto

  9. Globus Resource Manager Architecture Application Resource and Job specification (High level RSL expression) MDS RSL specialization Broker Resource discovery Create jobs (Ground RSL expressions) GRAM GRAM GRAM CONDOR LSF fork Site1 Site2 Site3 Massimo Sgaravatto

  10. Remote Data Access • GASS (Global Access to Secondary Storage) • Supported servers: GASS, HTTP, FTP • Uniform naming using URL: • I.e.: x-gass://host1.pd.infn.it:1234/filename • Possibility to access remote data sets (distributed file system) • Necessary to modify source code: • open()  globus_gass_open() • close()  globus_gass_close() • Caching • Possibility to move files from/to remote data storage systems Massimo Sgaravatto

  11. Other core services • Nexus • Unicast and multicast low level communication library • Supports different programming models (message passing, RPC, etc…) • HBM (Heartbeat Monitor) • Monitors the status of registered processes • GLOPERF (Globus Network Performance Measurement Tool) • Measures throughput and latency between node pairs Massimo Sgaravatto

  12. Evaluation of the Globus Toolkit • 5 sites Testbed (Bologna, CNAF, LNL, Padova, Roma1) • Use case: HLT CMS studies • MC Prod.  Complete HLT chain • Services • Resource management • fork()  Interface to different local resource managers (Condor, LSF) • Resources chosen by hand  Broker to implement a Global resource manager • Data Mover • to stage executable and input files • to retrieve output files Massimo Sgaravatto

  13. Use Case: CMS HLT studies Massimo Sgaravatto

  14. Status • Globus installed on 5 Linux PCs in 3 sites • GSI • Seems working • MDS • Lots of problems, maybe temporary? • GRAM, GASS, Gloperf, HBM • Work in progress Massimo Sgaravatto

  15. Other problems • Documentation not well organized • Collaboration with HPC Lab, University of Lecce is very useful ! Massimo Sgaravatto

More Related