ECECS Lecture 18 Grid Computing

1. ECECS Lecture 18Grid Computing Citation: B.Ramamurthy/Suny-Buffalo

2. Globus Material The presentation is based on the two main publications on grid computing given below: • The Physiology of the Grid, An Open Services Architecture for Distributed Systems Integration, by Ian Foster, Carl Kesselman, Jeffrey Nick, and Steven Tuecke, 2002. • The Anatomy of the grid, Enabling Scalable Virtual Organization, Ian Foster, Carl Kesselman, Steven Tuecke, 2001. • URL:http://www.globus.org/research/papers.html

3. Grid Technology • Grid technologies and infrastructures support the sharing and coordinated use of diverse resources in dynamic, distributed “virtual organizations”. • Grid technologies are distinct from technology trends such as Internet, enterprise, distributed and peer-to-peer computing. But these technologies can benefit from growing into the “problem space” addressed by grid technologies.

4. Virtual Organization: Problem Space • An industrial consortium formed to develop a feasibility study for a next generation supersonic aircraft undertakes a highly accurate multidisciplinary simulation of the entire aircraft. • A crisis management teams responds to a chemical spill by using local weather and soil models to estimate the spread of the spill, planning and coordinating evacuation, notifying hospitals and so forth. • Thousands of physicists come together to design, create, operate and analyze products by pooling together computing, storage, networking resources to create a Data Grid.

5. Resource Sharing Requirements • Members should be trustful and trustworthy. • Sharing is conditional. • Should be secure. • Sharing should be able to change dynamically over time. • Need for discovery and registering of resources. • Can be peer to peer or client/server. • Same resource may be used in different ways. • All these point to well defined architecture and protocols.

6. Grid Definition • Architecture identifies the fundamental system components, specifies purpose and function of these components, and indicates how these components interact with each other. • Grid architecture is a protocol architecture, with protocols defining the basic mechanisms by which VO users and resources negotiate , establish, manage and exploit sharing relationships. • Grid architecture is also a services standards-based open architecture that facilitates extensibility, interoperability, portability and code sharing. • API and Toolkits are also being developed.

7. High throughput Collab. design Remote control Application Toolkit Layer Data- intensive Remote viz Information Resource mgmt . . . Grid Services Layer Security Data access Fault detection Grid Services Architecture High-energy physics data analysis Collaborative engineering On-line instrumentation Applications Regional climate studies Parameter studies Grid Fabric Layer Transport . . . Multicast Instrumentation Control interfaces QoS mechanisms

8. Architecture Internet GRID Application Application Collective Resource Transport Connectivity Internet Fabric Link

9. Fabric Layer • Fabric layer: Provides the resources to which shared access is mediated by Grid protocols. • Example: computational resources, storage systems, catalogs, network resources, and sensors. • Fabric components implement local, resource specific operations. • Richer fabric functionality enables more sophisticated sharing operations. • Sample resources: computational resources, storage resources, network resources, code repositories, catalogs.

10. Connectivity Layer • Communicating easily and securely. • Connectivity layer defines the core communication and authentication protocols required for grid-specific network functions. • This enables the exchange of data between fabric layer resources. • Support for this layer is drawn from TCP/IP’s IP, TCL and DNS layers. • Authentication solutions: single sign on, etc.

11. Resources Layer • Resource layer defines protocols, APIs, and SDKs for secure negotiations, initiation, monitoring control, accounting and payment of sharing operations on individual resources. • Two protocols information protocol and management protocol define this layer. • Information protocols are used to obtain the information about the structure and state of the resource, ex: configuration, current load and usage policy. • Management protocols are used to negotiate access to the shared resource, specifying for example qos, advanced reservation, etc.

12. Collective Layer • Coordinating multiple resources. • Contains protocols and services that capture interactions among a collection of resources. • It supports a variety of sharing behaviors without placing new requirements on the resources being shared. • Sample services: directory services, coallocation, brokering and scheduling services, data replication service, workload management services, collaboratory services.

13. Applications Layer • These are user applications that operate within VO environment. • Applications are constructed by calling upon services defined at any layer. • Each of the layers are well defined using protocols, provide access to useful services. • Well defined APIs also exist to work with these services. • A toolkit Globus implements all these layers and supports grid application development.

14. Globus Toolkit Services • Security (GSI) • PKI-based Security (Authentication) Service • Job submission and management (GRAM) • Uniform Job Submission • Information services (MDS) • LDAP-based Information Service • Remote file management (GASS) • Remote Storage Access Service • Remote Data Catalogue and Management Tools • Support by Globus 2.0 released in 2002

15. High-level services Part II

16. Sample of High-Level Services • Resource brokers and co-allocators • DUROC, Nimrod/G, Condor-G, GridbusBroker Communication & I/O libraries • MPICH-G, PAWS, RIO (MPI-IO), PPFS, MOL • Parallel languages • HPC++, CC++, Nimrod Parameter Specification • Collaborative environments • CAVERNsoft, ManyWorlds • Others • MetaNEOS, NetSolve, LSA, AutoPilot, WebFlow

17. The Nimrod-G Grid Resource Broker • A resource broker for managing, steering, and executing task farming (parameter sweep/SPMD model) applications on the Grid based on deadline and computational economy. • Based on users’ QoS requirements, our Broker dynamically leases services at runtime depending on their quality, cost, and availability. • Key Features • A single window to manage & control experiment • Persistent and Programmable Task Farming Engine • Resource Discovery • Resource Trading • Scheduling & Predications • Generic Dispatcher & Grid Agents • Transportation of data & results • Steering & data management • Accounting • Uses Globus – MDS, GRAM, GSI, GASS

18. Condor-G: Condor for the Grid • Condor is a high-throughput scheduler • Condor-G uses Globus Toolkit libraries for: • Security (GSI) • Managing remote jobs on Grid (GRAM) • File staging & remote I/O (GSI-FTP) • Grid job management interface & scheduling • Robust replacement for Globus Toolkit programs • Globus Toolkit focus is on libraries and services, not end user vertical solutions • Supports single or high-throughput apps on Grid • Personal job manager which can exploit Grid resources

19. Production Grids & Testbeds • Production deployments underway at: • NSF PACIs National Technology Grid • NASA Information Power Grid • DOE ASCI • European Grid • Research testbeds • EMERGE: Advance reservation & QoS • GUSTO: Globus Ubiquitous Supercomputing Testbed Organization • Particle Physics Data Grid • World-Wide Grid (WWG)

20. Production Grids & Testbeds NASA’s Information Power Grid The Alliance National Technology Grid GUSTO Testbed

21. WW Grid World Wide Grid (WWG) Australia North America GMonitor Melbourne+Monash U: VPAC, Physics ANL: SGI/Sun/SP2 NCSA: Cluster Wisc: PC/cluster NRC, Canada Many others Gridbus+Nimrod-G MEG Visualisation Solaris WS Internet @ SC 2002/Baltimore Europe Grid MarketDirectory ZIB: T3E/Onyx AEI: Onyx CNR: Cluster CUNI/CZ: Onyx Pozman: SGI/SP2 Vrije U: Cluster Cardiff: Sun E6500 Portsmouth: Linux PC Manchester: O3K Cambridge: SGI Many others Asia AIST, Japan: Solaris Cluster Osaka University: Cluster Doshia: Linux cluster Korea: Linux cluster

22. Example Applications Projects (via Nimrod-G or Gridbus) • Molecular Docking for Drug Discovery • Docking molecules from chemical databases with target protein • Neuro Science • Brain Activity Analysis • High Energy Physics • Belle Detector Data Analysis • Natural Language Engineering • Analyzing audio data (e.g., to identify emotional state of a person!)

23. Example Application Projects • Computed microtomography (ANL, ISI) • Real-time, collaborative analysis of data from X-Ray source (and electron microscope) • Hydrology (ISI, UMD, UT; also NCSA, Wisc.) • Interactive modeling and data analysis • Collaborative engineering (“tele-immersion”) • CAVERNsoft @ EVL • OVERFLOW (NASA) • Large CFD simulations for aerospace vehicles

24. Example Application Experiments • Distributed interactive simulation (CIT, ISI) • Record-setting SF-Express simulation • Cactus • Astrophysics simulation, viz, and steering • Including trans-Atlantic experiments • Particle Physics Data Grid • High Energy Physics distributed data analysis • Earth Systems Grid • Climate modeling data management

25. The Globus Advantage • Flexible Resource Specification Language which provides the necessary power to express the required constraints • Services for resource co-allocation, executable staging, remote data access and I/O streaming • Integration of these services into high-level tools • MPICH-G: grid-enabled MPI • globus-job-*: flexible remote execution commands • Nimrod-G Grid Resource broker • Gridbus: Grid Business Infrastructure • Condor-G: high-throughput broker • PBS, GRD: meta-schedulers

26. Resource Management • Resource Specification Language (RSL) is used to communicate requirements • The Globus Resource Allocation Manager (GRAM) API allows programs to be started on remote resources, despite local heterogeneity • A layered architecture allows application-specific resource brokers and co-allocators to be defined in terms of GRAM services

27. Broker Co-allocator Resource Management Architecture RSL specialization RSL Application Information Service Queries & Info Ground RSL Simple ground RSL Local resource managers GRAM GRAM GRAM LSF EASY-LL NQE

28. GRAM Components MDS client API calls to locate resources Client MDS: Grid Index Info Server Site boundary MDS client API calls to get resource info GRAM client API calls to request resource allocation and process creation. MDS: Grid Resource Info Server Query current status of resource GRAM client API state change callbacks Globus Security Infrastructure Local Resource Manager Allocate & create processes Request Job Manager Create Gatekeeper Process Parse Monitor & control Process RSL Library Process

29. A simple run • [raj@belle raj]$ globus-job-run belle.anu.edu.au /bin/date • Mon May 3 15:05:42 EST 2004

30. Resource Specification Language (RSL) • Common notation for exchange of information between components • Syntax similar to MDS/LDAP filters • RSL provides two types of information: • Resource requirements: Machine type, number of nodes, memory, etc. • Job configuration: Directory, executable, args, environment • API provided for manipulating RSL

31. RSL Syntax • Elementary form: parenthesis clauses • (attribute op value [ value … ] ) • Operators Supported: • <, <=, =, >=, > , != • Some supported attributes: • executable, arguments, environment, stdin, stdout, stderr, resourceManagerContact,resourceManagerName • Unknown attributes are passed through • May be handled by subsequent tools

32. Constraints: “&” • globusrun -o -r belle.anu.edu.au "&(executable=/bin/date)" • For example: & (count>=5) (count<=10) (max_time=240) (memory>=64) (executable=myprog) “Create 5-10 instances of myprog, each on a machine with at least 64 MB memory that is available to me for 4 hours”

33. Disjunction: “|” • For example: • & (executable=myprog) • ( | (&(count=5)(memory>=64)) • (&(count=10)(memory>=32))) • Create 5 instances of myprog on a machine that has at least 64MB of memory, or 10 instances on a machine with at least 32MB of memory

34. Multirequest: “+” • A multi-request allows us to specify multiple resource needs, for example + (& (count=5)(memory>=64) (executable=p1)) (&(network=atm) (executable=p2)) • Execute 5 instances of p1 on a machine with at least 64M of memory • Execute p2 on a machine with an ATM connection • Multirequests are central to co-allocation

35. Co-allocation • Simultaneous allocation of a resource set • Handled via optimistic co-allocation based on free nodes or queue prediction • In the future, advance reservations will also be supported • globusrun and globus-job-* will co-allocate specific multi-requests • Uses a Globus component called the Dynamically Updated Request Online Co-allocator (DUROC)

36. DUROC Functions • Submit a multi-request • Edit a pending request • Add new nodes, edit out failed nodes • Commit to configuration • Delay to last possible minute • Barrier synchronization • Initialize computation • Bootstrap library • Monitor and control collection

37. RM1 RM2 RM3 Job 1 Job 2 Job 3 RM4 Job 4 Job 5 DUROC Architecture Controlled Jobs Subjobstatus Controlling Application RSL multi-request Edit request Barrier

38. RSL Creation Using globus-job-run • globus-job-run can be used to generate RSL from command-line args: globus-job-run –dumprsl \ -: host1 -np N1 [-s] executable1 args1 \ -: host2 -np N2 [-s] executable2 args2 \ ... > rslfile • -np: number of processors • -s: stage file • argument options for all RSL keywords • -help: description of all options

39. Job Submission Interfaces • Globus Toolkit includes several command line programs for job submission • globus-job-run: Interactive jobs • globus-job-submit: Batch/offline jobs • globusrun: Flexible scripting infrastructure • Other High Level Interfaces • General purpose • Nimrod-G, Condor-G, PBS, GRD, etc • Application specific • ECCE’, Cactus, Web portals

40. globus-job-run • For running of interactive jobs • Additional functionality beyond rsh • Ex: Run 2 process job w/ executable staging globus-job-run -: host –np 2 –s myprog arg1 arg2 • Ex: Run 5 processes across 2 hosts globus-job-run \ -: host1 –np 2 –s myprog.linux arg1 \ -: host2 –np 3 –s myprog.aix arg2 • For list of arguments run: globus-job-run -help

41. globus-job-submit • For running of batch/offline jobs • globus-job-submit Submit job • Same interface as globus-job-run • Returns immediately • globus-job-status Check job status • globus-job-cancel Cancel job • globus-job-get-output Get job stdout/err • globus-job-clean Cleanup after job

42. globusrun • Flexible job submission for scripting • Uses an RSL string to specify job request • Contains an embedded globus-gass-server • Defines GASS URL prefix in RSL substitution variable: (stdout=$(GLOBUSRUN_GASS_URL)/stdout) • Supports both interactive and offline jobs • Complex to use • Must write RSL by hand • Must understand its esoteric features • Generally you should use globus-job-* commands instead

43. “Perform a parameter study involving 10,000 separate trials” Parameter study specific broker " . . ." “Create a shared virtual space with participants X, Y, and Z” Collaborative environment-specific resource broker " . . ." Resource Brokers “Run a distributed interactive simulation involving 100,000 entities” “Supercomputers providing 100 GFLOPS, 100 GB, < 100 msec latency” DIS-Specific Broker Information Service Supercomputer resource broker “80 nodes on Argonne SP, 256 nodes on CIT Exemplar 300 nodes on NCSA O2000” Simultaneous start co-allocator "Run SF-Express on 80 nodes” "Run SF-Express on 256 nodes” “Run SF-Express on 300 nodes” Argonne Resource Manager CIT Resource Manager NCSA Resource Manager

44. Brokering via Lowering • Resource location by refining a RSL expression (RSL lowering): (MFLOPS=1000)Þ (& (arch=sp2)(count=200))Þ (+ (& (arch=sp2) (count=120) (resourceManagerContact=anlsp2)) (& (arch=sp2) (count=80) (resourceManagerContact=uhsp2)))

45. Remote I/O and Staging • Tell GRAM to pull executable from remote location • Access files from a remote location • stdin/stdout/stderr from a remote location

46. What is GASS? (a) GASS file access API • Replace open/close with globus_gass_open/close; read/write calls can then proceed directly (b) RSL extensions • URLs used to name executables, stdout, stderr (c) Remote cache management utility (d) Low-level APIs for specialized behaviors

47. GASS Architecture &(executable=https://…) main( ) { fd = globus_gass_open(…) … read(fd,…) … globus_gass_close(fd) } (b) RSL extensions GRAM GASS Server HTTP Server (a) GASS file access API FTP Server Cache (c) Remote cache management (d) Low-level APIs for customizing cache & GASS server % globus-gass-cache

48. GASS File Naming • URL encoding of resource names https://quad.mcs.anl.gov:9991/~bester/myjob protocolserver address file name • Other examples https://pitcairn.mcs.anl.gov/tmp/input_dataset.1 https://pitcairn.mcs.anl.gov:2222/./output_data http://www.globus.org/~bester/input_dataset.2 • Supports http & https • Support ftp & gsiftp.

49. GASS RSL Extensions • executable, stdin, stdout, stderr can be local files or URLs • executable and stdin loaded into local cache before job begins (on front-end node) • stdout, stderr handled via GASS append mode • Cache cleaned after job completes

50. GASS/RSL Example &(executable=https://quad:1234/~/myexe) (stdin=https://quad:1234/~/myin) (stdout=/home/bester/output) (stderr=https://quad:1234/dev/stdout)