1 / 18

Part Five: Globus Job Management

Part Five: Globus Job Management. Part Five: Globus Job Management. A: GRAM B: Globus Job Commands C: Laboratory: globusrun. A: GRAM. GRAM: What is it?. Given a job specification: Create an environment for a job Stage files to/from the environment Submit a job to a local scheduler

lixue
Download Presentation

Part Five: Globus Job Management

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Part Five:Globus Job Management

  2. Part Five: Globus Job Management • A: GRAM • B: Globus Job Commands • C: Laboratory: globusrun

  3. A: GRAM

  4. GRAM: What is it? • Given a job specification: • Create an environment for a job • Stage files to/from the environment • Submit a job to a local scheduler • Monitor a job • Send job state change notifications • Stream a job’s stdout/err during execution

  5. GRAM: Some Terminology • We speak loosely most of the time, but: • Globus Job Management Service • Starts up and monitors jobs • Stages data in and out • GRAM • Protocol to communicate with the job management service • We often say “GRAM” as a shorthand for either of these

  6. Local Resource Manager Process Process Process GRAM: How Does it Work? Head Node a.k.a “Gatekeeper” Compute Resource Gatekeeper (Authenticates & Authorizes) GRAM Client Results Job Manager (Submits job & Monitors job)

  7. GRAM: What is a “Local Resource Manager?” • It’s usually a batch system that allows you to run jobs across a cluster of computers • Examples: • Condor • PBS • LSF • Sun Grid Engine • Most systems allow you to access “fork” • It’s the default • It runs on the gatekeeper: a bad idea in general, but okay for testing

  8. GRAM: RSL • The client describes the job with the Resource Specification Language (RSL) & (executable = a.out) (directory = /home/nobody ) (arguments = arg1 "arg 2") • You don’t usually need to specify RSL directly, unless you have special needs. • http://www.globus.org/gram/rsl_spec1.html

  9. GRAM: Security • GRAM uses GSI for security • Submitting a job requires a full proxy • The remote system & your job will get a limited proxy • The job will run—you had a full proxy when you submitted • But your job cannot submit other jobs

  10. Making your job batch ready • Must be able to run in the background: no interactive input, windows, GUI, etc. • Can still use STDIN, STDOUT, and STDERR (the keyboard and the screen), but files are used for these instead of the actual devices • Organize data files • Must be able to be run multiple times, sometimes incomplete

  11. GRAM: Basic Usage • globus-job-run hostX /bin/hostname • This runs /bin/hostname on hostX • It expects /bin/hostname to already be there • globusrun -o -r hostX ‘&(executable=/bin/echo) (arguments=Hello Grid)’ • This is the RSL • We could specify lots of things here, but we didn’t • These just ran with the fork job manager, not an “interesting” batch system

  12. GRAM: Running on a Batch System • Append the batch system to the hostname: • globus-job-runhostX/jobmanager-condor/bin/hostname • You will do this for most real work • The batch system can handle many more jobs • Batch systems are reliable and track your jobs • Fork is not reliable, and your job may be lost

  13. B: Globus Job Commands

  14. Globus Job Commands • globus-job-run ‘contact-string’ command • globus-job-submit ‘contact-string’ command • globus-job-status ‘contact-string’ • globus-job-get-output ‘contact-string’ • globus-job-clean ‘contact-string’ • globusrun

  15. Lab 5: globusrun

  16. Lab 5: globusrun • In this lab, you’ll: • Set up your environment for job submission • Submit simple jobs with globus-job-run and globus-job-submit • Use globus & RSL • Stage data with globusrun & RSL

  17. Credits • NSF disclaimer • Portions of this presentation were adapted from the following sources: • Jaime Frey, Condor Group, UW-Madison

More Related