1 / 13

A proposal for standardizing the working environment for a LCG/EGEE job

A proposal for standardizing the working environment for a LCG/EGEE job. David Bouvet - Grid Computing team - CCIN2P3 HEPIX Karlsruhe 13/05/2005. Motivation. Problem raised some months ago by Jeff Templon:

benita
Download Presentation

A proposal for standardizing the working environment for a LCG/EGEE job

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. A proposal for standardizing the working environment for a LCG/EGEE job David Bouvet - Grid Computing team - CCIN2P3 HEPIX Karlsruhe 13/05/2005

  2. Motivation • Problem raised some months ago by Jeff Templon: • D0 jobs encountered problems at Lyon due to different use of environment variables to address scratch/temp disk space • Standard is defined for: • Environment Variables « IEEE Std 1003.1, 2004 POSIX Part 1: Base definitions, Amendment 8 » http://www.opengroup.org/onlinepubs/009695399/basedefs/xbd_chap08.html • among which: HOME, PATH, PWD, SHELL, TMPDIR, USER • Batch Environment Services « IEEE Std 1003.1, 2004 POSIX Part 2: Shell and Utilities, Amendment 1 » http://www.opengroup.org/onlinepubs/009695399/utilities/xcu_chap03.html • PBS_ENVIRONMENT, PBS_JOBID, PBS_JOBNAME, PBS_QUEUE • PBS_O_HOME, PBS_O_HOST, PBS_O_LOGNAME, PBS_O_PATH, PBS_O_QUEUE, PBS_O_SHELL, PBS_O_WORKDIR  these variables are not directly used by the jobs • There is no standard definition of environment variables for grid batch jobs  Proposal for LCG/EGEE sites of a common definition of minimal set of environment variables for grid batch jobs David Bouvet – HEPIX Karlsruhe 13/05/20052

  3. Current status through several batch used on the grid • Environment variables for grid batch job have been checked on several LCG/EGEE sites (among which all the LCG T1s) Conditions of test: ATLAS VO, short queue David Bouvet – HEPIX Karlsruhe 13/05/20053

  4. Current status: POSIX variables  : defined: not defined on some sites  not all these variables are defined on the various batch systems David Bouvet – HEPIX Karlsruhe 13/05/20054

  5. Current status (cont.)  : defined : not defined on some sites  even for Globus, not all the sites define the same set of environment variables. David Bouvet – HEPIX Karlsruhe 13/05/20055

  6. Current status: LCG environment variables (middleware related) (list from the LCG Users Guide) David Bouvet – HEPIX Karlsruhe 13/05/20056

  7. possible uniformization to POSIX name: TMPDIR ? Current status: LCG environment variables (job related) (list from the LCG Users Guide) David Bouvet – HEPIX Karlsruhe 13/05/20057

  8. Current status: gLite environment variables • gLite environment variables on WN (in config. files and scripts) from gLite installation guide: • GLITE_LOCATION /opt/glite • GLITE_LOCATION_VAR /var/glite • GLITE_LOCATION_LOG /var/log/glite • GLITE_LOCATION_TMP /tmp/glite • GLITE_LOCATION_TMP  another tmp directory to clean! David Bouvet – HEPIX Karlsruhe 13/05/20058

  9. Proposal for standardization David Bouvet – HEPIX Karlsruhe 13/05/20059

  10. Proposal for standardization (cont.) • Use of POSIX variable when existing • TMPDIR: POSIX variable which can replace LCG_TMP, EDG_TMP, GLITE_LOCATION_TMP • HOME: MPI jobs need a home directory David Bouvet – HEPIX Karlsruhe 13/05/200510

  11. Proposal for standardization (cont.) • Minimal set of environment variable (not related to middleware).The naming convention must be independant of grid middleware name for grid job portability • GRID_WORKDIR: work directory specific to the job (unix right 700) • e.g.:/scratch/atlas0011293.ccwl0092 • GRID_SITENAME: to know on which site the job run (same as siteName in the Information System) • e.g.:IN2P3-CC • GRID_HOSTNAME: could be useful to know the WN hostname for problem tracking (and parallel jobs?) • e.g.:ccwl0006.in2p3.fr • GRID_CEID: CE and queue names on which the job run (same as GlueCEUniqueID in Information System) • e.g.:heplnx201.pp.rl.ac.uk:2119/jobmanager-torque-short • GRID_LOCAL_JOBID: useful for problem tracking (and parallel jobs?) • e.g.: lcg0509104420-07243 • GRID_GLOBAL_JOBID: same as EDG_WL_JOBID for LCG • e.g.:https://lxn1188.cern.ch:9000/HPMN2WVHurMlji-Fnqba0A • GRID_USERID: DN of user’s certificate (already exists on some sites) • e.g.: /O=GRID-FR/C=FR/O=CNRS/OU=CC-LYON/CN=David Bouvet/Email=dbouvet@in2p3.fr David Bouvet – HEPIX Karlsruhe 13/05/200511

  12. Proposal for standardization (cont.) • When agreed on a set of variables and a naming convention, this standard should be implemented on all LCG/EGEE CEs. • Based on today’s discussion, a document will be distributed to site administrators and applications. • A possible deadline for discussion and beginning of deployment: end of June David Bouvet – HEPIX Karlsruhe 13/05/200512

  13. Proposal for standardization (discussion) David Bouvet – HEPIX Karlsruhe 13/05/200513

More Related