NGS UEE Uniform Execution Environment

1. NGS UEEUniform Execution Environment Jonathan ChurchillSTFC RAL 6 May 09

2. What is it ? • A wrapper script to run an application…. • in a standard location … • /usr/ngs • with a standard ‘API’. #!/bin/bash # # Run script for PFT2 # # Jonathan Churchill # Version Date Description #-----------+---------------+-------------------+ # 1.0 | 5-Sep-08 |Initial release | #-----------+---------------+-------------------+ # Load the required module. . /etc/profile.d/modules.sh module load pft3dr/32/2.0 # and run the program /apps/pft3dr/2.0/bin/.exe/pft2 $* exit $?

3. How’s that clever ? • Under NGS1 there was NGSMODULES: • Under UEE we instead have # globusrun -b -r ngs.rl.ac.uk/jobmanager-lsf \ '&(environment (NGSMODULES pft3dr/32/2.0)) (executable=/usr/local/bin/pft2)(jobType=single) (stdout=pft2.so)(stderr=pft2.e)' # globusrun -b -r ngs.rl.ac.uk/jobmanager-lsf \ '&(executable=/usr/ngs/PFT2) \ (jobType=single)(stdout=pft2.so)(stderr=pft2.e'

4. How’s that clever (2)? • Benefits for users: • Same “executable” at all supporting sites. • Commandline same/similar to launching on their desktop. • Only need the site + UEE name to run a globus job. • Common app launch page for an application on the NGS web site regardless of installer site (see later) • Benefits for Sites • Environment modules installation not required. • jobmanager NGSMODULES hack not required. • I get to write your app launch documentaton ;-)

5. What about MPI ? • UEE Script wraps the MPI linked executable. • mpirun added (as normal) by jobmanager #!/bin/bash # # Run script for DLPOLY2 2.1.6 # # Load the required module.. # Includes LD_LIBRARY_PATH entries for libmpi.so etc . /etc/profile.d/modules.sh module load dl_poly/2.16 # and run the program DLPOLY.X $* exit $?

6. What about MPI ? • So a normal globus mpi job submission : • Gets translated by the job manager to :(Apologies for LSF centric batch script ! And missing globus detail) # globusrun -b -r ngs.rl.ac.uk/jobmanager-lsf '&(executable=/usr/ngs/DLPOLY2_2_1_6) (jobType=mpi)(count=8) (stdout=out.o)(stderr=out.e)' #!/bin/bash#BSUB –n 8 #BSUB –o out.o #BSUB –e out.e mpirun /usr/ngs/DLPOLY2_2_1_6

7. API and SW Versions • All sites must follow the same “API” for a UEE. • Lead site for an app sets the API. Captures it:https://www.ngs.ac.uk/ops/privops/ngs2_config/UEEdocs.html • UEE Name must follow the format:APPNAME_1_2_3 • No ‘_’ in APPNAME • APPNAME must all by uppercase (for BDII see later) • First ‘_’ Denotes start of primary version name/number/dd,ww,mm combination • Version numbers don’t have to be digits or have major and minor release numbers. • Format assumed by consumers • Eg Applications list table at : http://www.ngs.ac.uk/apps • Sym link from /usr/ngs/APPNAME to /usr/ngs/APPNAME_1_2_3 • Users who don’t care get a bigger selection of sites to run at using /usr/ngs/APPNAME • Users who care can picked the specific version they need using /usr/ngs/APPNAME_1-2-3 • Indicates the ‘default’ not the latest.

8. UEE API • Note: Eg Emboss and BLAST TOOLBOX use 1 UEE script for different binaries. Stops generating 100’s of UEEs

9. Effects of other formats Other sites have _7_1_12 Digit not letter Has no underscore Mix and match digit and letter in same version field : 8a

10. One place to publish them all • BDII • GlueHostApplicationSoftwareRunTimeEnvironment • Values are UEE script names • vdt/1.10.1/lcg/var/gip/plugin/ngs-uee-gip-plugin

11. And one place to use them. • Resource Broker JDL. • Users also don’t need local home dir names Type = "Job"; JobType = "Normal"; Executable = "/usr/ngs/BLAST-NCBI"; Arguments = "blastall -p blastn -d est_hum -i aadnafra.fsa -a 1"; StdOutput = "blast_ncbi.out"; StdError = "blast_ncbi.err"; InputSandbox = {"aadnafra.fsa"}; OutputSandbox = {"blast_ncbi.out","blast_ncbi.err"}; Requirements = Member("BLAST-NCBI", other.GlueHostApplicationSoftwareRunTimeEnvironment); ShallowRetryCount = -1;

12. Where can I run this job ? # glite-job-list-match blast1.jdl Selected Virtual Organisation name (from UI conf file): dteam Connecting to host ngsrb01.ngs.rl.ac.uk, port 7772 *********************************************************** COMPUTING ELEMENT IDs LIST The following CE(s) matching your job requirements have been found: *CEId* ngs.oerc.ox.ac.uk:2119/jobmanager-pbs-workq ngs.rl.ac.uk:2119/jobmanager-lsf-ngs svr021.gla.scotgrid.ac.uk:2119/jobmanager-lcgpbs-q1d svr021.gla.scotgrid.ac.uk:2119/jobmanager-lcgpbs-q2d svr021.gla.scotgrid.ac.uk:2119/jobmanager-lcgpbs-q30m svr021.gla.scotgrid.ac.uk:2119/jobmanager-lcgpbs-q3d svr021.gla.scotgrid.ac.uk:2119/jobmanager-lcgpbs-q6h svr026.gla.scotgrid.ac.uk:2119/jobmanager-lcgpbs-q1d svr026.gla.scotgrid.ac.uk:2119/jobmanager-lcgpbs-q2d svr026.gla.scotgrid.ac.uk:2119/jobmanager-lcgpbs-q30m svr026.gla.scotgrid.ac.uk:2119/jobmanager-lcgpbs-q3d svr026.gla.scotgrid.ac.uk:2119/jobmanager-lcgpbs-q6h ngs.leeds.ac.uk:2119/jobmanager-pbs-mpi ********************************************* • Cool ! Lots ofplaces to run ? • Yes but how do Glasgow supportgridPP and NGShere ?

13. GridPP/NGS sw Publishing

14. VO-NGS.AC.UK • GridPP prefix with VO-<voname> to APPNAME • Eg: Brunel publish VO-biomed-GATE-4_0_0 and VO-biomed-GATE-3_1_2 • Same app as running at RAL (GATE_4_0_0, GATE_3_1_2) • Which one do you use? How do you request it? • We need to republish as:VO-NGS.AC.UK-APPNAME_1_2_3 • But publication transform (in ngs-uee-gip-plugin). • Not by UEE name change • Loose approximation to user familiar command names.

15. Documention. • Application Docs: • http://www.ngs.ac.uk/sites/ral/applications • Linked from the http://ngs.ac.uk/apps table • Command examples written to work at all NGS sites: • Highlights the jobmanager string to change • NGS3 +/- WMS work packages: • Opportunity for site agnostic application docs set • RB/WMS : • No home directory specifics • No Job manager specifics

16. References • Original UEE Definition • http://www.grid-support.ac.uk/files/UniformExecutionEnvironment.pdf • UEE API • https://www.ngs.ac.uk/ops/privops/ngs2_config/UEEdocs.html • How to install an App on the NGS • https://www.ngs.ac.uk/ops/privops/ngs2_config/ngs2-apps-install-flow.html • NGS Apps tables • http://www.ngs.ac.uk/apps