GridSAM and NGS

1. GridSAM and NGS MatteoTurilli

2. Outline • NGS deployment • Service provision: centralised Vs decentralised • Usage • Tests and examples • GridSAM NGS Users • Future work

3. GridSAM Deployment • Xen VM: • Scientific Linux 4.7 on LVM partition, java 1.6.0_13 • 1GB ram, 1 core • gridsam.oerc.ox.ac.uk • GridSAM version 2.1.4 • Globus connector submitting to ngs.rl.ac.uk • 818 jobs submitted since 1 Apr 2009 • 1 failure due to stress testing (Java heap space) • gridsam-test.oerc.ox.ac.uk • Testing instance, GridSAM version 2.1.6 • Globus connector submitting to ngs.oerc.ox.ac.uk • Used to validate installation How-to

4. Service Provision • Centralised:Belfast installed multiple instances of GridSAM 1.4 on a single container. One instance for every core node. Highly unstable, difficult to scale up due to resource constrains • distributed:every HEI may have its dedicated GridSAM instance following the installation how-to and the support provided by NGS. Single instance for single container proved stable. VM based instances increase scalability • Another submission tool:expand the offer without limiting constraints • How-to documentation: NGS3 support model for HEI • Easy deployment: VM + support

5. Usage • Clients • Command line OMII client for Unix/Windows • Java API • File staging • Stage in and stage out • NFS home or scratch directories

6. Tests and Examples Submission $ gridsam-submit \ -s http://gridsam-test.oerc.ox.ac.uk:18443/gridsam/services/gridsam \ -myproxy -myproxyuser <user_name> \ -myproxyhost myproxy.grid-support.ac.uk \ -j javatest.jsdl MyProxy password: urn:gridsam:2381fcc7229d68f80122c3b22aa5002b Status $ gridsam-status \ -s https://gridsam-test.oerc.ox.ac.uk:18443/gridsam/services/gridsam \ -j urn:gridsam:2381fcc7229d68f80122c3b22aa5002b Job Progress 'urn:gridsam:2381fcc7229d68f80122c3b22aa5002b': pending -> staging-in ->

7. Tests and Examples Javatest.jsdl <JobDefinitionxmlns="http://schemas.ggf.org/jsdl/2005/11/jsdl"> <Application> <POSIXApplicationxmlns="http://schemas.ggf.org/jsdl/2005/11/jsdl-posix"> <Executable>/usr/ngs/JAVA</Executable> <Argument>java</Argument> <Argument>-jar GridSamHelloWorld-test.jar</Argument> <Output>stdout.txt</Output> <Error>stderr.txt</Error> <WorkingDirectory>/home/ngs0001</WorkingDirectory> </POSIXApplication> </Application>

8. Tests and Examples Javatest.jsdl <DataStaging> <FileName>stdout.txt</FileName> <CreationFlag>overwrite</CreationFlag> <Target> <URI>gsiftp://ngs.oerc.ox.ac.uk:2811/stdout.txt</URI> </Target> </DataStaging> <DataStaging> <FileName>stderr.txt</FileName> <CreationFlag>overwrite</CreationFlag> <Target> <URI>gsiftp://ngs.oerc.ox.ac.uk:2811/stderr.txt</URI> </Target> </DataStaging>

9. Tests and Examples Stage_in.jsdl <DataStaging> <FileName>mydate.txt</FileName> <CreationFlag >overwrite</CreationFlag> <DeleteOnTermination>false</DeleteOnTermination> <Source> <URI>http://www.google.com/index.html</URI> </Source> </DataStaging>

10. GridSAM NGS Users • ... project • workflow based around a hypothetical airport emergency response scenario: a building fire occurs at an airport • Web portal to submit jobs to ngs.rl.ac.uk via GridSAM • ENGAGE project • The aims of the ENGAGE project are to promote the greater engagement of academic researchers in the UK with the UK's e-Infrastructure • Web portal to submit jobs to an application that submits to ngs.oerc.ox.ac.uk via GridSAM • Single user • Java API

11. Future Work • Production level service • Write user-oriented NGS howto • Further stress testing to address current instabilities • Migration to NGS cloud infrastructure • OMII collaboration • Code auditing • BES Interoperability Test