Overview of GridSAM and its Deployment

Overview of GridSAM and its Deployment Steve Crouch, OMII-UK s.crouch@omii.ac.uk 29/07/09

Contents • What is GridSAM and what does it offer? • For resource owners and end users • Example scenario • Packaging options • The job submission pipeline • Open development • Installation and configuration • Future developments

GridSAM Overview • What is GridSAM to the resource owners? • A web service to uniformly expose a computational resource • Condor • Portable Batch Scheduler (PBS) • Globus • Sun GridEngine • Platform Load Sharing Facility (LSF) • Single machine through Fork or SSH • Acts as a client to these resources • What is GridSAM to end-users? • A means to access computational resources in an open standards-based uniform way • A set of end-user command-line tools and client-side APIs to interact with GridSAM Web Services • Submit and monitor compute jobs • Cross-protocol file transfer (gsiftp, ftp, sftp, WebDav, http, https, soon SRB, iRODS) via Commons-VFS (http://sourceforge.net/projects/commonsvfsgrid)

For Resource Owners… Persistence provided by one of:Hypersonic, PostgreSQL, or existing MySQL X509 certificate DRM Computational Resource Manager GridSAM Service One of:PBS (Torque/OpenPBS/PBSPro)LSF, Condor, Sun GridEngine, Globus, Fork … Tomcat/Axis Tomcat: 5.0.23, 5.0.28, 5.5.23Axis: v1.2.1 Linux Many flavours:RHEL 3,4,5, Fedora 7,8, Scientific Linux 4 Java: JDK 1.5.0+ Linux + Java

For End-Users… MyProxy(for Globus/GridFTP) Globus-style Proxy Certificate Any/all of: GridSAM native interface, OGSA-BES v1.0, HPC Basic Profile v1.0 GridSAM Service Service Interface JSDL + MyProxycredentials Generic BES/HPC Basic ProfileClient HTTPS/HTTP GridSAM Client OSGA-BES … WS-Security: X509User/Password HPC Basic Profile X509 certificate Axis Windows/ Linux + Java Many flavours:RHEL 3,4,5, Fedora 7,8, Debian, Ubuntu, Scientific Linux 4, Windows XP, Windows Vista Java: JDK 1.5.0+

Example Scenario - CPOSS ENGAGE Project Job ManagementSystem GridSAM Condor … UCL Legion (PBS Torque + Moab) Portal ActiveBPEL Workflow … UK NGS GridSAM …

Package Options: CGT • Campus Grid Toolkit (CGT) – automated client or server install • Client/server solution that contains • Apache Tomcat/Axis/WSS4j (WS-Security) • GridSAM • Optional - Application Hosting Environment (AHE) • PostgreSQL or HyperSonic database • Easy, quick installation via further autoconfiguration: • Service: • Initial security setup for testing • Hooks into existing computational resources • Client: • Downloadable from existing CGT server and pre-configured for that server • End-to-end, ‘out of the box’ working examples • Currently version 1.1.3 (with GridSAM 2.1.6) • Installation videos – see http://www.omii.ac.uk/wiki/CGT

Package Options: Development Kit • Development Kit – heavily assisted client or server installation • Service comprised of: • Web Services Container: Tomcat/Axis with optional WS-Security • Data persistence: readily-installable PostgreSQL or HyperSonic databases, or pre-installed MySQL • Configurable Distributed Resource Manager (DRM) system • Client essentially ‘container’ of libraries and security configuration • Optionally downloadable and installable components for client and server: • GridSAM: Job submission and monitoring • AHE: Application Hosting Environment • Grimoires: UDDI + metadata registry • … • Currently version 3.4.4

Example Pipeline: Condor GridSAM e.g. with Condor • A staged event-driven architecture • Submission pipeline is constructed as a network of stages connected by event queues • Each stage performs a specific action upon incoming events

Open Community Development • GridSAM is Open Source, Open Community Development • GridSAM SourceForge project: • 99.03% activity, 1 release/month • SVN source code repository • Developer & discuss mailing lists http://sourceforge.net/projects/gridsam/

Installation Process – e.g. Development Kit • Pre-requisites: • Download omii-server-3.4.4.tar from OMII-UK website (Software -> Downloads) & unpack • Ensure JDK 1.5.0+ is installed & JAVA_HOME set & Java on PATH • e.g. for PBS • Ensure submit and monitor PBS commands available on PATH • Installable on PBS server node, or elsewhere with ssh access to PBS server node • cd to omii-server-3.4.4 unpacked directory and run ./OMIIstackInstall.pl once each for: • Database install: provide PostgreSQL version, install location & port • Tomcat/Axis container: Tomcat version, install location, database port, test certificate details • Database setup for optional components • Optional components: select GridSAM only • Install complete

Config. Process – e.g. Development Kit • For example, for PBS (any flavour): • GridSAM is a PBS client for submission and monitoring • Edit usermap_config.xml and supply DN -> local user account mappings • Edit batch_config.xml for local PBS configuration • PBS batch queue name to use • User with sudo privileges to enable job submission as submitting user • Ensure PBS qstat, pbsnodes, qnodes, qsub on PATH when starting container • Restart container • For CGT, you specify install requirements in an easily changed specification file prior to installation

Planned Future Developments • For end-users: • Refactored documentation and improved OGF standards coverage • Full support for JSDL DataStaging credential specification • Full support for JSDL Resource selection across PBS, Condor & Fork DRMs • JSDL Parameter Sweep Extension • Support for SRB and iRODS • For resource owners: • Packaging option as a standalone, manually configurable web archive (WAR) file • Job submission through multiple remote ssh accounts • Direct PBS deployment throughout NGS sites

Overview of GridSAM and its Deployment