OMII-UK Campus Grid Toolkit

1. OMII-UK Campus Grid Toolkit NW-GRID Campus Grids Workshop 31st October 2007 University of Liverpool Tim Parkinson OMII-UK Southampton Operations Manager

2. OMII-UK • What is OMII-UK? • Collaboration between Open Source Software developers at Southampton, Manchester, Edinburgh. • Mission: • OMII-UK aims to provide software and support to enable a sustained future for the UK e-Science community and its international collaborators. • Software Solutions for e-Research • Led by user / community requirements

3. Characteristics of a Campus Grid Generally Speaking.... • One or more • HPC Clusters • Condor Pools • Specialised Computation Devices or Data Resource • Owned and managed by a single institution • Subject to single AuthN regime and internal AuthZ policies? • Shared by users belonging to the institution • Applications / Services useful to users • Maintenance & support of that environment

4. Uses of a Campus Grid • Typically • To run large scale computations or simulations • That can be partitioned into scenarios or parameter sweeps. • That therefore benefit by running many scenarios in parallel or on larger machines. • Reduce Time to Publication • Obtain results that would be unobtainable in available time if done sequentially or on a smaller desktop.

5. Requirements of Campus Grid Users • Determined from a survey of the UK Campus Grid SIG membership • Chaired by David Wallom, OeRC • Backed up by the SUPER report • Study of User Priorities for e-Infrastructure for e-Research (NeSC 2007) • http://www.nesc.ac.uk/technical_papers/UKeS-2007-01.pdf

6. SIG Requirements • Five areas: Applications How to transparently execute applications across heterogeneous resources to maximise throughput and minimise execution time. Identify common applications for shrink-wrapping. Security Users: Just want to login once and use what they are allowed to use transparently or to be refused gracefully. Admins: Want to be able to apply suitable authentication method and suitable access control policy. Want to support attribute based virtual organisations (VOMS / Shibboleth) to support collaborations.

7. SIG Requirements • Five areas (ctd) • Accounting: • Admins: Driven by FEC. Associate usage with user. Ability to log, price, and ultimately to bill. • Users: Any such billing should be fair and trustworthy. • Monitoring • Admins: Track resource utilisation and availability • Users: See what is happening to their jobs.

8. SIG Requirements • Five areas (ctd) • Storage • Users: ability to seamlessly transfer data to and from a variety of distributed storage systems (such as SRB) or databases. • Admins: ability to configure and such storage mechanisms into job services and to monitor them.

9. SUPER Requirements • Similar to CG SIG • Spectrum of User Interaction Styles • Web Portals Wrapped Applications • Desktop GUI • CLI • Scripting Languages • Programmatic API Custom Applications

10. Quality Requirements • Installability • Ease of installation and configuration. • Reliability • Should stay up and running or at least fail gracefully • Portability / Availability • Should work on a range of different architectures and operating systems and back end job managers. • Scalability • Should work for personal installations up to Campus Wide and beyond.

11. What is the Campus Grid Toolkit? • An enhanced packaging of existing and future OMII-UK components, principally GridSAM, that provides • Consistent job submission across heterogeneous resources for the scientist (via OGF standard JSDL) over a Web Services interface. • Ease of installation and configuration for the administrator.

12. What is the Campus Grid Toolkit • Provides (ctd) • a range of interaction styles for the scientist (Desktop, CLI, Portal / Portlets) • a range of configurable security policies for the administrator (OMII-AuthZ, SPAM-GP, VOMS / Shibboleth integration). • a way to wrap legacy, unmodified applications (AHE, OGRSH) • a way to create new applications that access grid resources directly (SAGA and its scripting bindings)

13. What is the Campus Grid Toolkit? • A vehicle to deliver an integrated set of OMII-UK components that work together to enhance the scientist’s ability to submit large numbers of computational jobs to the resources available on campus and beyond in a seamless fashion. • Should become the installation of choice for Campus Grid providers.

14. First Release (Nov 2007) • Address the Installability of GridSAM onto a Condor pool. • GridSAM on Condor in a Box. • Autoconfiguration • Better example programs • Address reliablity issues • Prototype demonstrated at OGF21

15. The Old Way – Component Based • Follow an installation and configuration process (1-2 hours): • Install the OMII-Server bundle and select GridSAM as an option • Use temporary server certificate • Configure the Condor DRM yourself • Download and install the client • Use temporary client certificate • Test with uname application (too trivial) • Make client available to users • Replace temporary CA and cert with real ones

16. The New Way • Move towards one-step installation and configuration (10 mins): • Install CGT which offers the option to trust UK CA and configures server accordingly • Attempts to install trust for UK CA and your own ‘real’ certificate • Attempts to detect and auto-configure Condor DRM • Automatically installs and configures client to match the server that was just installed • Test with Mandelbrot set – a real application requiring significant processing time • Make pre-configured client available

17. Future Plans (no particular order) • Add one-step installation and auto-configure for AHE (Demo SC2007) • Extend one-step installation to other DRMs (Globus, PBS/Torque, Platform LSF etc) • Extend GridSAM to propagate user identity and to collect resource usage for jobs. • Investigate ways to add grid monitoring, perhaps with OGM • Investigate job service brokering perhaps with Knoogle or Grid-BS.

18. Future Development • Enhance all major OMII-UK components to interact with the main storage management solutions.

19. Questions?