StratusLab : Enhancing grid infrastructures with virtualization and cloud technologies

1. StratusLab: Enhancing grid infrastructures with virtualization and cloud technologies • C. Loomis (CNRS/LAL) • France-Grilles External Advisory Committee • 15 March 2011

2. Contents • StratusLab Project • Facts • Principles • Cloud Distribution • Integration of cloud technologies with EGI • Public, preview releases • Architecture for version 1.0 • Expectations from French Institutes • Conclusions

3. StratusLab Project • Information • 1 June 2010—31 May 2012 (2 years) • 6 partners from 5 countries • Budget : 3.3 M€ (2.3 M€ EC) • Goal • Create a comprehensive, open-sourcecloud distribution • Demonstrate production grid servicesover StratusLab cloud • Contacts • Site web: http://stratuslab.eu/ • Twitter: @StratusLab • Support: support@stratuslab.eu CNRS (FR) UCM (ES) GRNET (GR) SIXSQ (CH) TID (ES) TCD (IE)

4. Infrastructure as a Service (IaaS) • Goal • Offer remote access to computing resources • Advantages • Customized environments • Rapid access via simple API • Complete control (rootaccess) with “pay as yougo” model • Disadvantages • Non-standard interfaces(vendor lock-in) • Creating new virtualmachines is difficult Software as a Service (SaaS) Platform as a Service (PaaS) Infrastructure as a Service (IaaS)

5. Project Principles • Grid and cloud technologies are complementary • Uniform security model (grid) • Sharing of resources, algorithms, and expertise (grid) • Dynamic allocation of resources (cloud) • Customized environments (cloud) • Only develop new software when necessary • Integrate existing solutions if possible • Practical development real needs of users • Maintain production quality with rapid evolution • Use agile and scrum methodologies • Iterative integration: always maintain working distribution • Public releases approximately every 6 weeks

6. Grid and Cloud Integration users Grid Resource Center Grid Services Cloud API StratusLab Distribution PrivateCloud PublicClouds

7. Public, Preview Releases • Releases: • v0.1 (9 Nov. 2010) • v0.2 (17 Dec. 2010) • v0.3 (11 Mar. 2011) • v0.4 (~mid-Apr., +storage) • v1.0 (~end-May) • Goals • Provide complete, open-source cloud distribution • Generate feedback from users and administrators • Warning: • The APIs, commands, etc. can (and will) change! Documentation • User tutorial (web et video) • Reference deployment description • Installation guides Services • Appliance repository • Marketplace (v0.3) • Cloud infrastructure (GRNET) • User and administrator support (support@stratuslab.eu) Software • OpenNebula (2.0 in v0.1-2, 2.2 in v0.3) • Command line utilities (user and administrator) • Utilities for signing and validating image metadata (v0.2) • Site policy engine for specified images (v0.3) • IP Addresses: public, local, and private (v0.2) • Standard machine images (ttylinux, Ubuntu, CentOS) • Bioinformatics and grid services images (v0.3) • Web monitor • Manual and Quattor-based installation Security • Integration with grid certificates (v0.2) • Integration with VOMS proxies (v0.3) • Enhancing logging (v0.3) • Quarantine for forensic analysis (v0.3)

8. Demonstration

9. Roadmap • StratusLab Marketplace • “Registry” for image metadata • Tools for manipulating metadata • Appliances • Tools for creating images • Base images: CentOS, Ubuntu, ttylinux, SuSE? • Images for grid services • Specialized images: MATLAB, … • Contextualization • Dynamic configuration of machine network configuration • Configuration of SSH keys, grid certificates, etc.

10. Roadmap • Data (disk) management • Static images: fixed databases, standard data, … • Modifiable images: service state, logs, … • Management of services • Control of an ensemble of virtual machine images • Vary the resources dynamically and automatically

11. Architecture of StratusLab v1.0 users Appliance Repository HTTP(S) REST Interface Appliance Registry IaaS Cloud Appliance Stores (Web, Grid, Cloud) TCloud Service Manager (Claudia) OCCI OCCI (?), CDMI (?) Virtual Machine Manager (OpenNebula) Storage Manager KVM Xen VMware … iSCSI (?) … PhysicalComputingResources Physical Storage Resources

12. Expectation from French Institutes • CNRS/LAL • Run moderately sized public cloud for select users • Run internal cloud for laboratory services (already done) • Progressively migrate existing grid resources onto StratusLab cloud infrastructure with v1.0 • IBCP • Evaluate StratusLab distribution for use in bioinformatics • Push for migration of ReNaBi resources to StratusLab cloud • Other institutes • Expect most other (Quattor-managed) grid sites to follow LAL in using StratusLab • Already some interest in public, private clouds from others like LAPP and IRFU

13. Conclusions • StratusLab • Small project (3 M€, 20 people)  integration of existing services • StratusLab v0.2 available, v0.3 end of February • Reference infrastructure available (and open to public) • Support: support@stratuslab.eu • Expect Strong French Participation • Partners LAL and IBCP to adopt quickly • Expect other Quattor-managed grid sites to follow soon after • Interest at some sites in running public and/or private clouds independent of grid services • Would like to see France-Grille also offer cloud-based services