From Sandbox to Playground: Dynamic Virtual Environments in the Grid

1. From Sandbox to Playground: Dynamic Virtual Environments in the Grid Kate Keahey keahey@mcs.anl.gov Argonne National Laboratory Karl Doering University of California, Riverside Ian Foster Argonne National Laboratory

2. Realizing the Grid Vision • Quality of Service • Protocol, agreement, advance reservation • The ability to enforce what was agreed on • Quality of Life • Being able to find the right configuration on the Grid Kate Keahey

3. Quality of Service • Some form of control over remote nodes • Enforcement of multiple qualities • CPU, disk, memory, network traffic… • More than per-process enforcement • Process group: a master process starts other processes • Dynamically modifiable to reflect changing policies and state in the Grid • Not just quality of service • Quality of Protection, etc… QoX Kate Keahey

4. Quality of Life • The right node configuration is hard to find • Operating system and architectural differences • Different Linux distributions • 64 bit vs 32 bit • Library signature and versioning • The ability to customize a remote execution environment • Effortless configuration of remote nodes • Subject to policies • Quality of Life for multiple groups of Grid users • Avoiding maintenance nightmare, etc. Kate Keahey

5. We Need a Sandbox • A configurable execution environment, container • Virtualizes Grid Node Configuration • Sandbox = Dynamic Virtual Environment (DVE) • We need to be able to create and manage it • Quota, termination, etc. available technology requirements solutions • How can DVEs be implemented? • Relevance to our needs, quality of solution, etc. Kate Keahey

6. DVE: Interfaces • Implemented as Grid Services • OGSI, WSRF • Factory • Creates and configures a DVE in implementation-specific way • e.g., dynamic account, deploys a VM • Writes/configures access and management policy • E.g., modify the GT3 gridmapfile • DVE Service • Interface providing DVE management • E.g., explicit or soft-state termination (implies policy updates) • Access policy management • Allows for inspecting and modifying DVE properties • E.g., hardware properties such as quota or software configuration Kate Keahey

7. DVE Implementations: Requirements • What is a “container”? • General • Not require users to e.g., use a specific language • Non-invasive • Proof-carrying code, etc. • Strong protection environment • Otherwise users won’t trust sites and sites won’t trust users • Isolate users from each other • Fine-grain enforcement • Configurable architecture, software, environment • Configurable environment throughout the software stack • Application software/libraries/licenses • Potentially: execution state • Allow migration Kate Keahey

8. (1) DN DVEFactory Service (4) GSH PEP gridmapfile (2) (5) setuid (3) DVESservice local DVE implementation PEP (6) Request+GSH GRAM DVEs and the Globus Toolkit Client Kate Keahey

9. DVE Implementations • Unix accounts • Pros: efficient, ubiquitous • Cons: very limited enforcement • Enforcement properties can be improved if used in conjunction with other technologies • setrlimit, DSRT, chroot, chown, and others • Sandboxes • VServer: protection, sharing and fine-grain enforcement • Pros: efficient, fine-grain enforcement, typically very lightweight • Cons: limited state enforcement, configuration flexibility • Adjustments needed to fully leverage fine-grain enforcement Kate Keahey

10. DVE Implementations (cntd) • Virtual Machines • VMware (not evaluated, but very promising: Xen) • Pros: • Flexibility (run linux on linux, 32 on 64-bit, etc.) • Enhanced security, audit forensics, etc. • Great user state management • Freezing/migration • Customized environment • A promising distribution/deployment tool • Cons: • Potential for being less efficient (emulation) • Potential for resource overhead • Poor implementation of sharing, relatively little enforcement (but can be combined with other technologies for enforcement) • Maturity issues • The potential is excellent, but needs more work Kate Keahey

11. The Need for Speed Comparison using the Fusion EFIT application Kate Keahey

12. Other efficiency concerns • Startup time • Resource usage overhead • Memory use: VMware: 24MB + 1 MB per 32 MB memory allocated • Disk use: large for VMware Kate Keahey

13. Enforcement Capabilities Kate Keahey

14. DVE Comparison • Dynamic Accounts • Adduser versus pooled accounts • A limited but one that is here to stay… at least for now • VServer • Interesting: sharing and efficiency • VMware • No sharing • Least efficient • Migration, flexibility, etc. • General criteria • Efficiency: very acceptable, also see Xen • Enforcement: uneven, needs more research • Virtual Machines lead as far as configurability and user state representation • Sharing • Potential for replication • One VM per machine model? Kate Keahey

15. Implementation Status • Prototype available (GT 3.2) • Karl Doering: http://www-unix.mcs.anl.gov/~keahey/DS/DynamicSessions.htm • GT4 Implementation • adduser versus account pools • Better policy handling • Virtual machines and other implementations • Work in progress • SC04 poster: • P05: “Quality of Life in the Grids: VMs Meet Bioinformatics Applications”, with T. Freeman and D. Galron Kate Keahey

16. From Sandbox to Workspace • Virtual Workspaces • VWs are represented by an ontology description • Virtual resource characteristics, software stack, etc. • Potentially integrating community policy • They can be copied, etc. • They can be implemented using different technologies • They can be customized by the user • Deployed, managed and terminated in implementation-specific way • Entails some changes to the architecture Kate Keahey

17. request use existing VW deploy & suspend Virtual Workspaces in the Grids VW Factory create new VW VW EPR Create VW VW Repository inspect and manage Client Resource VW Manager VW start program Kate Keahey

18. From Sandbox to Playground • How will this affect interactions in the Grid? • Other than add many new capabilities • A larger role for the virtual organization • Account screening process: resource owner -> virtual organization • Should a VO be a legal entity? • Needs new privileges if takes on more responsibility • Administration of VWs • VW repository and other services, potentially VW certification • Sharing between VWs • More policies • Changes to many Grid services • May depend on the implementation we use • Security, networking, potentially others • Top-down model for building a Grid • Define a Grid in terms of requirements Kate Keahey

19. Conclusions • For Grids to scale we need a way to create and manage remote environments in the dynamically and effortlessly • Implementations will vary • Virtual is the new Real! • VMs present a very compelling solution… • Efficiency, flexibility, migration, etc. • …and introduce some new challenges • New services, different models of sharing, security, etc. • A growing role for Virtual Organizations • Policy, Policy, Policy… • Policy of resource owners, VOs, users… • Using WS-Agreement to negotiate virtual workspaces? • Have we exchanged one problem for another? • www.mcs.anl.gov/~keahey Kate Keahey