1 / 16

Virtual Workspaces in the Grid

Virtual Workspaces in the Grid. Kate Keahey keahey@mcs.anl.gov Argonne National Laboratory Ian Foster, Tim Freeman, Xuehai Zhang, Daniel Galron. What happens if a power station fails? . How do we store energy?. How do we charge for energy?. What elements make for a safe

enid
Download Presentation

Virtual Workspaces in the Grid

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Virtual Workspaces in the Grid Kate Keahey keahey@mcs.anl.gov Argonne National Laboratory Ian Foster, Tim Freeman, Xuehai Zhang, Daniel Galron

  2. What happens if a power station fails? How do we store energy? How do we charge for energy? What elements make for a safe and efficient power Grid? How do we ensure quality of service? How do we reliably deliver energy? How do we make sure that supply meets demand? The Grid Metaphor Grid Computing is much harder: heterogeneous and multi-dimensional Kate Keahey, Europar 2005

  3. Dream up an ideal environment magic happens Ideal environment is deployed The Missing Link in Grid Computing • We need to define mechanisms for and dynamic deployment and management of remote environments • Requirements: • Flexibly define an environment • The more we can customize it, the more useful it is • Deploy and manage such environments • Can such environments be deployed securely? • How fast/dynamic can this deployment be? • How can I control resources allocated to such an environment? Kate Keahey, Europar 2005

  4. Virtual Workspaces • Virtual Workspaces: environments that can be made available dynamically the Grid with well-understood properties • Examples: • A TeraGrid node with well-defined software environment and adjustable access and sharing policies • A physical cluster booted to a desired configuration (e.g. Cluster on Demand) • An “ATLAS node” dynamically configured using Pacman • A virtual machine configured to represent a specific environment whose resource consumption can be controlled Kate Keahey, Europar 2005

  5. 1.1 1.0 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0.0 L X V U L X V U L X V U L X V U SPEC INT2000 (score) Linux build time (s) OSDB-OLTP (tup/s) SPEC WEB99 (score) Virtual Machines as Workspaces • Virtual Machines • Highly customizable software configuration • Enforcement properties • Grid 2004 paper: “Dynamic environments in the Grid” • F. Cappello & lab: Comparison of different hypervisors • Pausing, serialization, migration • Performance: SOSP 2003 paper: “Xen and the Art of Virtualization” Kate Keahey, Europar 2005

  6. Workspace Template Aspects • Environment Aspect (workspace meta-data) • Generic information • Name, time to live, etc. • Software partition information • Software description: OS, “OSG configuration”, “application partition”, etc. • Software meta-data is bundled with the actual software and attested by its issuer • Services: ssh, GRAM, pre-configured job • Deployment independent • Resource allocation request (deployment time) • Memory, disk, networking, etc. • See GGF JSDL standard • On deployment the actual resource allocation information becomes available Kate Keahey, Europar 2005

  7. Atomic Workspaces and Virtual Clusters • Atomic workspace • One or more homogeneous workspaces • The only differences are in names • Cluster/aggregate workspace • A set of interdependent heterogeneous workspaces • Example: a headnode and a set of worker nodes • Interdependencies of metadata are expressed through tags and pointers Kate Keahey, Europar 2005

  8. Define workspace environment Manage workspace Negotiate workspace deployment characteristic request a workspace workspace meta-data negotiate workspace deployment manage/monitor/renegotiate workspace deployment Workspace Deploying Workspaces in the Grid Workspace Wizard (VW Factory) manage workspace environment Workspace Management Service (VW Repository) workspace metadata Workspace Service (VW Manager) terminate workspace deployment manage activities within the workspace Kate Keahey, Europar 2005

  9. Current Implementation • Current prototype using Globus Toolkit 4 • Leveraging standard Grid Service features such as lifetime management • Workspace Wizard • Returns workspace meta-data • Very rudimentary implementation • Workspace Serivce • Create: takes workspace meta-data and a deployment descriptor • Manage: • renegotiate resource allocation (moving towards a WS-Agreement model) • Also traditional Grid Service management: TTL, etc. • Destroy • Different options: pause, shutdown or destroy Kate Keahey, Europar 2005

  10. How dynamic is the deployment? • Automatic • Protocol-based • Moving towards better articulation of migration • Renegotiation of resource allocation • How fast is this deployment? • Deployment of workspace for EMBOSS suite: • Manual: ~45 minutes • Based on pre-configured Vmware VMs: ~6 minutes • Based on pre-configured Xen VM: < 1 second • How much overhead does workspace deployment add over what we have today? Kate Keahey, Europar 2005

  11. Workspace Service: Individual Workspaces • Using a paused VM allowed us to “save” on initiation time GRAM job execution GRAM job execution in a paused Xen VM job execution in a booted Xen VM (pre-configured job) Kate Keahey, Europar 2005

  12. Workspace Service: Virtual Clusters Kate Keahey, Europar 2005

  13. Deploying Workspaces Across Technologies • Basic node configuration (+/-boot from image) • Cluster on Demand, PXE, bcfg • On the order of many minutes (~30 minutes) • Refining configuration, creating access • Dynamic account with workspace service: < 1s (mostly GT4 request processing time) • Refining Installation: ~2 hours to configure an ATLAS node using Pacman • Virtual machines • Deploying images • Xen: ~100 ms • VMware Workstation: ~ several seconds Kate Keahey, Europar 2005

  14. program program program … VM deploy VM workspace (with hypervisor/OS) VM VM deploy hypervisor/OS workspace Hypervisor/OS procure hardware Physical machine Nested Workspaces Kate Keahey, Europar 2005

  15. Grid Power Station Hypervisor 1 Hypervisor 2 TeraGrid Configuration Grid Protocols Clients Computational Grids Kate Keahey, Europar 2005

  16. Conclusions • We need mechanisms for dynamically deploying and managing environments in the Grid • Workspaces are a fundamental building block of a Grid environment • Workspaces are implemented using wide variety of technologies • VMs are a highly promising one: a “computon” for the Grid • Workspace aspects • Deployment-independent environment definition • Deployment-time policy and enforcement negotiation • Many challenges remain • Security and deployment issues • Protocols, protocols, protocols • Leveraging the opportunities Kate Keahey, Europar 2005

More Related