1 / 32

Division of Labor: Tools for Growing and Scaling Grids

Division of Labor: Tools for Growing and Scaling Grids. Tim Freeman, Kate Keahey, Ian Foster, Abhishek Rana, Frank Wuerthwein, Borja Sotomayor. Division of Labor.

oma
Download Presentation

Division of Labor: Tools for Growing and Scaling Grids

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Division of Labor:Tools for Growing and Scaling Grids Tim Freeman, Kate Keahey, Ian Foster, Abhishek Rana, Frank Wuerthwein, Borja Sotomayor

  2. Division of Labor The greatest improvements in the productive powers of labour , and the greater part of the skill, dexterity, and judgment with which it is anywhere directed, or applied, seem to have been the effects of the division of labour. (Adam Smith) How can we implement division of labor in Grid computing? tools to implement an abstraction requirements for an abstraction ICSOC ‘06

  3. Overview • Problem Definition • The Edge Service Use Case • Workspace Service • Overview of the workspace service • Extensions to workspace service • Implementation and Evaluation • CPU enforcement • Network Enforcement • Status of the Edge Services Project • Conclusions ICSOC ‘06

  4. Overview • Problem Definition • The Edge Service Use Case • Workspace Service • Overview of the workspace service • Extensions to workspace service • Implementation and Evaluation • CPU enforcement • Network Enforcement • Status of the Edge Services Project • Conclusions ICSOC ‘06

  5. Has a limited number of resources Want the resources when they need them & as much as they need Has to balance the software needs of multiple users Want to use specific software packages Has to provide a limited execution environment for security reasons Wants as much control as possible over resources Providers and Consumers Resource provider Resource consumers ICSOC ‘06

  6. The Edge Service Use Case ICSOC ‘06

  7. Edge Services: Challenges • VO-specific Edge Services • Each VO has very specific configuration requirements • Resource management • The VOs would like to provide quality of service to their users • The resource needs of the VOs are change dynamically • Dynamic, policy-based deployment and management of Edge Services • Updates, ephemeral edge services, infrastructure testing, short-term usage ICSOC ‘06

  8. Division of Labor Dimensions • Environment and Configuration • Isolation • Critical from the point of view of the provider if the VOs are to be allowed some independence • Resource usage and accounting • Application-independent • Management along different resource aspects • Dynamically renegotiable/adaptable ICSOC ‘06

  9. Overview • Problem Definition • The Edge Service Use Case • Workspace Service • Overview of the workspace service • Extensions to workspace service • Implementation and Evaluation • CPU enforcement • Network Enforcement • Status of the Edge Services Project • Conclusions ICSOC ‘06

  10. GT4 workspace service • The GT4 Virtual Workspace Service (VWS) allows an authorized client to deploy and manage workspaces on-demand. • GT4 WSRF front-end • Leverages multiple GT services • Currently implements workspaces as VMs • Uses the Xen VMM but others could also be used • Current release 1.2.1 (December, 06) • http://workspace.globus.org ICSOC ‘06

  11. Workspace Service Usage Scenario The VWS manages a set of nodes inside the TCB (typically a cluster). This is called the node pool. Pool node Pool node Pool node The workspace service has a WSRF frontend that allows users to deploy and manage virtual workspaces VWS Service Pool node Pool node Pool node VWS Node Each node must have a VMM (Xen) installed, along with the workspace backend (software that manages individual nodes) Pool node Pool node Pool node Image Node Pool node Pool node Pool node VM images are staged to a designated image node inside the TCB Trusted Computing Base (TCB) ICSOC ‘06

  12. Adapter-based implementation model Transport adapters Default scp, then gridftp Control adapters Default ssh Deprecated: PBS, SLURM VW deployment adapter Xen Previous versions: VMware Deploying Workspaces Pool node Pool node Pool node VWS Service Workspace - Workspace metadata - Resource Allocation Pool node Pool node Pool node Pool node Pool node Pool node Image Node Pool node Pool node Pool node ICSOC ‘06

  13. Interacting with Workspaces The workspace service publishes information on each workspace as standard WSRF Resource Properties. Pool node Pool node Pool node VWS Service Pool node Pool node Pool node Users can query those properties to find out information about their workspace (e.g. what IP the workspace was bound to) Pool node Pool node Pool node Image Node Pool node Pool node Pool node Users can interact directly with their workspaces the same way the would with a physical machine. Trusted Computing Base (TCB) ICSOC ‘06

  14. Deployment Request Arguments • A workspace, composed of: • VM image • Workspace metadata • XML document • Includes deployment-independent information: • VMM and kernel requirements • NICs + IP configuratoin • VM image location • Need not change between deployments • Resource Allocation • Specifies availability, memory, CPU%, disk • Changes during or between deployments ICSOC ‘06

  15. Workspace Factory Service Workspace Service Workspace Resource Instance Workspace Service Interfaces Handles creation of workspaces. Also publishes information on what types of workspaces it can support Workspace Meta-data/Image Create() Resource Allocation authorize & instantiate inspect & manage Workspace Service notify Resource Properties publish the assigned resource allocation, how VW was bound to metadata (e.g. IP address), duration, and state Handles management of each created workspace (start, stop, pause, migrate, inspecting VW state, ...) ICSOC ‘06

  16. Extensions to Resource Allocation ICSOC ‘06

  17. Overview • Problem Definition • The Edge Service Use Case • Workspace Service • Overview of the workspace service • Extensions to workspace service • Implementation and Evaluation • CPU resource allocation • Network resource allocation • Status of the Edge Services Project • Conclusions ICSOC ‘06

  18. GRAM VO1 VO1 VO2 Edge Services Today Compute Element (CE) implemented as GT GRAM Both VOs share the same resource 7.83 jpm 8 jpm Job throughput is low as both VOs are equally impacted by the high VO1 traffic ICSOC ‘06

  19. GRAM GRAM VO1 VO1 VO2 Allocating Resources for Edge Services Resource Allocation: MEM: 896 MB CPU: CPU %: 45% CPU arch: AMD Athlon 4.18 jpm Resource Allocation: MEM: 896 MB CPU: CPU %: 45% CPU arch: AMD Athlon 22.36 jpm Workspace Service Dom0 CPU %: 10% Job throughput for VO2 is high as it is unimpacted by the high VO1 traffic ICSOC ‘06

  20. Tracking Requests Overtime • Histogram of • request throughput • - Resource usage • is enforced on an • “as needed” basis ICSOC ‘06

  21. Increasing Load on VO1 • Histogram of • request • throughput • The load on • VO1 increases • 2x and 3x • Request • throughput • for VO2 is • unimpacted ICSOC ‘06

  22. Network Resource Allocation • Processing network traffic requires CPU • In Xen: for both dom0 and guest domains • CPU allocation tradeoffs • Scheduling frequency • The mechanism is general • Save for direct drivers domU dom0 B domU ICSOC ‘06

  23. Network Resource Allocation • Network Allocation Implementation • CPU allocations based on a parameter sweep • Close to maximum bandwidth • Linux network shaping tools • Negotiating network resource allocations • Policy: accepting only CPU allocations that match the bandwidth ICSOC ‘06

  24. VO2 GridFTP VO1 GridFTP VO1 VO2 Storage Element (SE) Edge Service Resource Allocation: MEM: 128 MB CPU: CPU %: 6% CPU arch: AMD Athlon NIC: Incoming: 4.1 MB/s Resource Allocation: MEM: 128 MB CPU: CPU %: 6% CPU arch: AMD Athlon NIC: Incoming: 4.1 MB/s Workspace Service Dom0 CPU %: 22% ICSOC ‘06

  25. Negotiating Bandwidth ICSOC ‘06

  26. VO2 GridFTP VO1 GridFTP Renegotiating CPU and Bandwidth Resource Allocation: MEM: 128 MB CPU: CPU %: 6% CPU arch: AMD Athlon NIC: Incoming: 4.1 MB/s Resource Allocation: MEM: 128 MB CPU: CPU %: 14% CPU arch: AMD Athlon NIC: Incoming: 8.2 MB/s Resource Allocation: MEM: 128 MB CPU: CPU %: 6% CPU arch: AMD Athlon NIC: Incoming: 4.1 MB/s Workspace Service Dom0 CPU %: 22% ICSOC ‘06

  27. Renegotiating CPU and Bandwidth ICSOC ‘06

  28. VO2 GridFTP VO1 GridFTP Renegotiating CPU Resource Allocation: MEM: 128 MB CPU: CPU %: 14% CPU arch: AMD Athlon NIC: Incoming: 8.2 MB/s Resource Allocation: MEM: 128 MB CPU: CPU %: 34% CPU arch: AMD Athlon NIC: Incoming: 8.2 MB/s Resource Allocation: MEM: 128 MB CPU: CPU %: 6% CPU arch: AMD Athlon NIC: Incoming: 4.1 MB/s Workspace Service Dom0 CPU %: 22% ICSOC ‘06

  29. Renegotiating CPU ICSOC ‘06

  30. Edge Services: Status • OSG activity • www.opensciencegrid.org/esf • Edge Services in use (database caches) • ATLAS: mysql-gsi db built by the DASH project • CMS: frontier database • Base Image library • SDSC: SL3.0.3, FC4, CentOS4.1 • FNAL: SL3.0.3, SL4, LTS 3, LTS 4 • Sites • Production: SDSC • also testing at FNAL, UC and ANL ICSOC ‘06

  31. Related Work • Edge Service efforts • VO boxes, EGEE • APAC, static Edge Services • Grid-Ireland, static Edge Services • OGF efforts: WS-Agreement, JSDL • Managed Services • QoS with Xen • Padma Apparo, Intel (VTDC paper) • Rob Gardner & team, HP • Credit-based scheduler • Grid computing and virtualization • Work at University of Florida, Purdue, Northwestern, Duke and others ICSOC ‘06

  32. Conclusions • VM-based workspaces are a promising tool to implement “division of labor” • Renegotiation is an important resource management tool • Protocols • Enforcement methods: dynamic reallocation, migration, etc. • Aggregate resource allocations • Different resource aspects influence each other • More work on managing VM resources is needed ICSOC ‘06

More Related