1 / 28

Dynamic Virtual Clusters in a Grid Site Manager

This presentation focuses on the utilization of Cluster-On-Demand (COD) virtual clusters for dynamic grid services provisioning and middleware integration. It covers system architecture, virtual cluster management, GridEngine applications, and various provisioning policies, with experimental results highlighting the effectiveness of Minimum Reservation Policy implementation. The talk also delves into related cluster management technologies and proposes future work on economic-based policies for batch jobs and distributed resource trading in virtual clusters.

larochelle
Download Presentation

Dynamic Virtual Clusters in a Grid Site Manager

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Dynamic Virtual Clusters in a Grid Site Manager Jeff Chase, David Irwin, Laura Grit, Justin Moore, Sara Sprenkle Department of Computer Science Duke University

  2. Dynamic Virtual Clusters Grid Services Grid Services Grid Services

  3. Motivation • Next Generation Grid • Flexibility • Dynamic instantiation of software environments and services • Predictability • Resource reservations for predictable application service quality • Performance • Dynamic adaptation to changing load and system conditions • Manageability • Data center automation

  4. Cluster-On-Demand (COD) Virtual Cluster #1 • Differences: • OS (Windows, Linux) • Attached File Systems • Applications • User accounts COD DHCP Virtual Cluster #2 NIS NFS DNS • Goals for this talk • Explore virtual cluster provisioning • Middleware integration (feasibility, impact) COD database (templates, status)

  5. Cluster-On-Demand and the Grid • Safe to donate resources to the grid • Resource peering between companies or universities • Isolation between local users and grid users • Balance local vs. global use • Controlled provisioning for grid services • Service workloads tend to vary with time • Policies reflect priority or peering arrangements • Resource reservations • Multiplex many Grid PoPs • Avaki and Globus on the same physical cluster • Multiple peering arrangements

  6. Outline • Overview • Motivation • Cluster-On-Demand • System Architecture • Virtual Cluster Managers • Example Grid Service: SGE • Provisioning Policies • Experimental Results • Conclusion and Future Work

  7. System Architecture A GridEngine Commands Middleware Layer Provisioning Policy VCM GridEngine COD Manager B VCM GridEngine VCM GridEngine XML-RPC Interface Node reallocation Sun GridEngine Batch Pools within Three Isolated Vclusters C

  8. Virtual Cluster Manager (VCM) • Communicates with COD Manager • Supports graceful resizing of vclusters • Simple extensions for well-structured grid services • Support already present • Software handles membership changes • Node failures and incremental growth • Application services can handle this gracefully COD Manager Vcluster VCM Service add_nodes remove_nodes resize

  9. Sun GridEngine • Ran GridEngine middleware within vclusters • Wrote wrappers around GridEngine scheduler • Did not alter GridEngine • Most grid middleware can support modules COD Manager Vcluster VCM Service add_nodes remove_nodes resize qconf qstat

  10. Pluggable Policies • Local Policy • Request a node for every x jobs in the queue • Relinquish a node after being idle for y minutes • Global Policies • Simple Policy • Each vcluster has a priority • Higher priority vclusters can take nodes from lower priority vclusters • Minimum Reservation Policy • Each vcluster guaranteed percentage of nodes upon request • Prevents starvation

  11. Outline • Overview • Motivation • Cluster-On-Demand • System Architecture • Virtual Cluster Managers • Example Grid Service: SGE • Provisioning Policies • Experimental Results • Conclusion and Future Work

  12. Experimental Setup • Live Testbed • Devil Cluster (IBM, NSF) • 71 node COD prototype • Trace driven---sped up traces to execute in 12 hours • Ran synthetic applications • Emulated Testbed • Emulates the output of SGE commands • Invisible to the VCM that is using SGE • Trace driven • Facilitates fast, large scale tests • Real batch traces • Architecture, BioGeometry, and Systems groups

  13. Live Test

  14. Architecture Vcluster

  15. Emulation Architecture Architecture Trace Load Generation Systems Trace BioGeometry Trace Provisioning Policy VCM Emulated GridEngine FrontEnd COD Manager Emulator VCM qstat VCM XML-RPC Interface • Each Epoch • Call resize module • Pushes emulation forward one epoch • qstat returns new state of cluster • add_node and remove_node alter emulator COD Manager and VCM are unmodified from real system

  16. Minimum Reservation Policy

  17. Emulation Results • Minimum Reservation Policy • Example policy change • Removed starvation problem • Scalability • Ran same experiment with 1000 nodes in 42 minutes making all node transitions that would have occurred in 33 days • There were 3.7 node transitions per second resulting in approximately 37 database accesses per second. • Database scalable to large clusters

  18. Related Work • Cluster Management • NOW, Beowulf, Millennium, Rocks • Homogenous software environment for specific applications • Automated Server Management • IBM’s Oceano and Emulab • Target specific applications (Web services, Network Emulation) • Grid • COD can support GARA for reservations • SNAP combines SLAs of resource components • COD controls resources directly

  19. Future Work • Experiment with other middleware • Economic-based policy for batch jobs • Distributed market economy using vclusters • Maximize profit based on utility of applications • Trade resources between Web Services, Grid Services, batch schedulers, etc.

  20. Conclusion • No change to GridEngine middleware • Important for Grid services • Isolates grid resources from local resources • Enables policy-based resource provisioning Policies are pluggable • Prototype system • Sun GridEngine as middleware • Emulated system • Enables fast, large-scale tests • Test policy and scalability

  21. Example Epoch Architecture Nodes 4,6. Format and Forward requests 2a. qstat 1abc.resize 3a.nothing VCM GridEngine COD Manager Systems Nodes 3b.request 2b. qstat VCM GridEngine 8b. qconf add_host 7b.add_node VCM GridEngine 3c.remove 5. Make Allocations Update Database Configure nodes 7c.remove_node 2c. qstat Node reallocation 8c. qconf remove_host Sun GridEngine Batch Pools within Three Isolated Vclusters BioGeometry Nodes

  22. New Cluster Management Architecture • Cluster-On-Demand • Secure isolation of multiple user communities • Custom software environments • Dynamic policy-based resource provisioning • Acts as a Grid Site Manager • Virtual clusters • Host different user groups and software environments in isolated partitions • Virtual Cluster Manager (VCM) • Coordinates between local and global clusters

  23. Dynamic Virtual Clusters • Varying demand over time • Negotiate resource provisioning by interfacing with application specific service manager • Logic for monitering load and changing membership • Fundamental for the next-generation grid • COD controls local resources • Exports a resource negotiation interface to local grid service middleware • Vclusters encapsulate batch schedulers, Web services, Grid Services • No need to place more complicated resource management into grid service middleware

  24. Resource Negotiation • Flexible, extensible policies for resource management • Secure Highly Available Resource Peering (SHARP) • Secure external control of site resources • Soft-state reservations of resource shares for specific time intervals • COD Manager and VCM communicate through XML-RPC interface

  25. Cluster-On-Demand (COD) Clients Web Services VCM • Differences: • OS (Windows, Linux) • Attached File Systems • Applications • User accounts COD DHCP Batch Scheduler NIS VCM NFS Clients DNS • Goals • Explore virtual cluster provisioning • Middleware integration (feasibility, impact) • Non-goals • Mechanism for managing and switching configurations COD database (templates, status)

  26. Example Node Reconfiguration • Node comes online • DHCP queries status from database • If new config—loads minimum trampoline OS PXELinux • Generic x86 Linux kernel and RAM-based root file system • Sends summary of hardware to confd • Confd directs trampoline to partition drives and install images (from database) • COD assigns IP addresses within a subnet for each vcluster • Vcluster occupies private DNS domain (MyDNS) • Executes within predefined NIS domain, enables access for user identities • COD exports NFS file storage volumes • Nodes obtain NFS mount map through NIS • Web Interface • Differences: • OS (Windows, Linux) • Attached File Systems • Applications • User accounts

  27. System Architecture Architecture Nodes GridEngine Commands Local Provisioning Policy Middleware Layer Global Provisioning Policy Load from users VCM GridEngine COD Manager Systems Nodes VCM GridEngine VCM GridEngine add_nodes remove_nodes resize Load from users XML-RPC Interface qconf qstat qsub Node reallocation Sun GridEngine Batch Pools within Three Isolated Vclusters BioGeometry Nodes

  28. Outline • Overview • Motivation • Cluster-On-Demand • System Architecture • System Design • Provisioning Policies • Experimental Results • Conclusion and Future Work

More Related