Virtualization in Clusters and Grids

1. Virtualization in Clusters and Grids Dr. Lizhe Wang

2. Virtualization in Cluster/Grids • On demand computing resource provision • with desired OS, software configuration, • with “root” privilege • Easy management from resource provision side • Resource accounting • Startup/shutdown/clone/migration,

3. Topics • Virtualization for a cluster scheduler • Xen Grid Engine • COD: cluster on demand • In-VIGO @ UFL • Virtuoso @ NWU • SODA

4. Virtual machine in Cluster:Computing cluster context • Existing cluster scheduler distributes jobs to cluster nodes • Jobs may come from local users or remote users (grid) • Problem: • Jobs have different resource requirements: OS, software package • Jobs may require QoS guarantee • Security issues

5. Virtual machine in Cluster:Solution • Prepare a set of virtual machine templates • On demand start up virtual machines when jobs come • Cluster scheduler distributes jobs to virtual machine nodes • No change on existing cluster scheduler • Programming with cluster scheduler interface

6. Virtual machine in Cluster:Implementation • With Maui/Torque • In University Karlsruhe, Germany • Used for LCG Grid project • Computing jobs for huge data processing

7. XGE: Xen & cluster scheduling • A share-used compute cluster • Improve the performance of cluster usage • Work from Marburg, Germany • Based on Sun Grid Engine

8. XGE: Cluster usage

9. XGE: Cluster scheduling • Parallel job submission • qsub with reservation • qsub without reservation • Backfilling • Problem: • My quota, why backfilling? • I did not get quick response!

10. XGE: requirements • User should be entitled to speedy job execution within their quotas. • Unused CPU time of a user may be consumed freely byother users when needed. • To maximize overall cluster performance, serial jobs should run whenever possible. • Parallel jobs should have waiting times as short as possible. • To minimize response time, parallel jobs should get as many CPUs as needed (definitively more than 32) without increasing the waiting time or reducing the overall cluster performance. • Any modification of the scheduling strategy should be easy to use and transparent for administrators and users to avoid arguments.

11. XGE: solution

12. XGE: implementation

13. Cluster on Demand: goals • Secure isolation of multiple user communities • Custom software environments • Dynamic policy-based resource provisioning • As a Grid site manager • Balancing local vs. global resource use • Controlled provisioning for grid services • Resource reservation

14. Node Management • As the node boots, the COD servers shape its view of its environment: • COD assigns node IP addresses within a subnet for each vcluster. • Each vcluster occupies aprivate DNS subdomain de rived from the vcluster’s symbolic name assigned at creation time. • Each vcluster executes within a predefined NIS do main, which enables access for user identities and net groups enabled for the vcluster. • COD exports NFS file storage volumes as groups and vclusters are defined.

15. COD architecture

16. Virtual Cluster Manager of COD • for each vcluster that hosts a dynamic service: vcm • contain the logic for monitoring load and changing membership in the active server set for the specific application environment. • handles the details of resource negotiation with the COD manager.

17. VCM implements SGE scheduler • Add_node • Remove_node • Resize

18. VMShop • In-VIGO from UFL • a virtual machine management system • providing application VM based execution environments for Grid Computing. • http://www.acis.ufl.edu/~aganguly/vmshop/

19. VMShop operations • Creating new VM. • Configuring existing VM. • Estimate cost of creating a new VM. • Attribute-value based querying of VMs. • Collect (or destroy) VM.

20. VMShop architecture

21. VM description • VMs are described using a DAG encoded in XML strings. • The VMPlant servers maintain a library of cached VM images, from which new VMs can be cloned • The new VM DAG starts with the node identifying the cached image from which to clone, followed by nodes which may include configuring network, mounting application data files etc.

22. In-VIGO • In-VIGO provides a distributed environment where multiple application instances can coexist in virtual or physical resources, such that clients are unaware of the complexities inherent to grid computing. • From UFL • http://invigo.acis.ufl.edu/

23. Three layer of virtualization • virtual resource, “primitive” components: • virtual machines • virtual data • virtual applications • virtual networks. • Virtual computing grids • grid applications are instantiated as services • Virtual interface • aggregated services (possibly presented to users via portals) export interfaces

24. Three layer of virtualization

25. Virtuoso • Distributed/Grid Computing Using VMs • A complete system with VM provision, scheduling, virtual network, automatic application environment provision, information service • http://virtuoso.cs.northwestern.edu/ • From Northwestern Univ.

26. Complexity from User’s Perspective • Process or job model • Lots of complex state: connections, special shared libraries, licenses, file descriptors • Operating system specificity • Perhaps even version-specific • Symbolic supercomputer example • Need to buy into some Grid API • Install and learn potentially complex Grid software

27. Complexity from Resource Owner’s Perspective • Install and learn potentially complex Grid software • Deal with local accounts and privileges • Associated with global accounts or certificates • Protection/Isolation • Support users with different OS, library, license, etc, needs.

28. The Virtuoso Model (1) • User orders raw machine(s) • Specifies hardware and performance • Basic software installation available • Virtuoso creates raw image and returns reference • Image contains disk, memory, configuration, etc. • User “powers up” machine • Virtuoso chooses provider • Information service • Virtuoso migrates image to provider • Efficient network transfer

29. The Virtuoso Model (2) • Provider instantiates machine • Virtual networking ties machine back to user’s home network • Remote device support makes user’s desktop’s devices available on remote VM • Remote display support gives user the console of the machine (VNC) • Resource control to give user expected performance • User goes to his network admin to get address, routing for his new machine • User customizes machine • Feeds in CDs, floppies, ftp, up2date, etc.

30. The Virtuoso Model (3) • User uses machine • Shutdown, hibernate, power-off, throw away • Virtuoso continuously monitors and adapts • Virtual network as a monitoring platform • Various mechanisms, all invisible to user • Migrating the machine • Routing traffic between machines • Virtual network topology • Predictive scheduling versus reservations • Various goals • Price • Interactivity • Direct User Feedback

31. SODA • A Service-On-Demand Architecture for Application Service Hosting Utility Platforms • Utility computing concept • Application service • On-demand providing service on the Hosting Utility Platform • From Purdue Univ.

32. SODA architecture