自動化負載平衡子計畫

1. 自動化負載平衡子計畫 第一季報告 (2009.07~2009.09) 董信煌 樹德科技大學資管系 tungsh@mail.stu.edu.tw

2. Outline • Objectives • Resource Management in Xen • CPU Schedulers of Xen • Monitoring Performance of Xen • Methodology • Summary • Outlook

3. Objectives • 系統規劃與需求分析. 了解及熟悉 Xen 環境中與資源分配及效能分析有關之功能, 例如 Xen 對於多個執行於同一系統之虛擬機器(VM)的工作排程機制(bvt, sedf, credit等scheduler)

4. Xen • An open source server virtualization software package • This virtual machine monitor (hypervisor) offers near-native VM performance through • Paravirtualization • Hardware assisted virtualization

5. Resource Management in Xen • Each domain including host and guests consume three types of resource: • CPU • Memory • I/O

6. CPU Resource Management • The main target of this project • Each guest domain can be assigned one or more VCPUs (Virtual CPUs) to be realized by physical CPU (or CPUs in SMP systems) • This realization process is called CPU scheduling • Xen has built-in schedulers

7. Memory Resource Management • Each guest domain can be assigned a predetermined amount of memory • With proper monitoring, memory utilization can be determined • The goal of memory resource management is to warrant each VM has enough memory without paging mechanism, and no VM wastes memory • Domain memory can be dynamically adjusted at run time with ‘xm mem-set’

8. I/O Resource Management • Current I/O model in Xen is done through drivers in Dom0 • Dom0’s memory assignment is critical for I/O performance (Source: Cherkasova et al., Comparison Of the Three CPU Schedulers in Xen, Sigmetrics Perf Eval Review)

9. Terminology for CPU Scheduling • Terms have been extended from the domain of process scheduling to the domain of CPU scheduling • Proportional Share/Fair Share • Work Conserving/Non Work Conserving • Preemptive/Non Preemptive

10. Proportional Share (PS) • PS scheduling allocates CPU to VMs according to their shares (weights) • PS scheduler provides instantaneous form of sharing • Fair Share (FS) schedulers tend to provide a time-averaged form of proportional share based on the actual use over long time periods

11. Work Conserving (WC) • In the WC-mode, the shares are merely guarantees • In a case of two VMs with equal shares, if one is blocked, the other one can consume the entire CPU • In a Non-WC mode, the shares are caps • In a case of two VMs with equal shares, if one has consumed 50% of a CPU, it can not consume any more of the CPU • Bad-behaved domains can be isolated

12. Preemptive • Preemptive schedulers rerun the scheduling decision when a new guest becomes ready • The new guest may preempt the running guest if it has a higher priority • Non-Preemptive schedulers make decisions when the running guest gives up CPU

13. CPU Schedulers of Xen • Three built-in schedulers • BVT (Borrowed Virtual Time) • SEDF (Simple Earliest Deadline First) • Credit • The scheduler is determined when Xen boots up and can not be changed after that moment • Each scheduler takes input parameters to fine tune the performance at run time

14. BVT • Based on the concept of virtual time (VT) consumed by a guest domain (virtual machine) • The domain with a lower VT has a higher priority • PS scheduling • Work conserving • Low latency support for real time apps through warping VT

15. SEDF • Support for both WC and Non-WC mode, bad-behaved VM can be isolated when the cap is reached (Non-WC mode) • Preemptive • Real time apps support • Queue is per CPU • No global load balancing among multiple CPUs in SMP systems

16. Credit • PS schedulers • Support for WC and Non-WC mode • Global load balancing in SMP systems • Non-preemptive • The default scheduler in Xen 3.0

17. Monitoring Performance of Xen • Schedulers manage CPU resources for guest domains • Performance of Xen can be monitored by • Xen-based tools • Apps-based tools • Performance becomes a feedback to tune schedulers

18. Xen-based Monitoring • Xen offers several utilities to monitor the performance of a Xen system including Dom0 and guest domains • xenmon.py, a Python-based utility, was designed to compare Xen performance under different CPU weightings of the domains • xenstore stores configuration, event and status information about Xen domains

19. Apps-based Monitoring • VM performance can also be monitored through specific apps designated to run on the VM, e.g. • Web server • Database server • Computation server

20. Adaptive Scheduling • Idea • Use Credit scheduler to take advantages of global load balancing in SMP systems • Credit accepts two input parameters (weight and cap) for each domain to fine tune its behavior • Use proper monitoring tools to get domain performance • Performance is fed back to adjust weight and cap

21. Methodology

22. Summary (1) • In this quarter, we have done • Study on the resource management of Xen domains, particularly from the perspective of CPU resource management • Study on Xen schedulers for managing CPU resources • Study on Xen performance monitoring tools • Study on fine-tuning Xen schedulers at run time

23. Summary (2) • 系統規劃 • 使用open source軟體為開發基礎 • 使用Xen平台為server virtualization基礎 • 使用commodity level多核(multi-core)硬體為平台 • 假設雲端系統為提供高效率之計算平台，例如使用Hadoop之map-reduce，設計一個以CPU資源應用為主的自動化負載平衡機制 • 暫不考慮I/O、memory負載平衡 • 先以單機多核的負載平衡為標的再拓展到多機群集

24. Summary (3) • 需求分析 • 以CPU資源應用為主，如何在Xen的環境下管理CPU之資源應用 • Xen的內建scheduler分配實體CPU的資源給予VM之虛擬VCPU，Xen 3.0預設使用的Credit scheduler能使用WC與non-WC mode (改善了BVT)及多核的負載平衡(改善了SEDF) • Credit scheduler接受參數輸入來設定每一個domain的weight及cap值 • 預設之weight及cap值不一定對所有的domain來說都是最佳的，domain實際執行的應用會影響到效能 • 必須能偵測domain之效能以回饋到Credit scheduler來動態調整每一個domain的weight及cap值 • 使用Xen-based或Apps-based方式來偵測domain之效能

25. Outlook • Looking forward, we plan to • Xen 系統實做及資料收集. 實做 Xen 環境及上述之VM scheduler於多處理器平台, 收集效能分析資料並參考服務品質(QoS)專案之研究, 設定效能指標及環境變數.