Virtual Machine Scheduling for Parallel Soft Real-Time Applications

1. Virtual Machine Scheduling for Parallel Soft Real-Time Applications Like Zhou, Song Wu, Huahua Sun, Hai Jin, Xuanhua Shi Services Computing Technology and System Lab Cluster and Grid Computing Lab School of Computer Science and Technology Huazhong University of Science and Technology

2. Outline • Introduction • Motivation • Design • Implementation • Evaluation • Discussion and Future Work • Conclusion

3. Introduction • Many soft real-time applications use parallel programming models to utilize hardware resources better and possibly shorten response time • More and more cloud services including such parallel soft real-time applications (PSRT applications) are running in virtualized environment computer vision cloud-based live transcoding distributed real-time stream computing

4. Introduction • When running in virtualized environment, PSRT applications do not behave well and only obtain inadequate performance

5. Outline • Introduction • Motivation • Design • Implementation • Evaluation • Discussion and Future Work • Conclusion

6. Motivation soft real-time scheduling How to design and implement the parallel soft real-time scheduling algorithm which addresses soft real-time constraints and synchronization problems simultaneously? 1st 2nd co-scheduling 1st 2nd 3rd parallel soft real-time scheduling 1st 2nd 3rd

7. Outline • Introduction • Motivation • Design • Implementation • Evaluation • Discussion and Future Work • Conclusion

8. Overall Design

9. Address Soft Real-Time Constraints • How to handle soft real-time constraints of event-driven soft real-time applications? PCPU0 VCPU0 RT-VCPU0 VCPU1 VCPU2 VCPU3 receive external events PCPU0 RT-VCPU0 VCPU0 VCPU1 VCPU2 VCPU3 descheduled PCPU0 VCPU0 VCPU1 RT-VCPU0 VCPU2 VCPU3 real-time boost under over

10. Address Soft Real-Time Constraints • How to handle soft real-time constraints of time-driven soft real-time applications? VCPU0 VCPU1 VCPU2 RT-VCPU0 VCPU0 VCPU1 VCPU2 has RT-VMs no RT-VMs VCPU0 VCPU1 VCPU2 RT-VCPU0 VCPU0 VCPU1 VCPU2 RT-VCPU0 VCPU0 VCPU2 RT-VCPU0 VCPU0 VCPU1 VCPU2 VCPU0 VCPU1 VCPU1 RT-VCPU0

11. Address Soft Real-Time Constraints • How to calculate time slice? • Calculate WCSL: • L and WCSL must meet: • Calculate S:

12. Address Soft Real-Time Constraints • How to determine the approximate value of the expected latency? • We use the VoIP test of MyConnection Server (MCS) to conduct an experiment • The time slice with the value of 5ms is good enough to guarantee the quality of VoIP while minimizing the impact on other applications • The value of L is calculated as 15ms

13. Solve Synchronization Problems • How to handle synchronization problems? PCPU0 VCPU0 VCPU1 VCPU2 VCPU3 RT-VCPU0 PCPU1 soft interrupt VCPU3 RT-VCPU3 VCPU1 VCPU2 VCPU0 PCPU2 VCPU1 VCPU0 VCPU3 VCPU2 RT-VCPU2 PCPU3 RT-VCPU1 VCPU2 VCPU3 VCPU1 VCPU0 real-time boost under over

14. Solve Synchronization Problems • How to address the VCPU migration problem? PCPU0 RT-VCPU0 VCPU1 VCPU2 VCPU3 VCPU0 VCPU migration problem steal PCPU1 VCPU0 VCPU1 VCPU2 VCPU3 RT-VCPU1 real-time boost under over RT-VM RT-VCPU0 RT-VCPU1 affinity exchange CPU affinity CPU affinity 0 1 0 1 1 0 1 0

15. Parallel Soft Real-Time Scheduling Calculate time slice Schedule all runnable VCPUs of a RT-VM

16. Outline • Introduction • Motivation • Design • Implementation • Evaluation • Discussion and Future Work • Conclusion

17. Implementation • Poris: parallel soft real-time scheduler • User interface: • add a field named type to csched_dom • add a field named latency to csched_dom • add a new command xm sched-rt • Modification to the Credit scheduler (sched_poris): • add a new priority (CSCHED _PRI _TS _RT) as the real-time priority • modify event processing • modify the VCPU and PCPU operating functions • manage CPU affinity and modify csched_schedule() to co-schedule all runnable VCPUs of a RT-VM

18. Outline • Introduction • Motivation • Design • Implementation • Evaluation • Discussion and Future Work • Conclusion

19. Experiment Setup • Hardware and VM configuration • Software • Hypervisor: Xen-4.0.1 • OS: CentOS 5.5 distribution with the Linux-2.6.31.8 kernel • Interfering configuration • CPU-intensive interfering configuration: all interfering VMs run CPU-intensive workloads • mixed interfering configuration: some interfering VMs run CPU-intensive workloads, and some run I/O-intensive workloads

20. Experiments • Does Poris guarantee the QoS of VoIP applications? • Experiments with MyConnection Server • Is Poris suitable for client-side virtualization? • Experiments with Media Player • Does Poris surpass other schedulers? • Experiments with PARSEC Benchmark • What is the impact of Poris on non-real-time workloads? • Experiments with Non-real-time Workloads (Kernel compilation, Postmark, Stream benchmark)

21. MyConnection Server • Upstream jitter (a) CPU-intensive interfering configuration (b) Mixed interfering configuration

22. MyConnection Server • VoIP test results 61.34% ↓ 87.5% ↓ 56.73% ↓ 90.77% ↓

23. Media Player • Play low resolution video 71.19% ↑ (a) CPU-intensive interfering configuration 40.68% ↑ (b) Mixed interfering configuration

24. Media Player • Play high resolution video 135.94% ↑ (a) CPU-intensive interfering configuration 95.31% ↑ (b) Mixed interfering configuration

25. PARSEC Benchmark The performance of Poris is up to 44.12% better than Credit, 41.28% better than RS, and 28.02% better than PS.

26. Non-real-time workloads Kernel compilation Postmark • Because Poris promotes the priorities of RT-VCPUs temporarily and uses dynamic time slices, the interferences of Poris on non-real-time workloads are slight and acceptable. • Poris even increases the performance of some types of non-real-time workloads, such as I/O-intensive workloads. Stream benchmark

27. Outline • Introduction • Motivation • Design • Implementation • Evaluation • Discussion and Future Work • Conclusion

28. Discussion and Future Work • Determining VM type and expected latency • provide APIs to programmers • analyze runtime characteristics of applications • Many applications running in a VM • use previous techniques to identify real-time applications • Supporting multiple VMs running the same PSRT applications • co-schedule multiple VMs by analyzing the communication patterns of VMs running the same PSRT applications

29. Outline • Introduction • Motivation • Design • Implementation • Evaluation • Discussion and Future Work • Conclusion

30. Conclusion • We identify the scheduling problems in virtualized environment, and find existing CPU scheduling mechanisms do not fit for PSRT applications • We propose a novel parallel soft real-time scheduling algorithm • We implement a prototype in the Xen hypervisor based on the algorithm, named Poris • We verify the effectiveness of Poris through various applications. The experimental results show that Poris can improve the performance of PSRT applications significantly

31. Thank you！

32. System Virtualization

33. Credit Scheduler • CPU resources (or credits) are distributed to VCPUs of VMs in proportion to their weight • three kinds of VCPU priorities: boost, under, and over • VCPUs with the same priority are scheduled in FCFS manner • supports SMP platforms well