1 / 24

Scheduling I/O in Virtual Machine Monitors

Diego Ongaro , Alan L. Cox, and Scott Rixner Rice University March 5, 2008. Scheduling I/O in Virtual Machine Monitors. Virtualized Host. Guest Domain 1. Guest Domain 2. Guest Domain 3. Guest Domain 4. . . . Server Consolidation. Streaming Media Server.

tahir
Download Presentation

Scheduling I/O in Virtual Machine Monitors

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Diego Ongaro, Alan L. Cox, and Scott Rixner Rice University March 5, 2008 Scheduling I/O in Virtual Machine Monitors Scheduling I/O in Virtual Machine Monitors

  2. Scheduling I/O in Virtual Machine Monitors Virtualized Host Guest Domain 1 Guest Domain 2 Guest Domain 3 Guest Domain 4 . . . Server Consolidation Streaming Media Server • Virtual machine monitors enable server consolidation • Consolidated domains have wildly different workloads • Do current solutions actually achieve fairness? Voice-over-IP Server consolidation Application Server Web Server Database Server . . .

  3. Scheduling I/O in Virtual Machine Monitors Xen Performance Ping additional idle guest (G8): Ping latency to G8: 259.2ms Ping latency to G8: 6.6ms

  4. Xen Architecture Guest Domain User Software Driver Domain Xen Control Software Guest Domain User Software Guest Domain User Software • Hypervisor – thin virtualization layer • Driver Domain – hardware control • Guest domains given restricted access Xen Hypervisor Hardware Scheduling I/O in Virtual Machine Monitors

  5. Scheduling I/O in Virtual Machine Monitors time Receiving a Network Packet Guest Domain User Software Driver Domain Xen Control Software Guest Domain User Software Guest Domain User Software Xen Hypervisor Hardware VirtualInterrupt VirtualInterrupt HardwareInterrupt Scheduler Scheduler Driver Domain Packet Arrives Target Domain

  6. Scheduling I/O in Virtual Machine Monitors Scheduling Delays • Scheduler has a large effect on latency • As packets arrive, two opportunities to run any number of domains, delaying I/O • Delay proportional to the number of runnable domains time Scheduler Scheduler Driver Domain Packet Arrives Target Domain Other Domains Other Domains Virtual Interrupt Virtual Interrupt Hardware Interrupt

  7. Scheduling I/O in Virtual Machine Monitors Scheduler Fairness • Virtual machine monitors must fairly support both computation and I/O domains • Xen supports computation domains well, but has mixed results with I/O domains • The scheduler plays a significant role in I/O performance • Improvements for I/O are not obvious • Some of Xen’s current scheduler optimizations for I/O are beneficial, others are not

  8. Scheduling I/O in Virtual Machine Monitors Outline • Xen's Credit scheduler • Scheduler “optimizations” for I/O • Conclusions

  9. Scheduling I/O in Virtual Machine Monitors Xen’s Credit Scheduler • Credits are assigned to each domain • Approximate the fraction of processor resources each domain will receive • Do not indicate when each domain will receive its fraction • Scheduler increments/decrements credits • Periodically deducts credits from running domain • Adds credits when majority of credits in the system have been consumed

  10. Scheduling I/O in Virtual Machine Monitors U4 U3 U2 Scheduler Operation • Domain states • Under – domain has credits remaining • Over – domain is over its credit allowance • Domains are run in FIFO order by state • Over domains only run if no under domains • A domain may run for up to 30ms if it has enough credits • After running, return to the run queue by state • Behind all other domains in the same state • Regardless of runtime or remaining credits Run Queue Next domain to run O2 O1 U1 This approach is biased against I/O

  11. Scheduling I/O in Virtual Machine Monitors Scheduler “Optimizations” for I/O • I/O Domain Requirements • Low latency • High bandwidth • Independent of other domains’ workloads • “Optimizations” • Boosting idle domains • Included in Xen • Ordering the run queue by credits • Proposed for Xen • Tickling the scheduler • Included in Xen

  12. Scheduling I/O in Virtual Machine Monitors O2 O2 O1 O1 U4 U4 U3 U3 U2 U2 U1 U1 Boosting Idle Domains • Initially, scheduler had no special features for I/O domains • Additional scheduling state: boost • Higher priority than under • Used when an idle domain is sent a virtual interrupt • With boost Next domain to run U5 I1 Next domain to run B1 I1

  13. Scheduling I/O in Virtual Machine Monitors Impact of Boost • Boost improves latency for I/O domains

  14. Scheduling I/O in Virtual Machine Monitors O2 O2 O1 O1 U4 B4 B3 U3 U2 B2 B1 U1 Next domain to run vs. Caveats with Boost • Ineffective for I/O domains under heavy loads • Must be idle to receive boost • Must block quickly to sustain boost • Effect is negated with many I/O domains

  15. Scheduling I/O in Virtual Machine Monitors Ordering the Run Queue • I/O domains tend to quickly block • Within each state, sorts domains by credits remaining • Short-running I/O domains are re-inserted near the head of the run queue

  16. Scheduling I/O in Virtual Machine Monitors Impact of Ordering the Run Queue • Ordering the run queue can reduce latency for I/O domains under larger loads • Surprisingly, often complements boost

  17. Scheduling I/O in Virtual Machine Monitors Tickling the Scheduler • Xen tickles the scheduler after each virtual interrupt • Tickling can reduce latency • Often runs virtual interrupt recipient immediately time Scheduler Scheduler Driver Domain Packet Arrives Target Domain Other Domains Other Domains Virtual Interrupt Virtual Interrupt Hardware Interrupt

  18. Scheduling I/O in Virtual Machine Monitors Tickling Too Soon? • Tickling as virtual interrupts are sent may be too soon • Scheduler is uninformed about upcoming virtual interrupts • Gives arbitrary priority to first domain(s) to be sent a virtual interrupt • Don’t tickle while sending virtual interrupts • All recipient domains are then runnable • Scheduler runs the domain with the highest priority

  19. Scheduling I/O in Virtual Machine Monitors Impact of Tickling

  20. Scheduling I/O in Virtual Machine Monitors Transcending Xen • Scheduling mixed domains • Native vs. VMM scheduling

  21. Scheduling I/O in Virtual Machine Monitors Mixed Domains • Mixed I/O- and compute- domains don’t underutilize resources • VMM scheduler can not schedule only the I/O process • Low latency for a mixed domain can not be guaranteed Driver Domain Guest Domain Guest Domain Guest Domain I/O process CPU process CPU process CPU process Hypervisor Scheduler

  22. Scheduling I/O in Virtual Machine Monitors Native Driver Domain Guest Guest Guest I/O CPU process CPU CPU CPU CPU process I/O process ? CPU process Hypervisor Scheduler Scheduler Native vs. VMM Scheduling Native VMM

  23. Scheduling I/O in Virtual Machine Monitors Summary of Results • Boost reduces latency with few I/O domains • Does little harm in other cases • Ordering the run queue favors short-running I/O domains • Often complementary to boost • Tickling reduces latency in some cases • Often causes unfairness in bandwidth and latency • I/O-intensive applications need their own domains

  24. Scheduling I/O in Virtual Machine Monitors Conclusions • Scheduling has a large impact on I/O fairness • Can cause exorbitant latency • Can cause bandwidth inequity • Xen’s Credit scheduler is not fair for I/O • Does well with compute domains • No notion of timeliness, which is needed for I/O • Further work needs to be done to solve the mixed domains problem

More Related