John Regehr March 20, 2001

1. Using Hierarchical Scheduling to Support Soft Real-Time Applications in General-Purpose Operating Systems John Regehr March 20, 2001

2. Outline • Motivation and Approach • Guarantees • HLS Design • Augmented Reservations • Conclusions and Demo

3. Overview of Contributions • A general hierarchy of soft real-time schedulers can provide guaranteed scheduling behavior • The Hierarchical Loadable Scheduler (HLS) architecture… • Can be efficiently implemented in a general-purpose OS • Increases application performance compared to case where scheduler and apps are mismatched • Augmented CPU reservations increase predictability when the OS steals CPU time from real-time applications

4. Motivation • People use general-purpose OSs (GPOSs) for many kinds of tasks • e.g. Unix, Windows, MacOS variants • Compatibility, commodity, convenience • Applications have diverse scheduling requirements • Time-sharing • Soft real-time • Isolation • Co-scheduling between processors or machines

5. The Problem • One size does not fit all • Supporting evidence: companies sell scheduler extensions • Ensim: ServerXchange • Sun: Solaris Resource Manager • Aurema: Active Resource Management • TimeSys: Linux/RT

6. Motivating Data: Unfair CPU Allocation Between Users • User 1 gets little CPU time when User 2 creates many threads

7. Approach • Allow a hierarchy of CPU schedulers to control processor allocation • Thesis Statement: • It is useful and feasible to extend a GPOS with a general, heterogeneous scheduling hierarchy. • A heterogeneous hierarchy employs different schedulers • A general hierarchy does not impose a fixed scheduling model

8. FP RES SFQ Example Hierarchy H L J Video Player Word Processor Voice Recognition

9. Time Scales • Long: • System is used in a particular way • Duration: usually uptime of a system or longer • Medium: • Applications start, end, and change requirements • Duration: seconds, minutes, or longer • Short: • Individual scheduling decisions made by schedulers within the hierarchy • Duration: milliseconds or 10s of ms

10. What Microsoft Should Do • Put HLS into consumer Windows! • By default • Support interactive, batch, and multimedia applications for a single user • However, also include • Library of useful schedulers and API for composing them • API for implementing new schedulers

11. Reminder • HLS is an architecture for flexibly composing schedulers

12. Outline • Motivation and Approach • Guarantees • HLS Design • Augmented Reservations • Conclusions and Demo

13. Guarantee • Definition: • Ongoing lower (and possibly upper) bound on CPU allocation • Goals: • Describe the scheduling behavior required and provided by schedulers, and required by applications • Permit these behaviors to be reasoned about • Syntax: • TYPE p1 p2 …

14. Example Guarantees • 100% of a CPU: • ALL • Strictly best-effort scheduling: • NULL • Proportional share: • PS s • PSBE sδ • CPU Reservations: • RESBS x y, RESBH x y • RESCS x y, RESCH x y

15. CPU Reservation Guarantees • Hard / Soft: • “Hard CPU reservation” ≠ hard real-time • Soft reservations guarantee a lower bound • Hard reservations also guarantee an upper bound • Basic / Continuous:

16. Guarantee Conversion by Schedulers • Schedulers require and provide guarantees • SFQ: PSBE → PSBE+ • Rez: ALL → RESBH+ • Schedulers determine if specific guarantees can be provided • ALL → RESBH 5 10, RESBH 25 100 • EDF-based reservation scheduler • Naïve rate monotonic reservation scheduler

17. Selected Conversions by Schedulers • Full table contains 23 schedulers

18. Guarantee Conversion by Rewrite Rules • Guarantees can be converted without a scheduler, using rewrite rules • Trivial examples: • PSBE sδ→ PS s • RESBH x y → RESBS x y • Non-trivial examples: • RESBS x y → RESCS x (2y-x+c) for any c ≥ 0 • RESCS x y → PSBE (x/y) x/y (y-x)

19. Partial Rewrite Rule Table T = rewrite rule exists F = rewrite rule does not exist

20. Another View of Some Rewrite Rules RESBH RESBS PSBE RESCH RESCS PS

21. FP RES SFQ Guarantees in Action ALL ALL NULL * RESBH 5 33 * RESBH 10 20 J RESBS 10 20 → Video Player PSBE 0.5 10 PSBE 0.3 35 PSBE 0.1 15 Word Processor Voice Recognition

22. Related Work for Hierarchical Guarantees • Hierarchical start-time fair queuing [Goyal et al. 96] • Uniformly slower processors • Open environment for real-time applications [Deng et al. 99] • BSS-I and PShED [Lipari et al. 00]

23. Guarantee Contributions • Created a framework for reasoning about composition of schedulers • Derived rewrite rules • Integrated more than 20 schedulers • Guarantees provide a model of soft real-time CPU allocation • Independent of particular scheduling algorithms • Developers can program to • Users can ensure that application requirements are met

24. Outline • Motivation and Approach • Guarantees • HLS Design • Augmented Reservations • Conclusions and Demo

25. Inside a Non-Hierarchical CPU Scheduler CPU • Scheduling decisions are made in response to timers and thread state changes SCHEDULER W W R W T1 T2 T3 T4 T5

26. Inside a Hierarchical CPU Scheduler parent virtual processor VP1 • Distinguishing feature of hierarchical schedulers: revocation SCHEDULER R W W R R W child virtual processors VP2 VP3 VP4 VP5 VP6

27. Hierarchical Scheduler Infrastructure (HSI) Design • Schedulers are implemented by code libraries loaded into the kernel • Scheduler instances are active entities in the hierarchy • Virtual Processors • Represent potential for physical processor allocation • Used to notify schedulers

28. Scheduler Notifications • Hierarchical infrastructure notifies a scheduler whenever: • A child VP requests or releases a processor • A parent VP grants or revokes a processor • A timer expires • Threads implicitly notify schedulers when they block and unblock • Notifications are cheap: virtual function calls

29. HSI and Scheduler Implementation • HSI runs in Windows 2000 kernel • Serialized by a spinlock • Added ~3100 lines of code • Loadable schedulers: • Time sharing / fixed priority, CPU reservation, proportional share, join • A representative set of schedulers, but not a complete one • Implemented Rez in about two days, PS scheduler in a few hours

30. Performance • Test machine is a 500MHz Pentium III • Most mode change operations run in less than 40μs • Create / destroy scheduler instance, begin / end CPU reservation, etc. • Median context switch time • Unmodified Windows 2000: 7.1μs • HLS time-sharing scheduler: 11.7μs • Cost of each extra level in the scheduling hierarchy is 0.96μs • Many opportunities for optimization

31. Performance Data:Isolating Resource Principals

32. Scheduling a Real-Time, CPU-Bound Application • Synthetic test application • Represents a virtual environment or game • Must provide 1 frame per 33ms (~30 FPS) • 10ms of CPU time for each frame • More frames are acceptable and desirable • Machine also runs background work • Goals • Application should never miss a deadline • Non-real-time work should make progress

33. Performance Data: CPU Bound Real-Time Application • 30-second runs

34. Related Work for HSI • Scheduler activations • [Anderson et al. 91] • CPU inheritance scheduling • [Ford and Susarla 96] • Vassal • [Candea and Jones 98]

35. Contributions • Virtual processors are a novel extension of scheduler activations [Anderson et al. 91] • Permit a wide range of schedulers to dynamically create a scheduling hierarchy in kernel of a GPOS • First system to do this • Simplifies scheduling programming model by hiding OS and hardware complexities

36. Outline • Motivation and Approach • Guarantees • HLS Design • Augmented Reservations • Conclusions and Demo

37. Problem: Stolen Time • We have assumed until now that the OS does not interfere with guarantees • However, while processing asynchronous data the OS may steal CPU time from applications • Time used by bottom-half mechanisms is not accounted for correctly • A solution: • Augmented CPU reservations

38. Time Stolen by Network Receive Processing

39. Augmented Reservations • Strategy: accurately measure stolen time in order to compensate threads • Rez-C: avoid “charging” threads for stolen time • Rez-FB: use a feedback loop to assign a larger CPU reservation so requirements are met even when time is stolen • Implemented as HLS schedulers

40. Augmented Reservation Performance

41. Related Work for Augmented Reservations • Scheduling bottom-half activity • Mach [Rashid et al. 89] • Nemesis [Leslie et al. 96] • Scheduling bottom-half activity in a GPOS • FreeBSD [Jeffay et al. 98] • Feedback-based scheduling • FC-EDF [Lu et al. 99] • Moving code into scheduled contexts • Soft modems [Jones and Saroiu 01]

42. Augmented Reservation Contributions • Rez-C and Rez-FB • 6% over-reservation to eliminate most deadline misses • vs. 24% over-reservation for plain Rez • Quantified severity of stolen time • Windows 2000 + Rez and Linux/RT • Network, disk, software modem, USB

43. Outline • Motivation and Approach • Guarantees • HLS Design • Augmented Reservations • Conclusions and Demo

44. Conclusions • HLS is feasible: • Composition of soft real-time schedulers can be reasoned about • Implemented and performs well • HLS is useful: • Permits scheduling behavior to be tailored to specific needs • New schedulers can be implemented more easily

45. FP H L L TS PS RES HLS Demonstration • All threads scheduled by default time-sharing scheduler • Create a CPU reservation • Make proportional share scheduler the default and move time-sharing threads • End CPU reservation T T TTTTTT TTTTTT TTTTTT TTTTTT TTTTTT All threads scheduled by default time-sharing scheduler Create a CPU reservation Make proportional share scheduler the default and move time-sharing threads End CPU reservation All threads scheduled by default time-sharing scheduler Create a CPU reservation Make proportional share scheduler the default and move time-sharing threads End CPU reservation All threads scheduled by default time-sharing scheduler All threads scheduled by default time-sharing scheduler Create a CPU reservation Make proportional share scheduler the default and move time-sharing threads End CPU reservation

46. The End • Papers and more info at: http://www.cs.virginia.edu/~jdr8d