Scheduler Activations

1. Scheduler Activations Effective Kernel Support for the User-Level Management of Parallelism

2. Contents • Problem Statement • New Approach • Scheduler Activation • User-Level Threads Package • Explicit Vectoring of Kernel Events to the User-Level Threads Scheduler • Critical Section • Performance • Summary

3. Problem Statement • The performance of kernel threads are inherently worse than that of user-level threads. • The program must cross extra protection boundary on kernel thread operation • With kernel threads management, a single implementation is imposed to all applications • With user-level threads management, application specific implementation can be easily done

4. TEST benchmark: Null Fork (the overhead of forking a thread) and Signal-Wait (the time for signal a waiting process and wait on a condition)

5. Why not use the user-level threads for the entire program?

6. What if a user-level thread makes a blocking IO call or has page faults?

7. What if the kernel preempts a process running a user level thread? • The user level system cannot make scheduling decision

8. Effective user-level threads package that achieve the same functionality as kernel threads is on demand • To integrate user-level threads with other system services, kernel level support is required

9. Approach • Provide each application with a virtual multiprocessor • Each application knows how many processors are allocated to it • Each application has complete control over which threads run on those processors • The operating system has complete control over processor allocation among address space

10. The kernel notifies the user-level system of every kernel event that affects address space • The user-level system notifies the kernel any events related to processor allocation

11. Scheduler Activations • The kernel mechanism used to implement this approach • The execution context for vectoring control from kernel to user-level threads system on kernel event • Notifies user-level threads system of kernel event by invoking a procedure in the user level threads library (upcall) • The user-level system uses scheduler activation to handle user-level event such as execution of user-level threads and making requests of kernel • Provides space in kernel to save a processor context

12. Explicit Vectoring of Kernel Events to the User-Level Thread Scheduler • The communication between the kernel processor allocator and the user-level threads system is made in terms of scehduler • Each scheduler activation has two execution stacks • One maps into the kernel • One maps into application address space • When a program starts, the kernel creates a scheduler activation and assigns it to a processor

13. The kernel upcalls into the application at fixed entry point

15. What is user-level threads package? • The thread package view each process as a virtual processor • The virtual processors are multiplexed across the physical processors by the kernel • User- level code in each virtual processor pulls threads off the ready list and runs them

16. User- level code in each virtual processor pulls threads off the ready list and runs them

17. Multiprocessor operating systems such as Mach, Topaz, and V provide kernel support for multiple threads per address space • No processors are idle in the presence of ready list, if there is work to do • No high-priority threads wait while a low-priority thread runs • When a thread blocks, the processor the thread had been running on can be used to run another process

18. What if an activation’s user-level thread is stopped by kernel? • Once a user-level thread is stopped by kernel, the thread is not resumed by the kernel • A new scheduler activation is created • The kernel notifies the user system that the thread is blocked • The user-level system removes the state of the thread from the old activation and notifies the kernel • The user-level system decides which thread to run on the processor

19. Traditional kernel threads • When the kernel stops a user-level thread, it never notifies the user-level of the event • The kernel resumes the thread

23. Critical Section • FastThreads uses unlocked per-processor free lists of thread control block • Access to the free lists must be done atomically • Adopt a solution based on recovery • When a user-level thread is blocked, the user-level system checks whether it was executing in critical section • If so, the thread is continued via user-level context switch • When the thread exists, it hands the control over to the upcall via user context switch • The kernel places the thread onto the ready list

24. Implementation • A prototype is built on the DES SRC FireFly multiprocessor workstation • Moderate modification was applied to the kernel threads of Topaz, the operating system of FireFly • Modified Topaz to perform upcalls listed in Table II • Straightforward modification was applied to FastThreads, the user-level system • Default policy in FastThreads • Uses per-processor ready list

25. Last-in-first-out to improve cache locality

26. Performance enhancement • Discarded activation can be cached for future use • Discarded scheduler activations can be collected and returned to the kernel in bulk

29. Summary • Processor allocation must be done by the kernel • Thread scheduling must be done by each address space • The kernel notifies the user-level thread scheduler of all the events that affect address space • The user-level space notifies the kernel all the events that may affect the processor allocation decision