Priority Inversion Problem in Windows WRK

1. Priority Inversion Problem in Windows WRK Practical session #3 バリ　ゲローフィ Advanced operating systems course, The University of Tokyo

2. Outline • Note: git issue from the previous class • Motivation: Mars mission • The Priority Inversion problem • Dispatcher Objects • Windows user-space synchronization • Assignment #2 – Priority inversion • Windows synch kernel structures • Assignment #3 - Priority inheritance • Deadlines

3. Git issue • Some people reported that they cannot commit because git runs out of memory • Please check if your disk image is in your repository and move it somewhere else if it is!

4. Priority Inversion – Motivation(Present in Windows WRK kernel) • Mars rover failed in 1997: • High priority device driver • Low priority user-land process • Synchronized in kernel (high prio waits for low) • Middle priority thread kicks in, leaving no space for the high priority one • (Demonstration)

5. Priority Inversion problem priority m.lock() -> wait high priority thread waiting for a lower one, without explicit synchronization preempt -> wait m.lock() -> grab time

6. Dispatcher objects – the basis of multithread synchronization • Dispatcher object: • An object for which a Thread may wait • Embeds a WaitList (list of Threads) • Can be in signaled or non-signaled state • Signaled: the event that wakes up waiting thread(s) occurred • The definition of signaled state varies based on the particular dispatcher object

7. Dispatcher objects System events and resultingstate change Dispatcher object Effect of signaled stateon waiting threads Owning thread releases mutex Mutex (kernel mode) Kernel resumes one waiting thread nonsignaled signaled Resumed thread acquires mutex Owning thread or otherthread releases mutex Mutex (exported to user mode) Kernel resumes one waiting thread nonsignaled signaled Resumed thread acquires mutex One thread releases thesemaphore, freeing a resource Semaphore Kernel resumes one or more waiting threads nonsignaled signaled A thread acquires the semaphore.More resources are not available Note: Mutant objects (i.e., Mutex, Semaphore) are initially signaled! (base\ntos\ke\mutntobj.c)

8. Dispatcher objects (cont.) Dispatcher object System events and resultingstate change Effect of signaled stateon waiting threads A thread sets the event Event Kernel resumes one or more waiting threads nonsignaled signaled Kernel resumes one or more threads Dedicated thread sets oneevent in the event pair Event pair Kernel resumes waitingdedicated thread nonsignaled signaled Kernel resumes theother dedicated thread Timer expires Timer Kernel resumes all waiting threads nonsignaled signaled A thread (re) initializes the timer

9. Dispatcher objects (cont.) Dispatcher object System events and resultingstate change Effect of signaled stateon waiting threads IO operation completes File Kernel resumes waitingdedicated thread nonsignaled signaled Thread initiates wait on an IO port Process terminates Process Kernel resumes all waiting threads nonsignaled signaled A process reinitializesthe process object Thread terminates Thread Kernel resumes all waiting threads nonsignaled signaled A thread reinitializesthe thread object

10. Windows synch. primitives – thread creation / priorities • WinAPI: • CreateThread • Returns HANDLER • SetThreadPriority • HANDLE, prio • MFC: • AfxBeginThread • Returns CWinThread • SetThreadPriority • Method of CWinThread

11. Windows synch. primitives – events, mutexes • WinAPI: • CreateEvent • Returns HANDLER • CreateMutex • Returns HANDLER • MFC: • CEvent class • Returns CEvent obj. • m_hObject (HANDLER) • CMutex class: • Returns Cmutex obj.

12. Windows synch. primitives – waiting and signaling • WinAPI: • WaitForSingleObject • expects HANDLER • WaitForMultipleObject • expects HANDLER • SetEvent(), ResetEvent() (for Event objects) • expects HANDLER • ReleaseMutex() (for mutexes) • MFC: • CSingleLock / CMultiLock • Constructor expects a synch object(s) • Lock() / Unlock() • CEvent: • SetEvent() ResetEvent() methods

13. Assignment #2 • Write a user-space program that triggers the priority inversion problem • Use three threads, with priority parameters for SetThreadPriority(): • THREAD_PRIORITY_LOWEST (-2) • THREAD_PRIORITY_BELOW_NORMAL (-1) • THREAD_PRIORITY_HIGHEST (2)

14. Assignment #2 - Hints • You have to ensure that the low prio thread gets the mutex first • The middle prio thread should start (resume?) when the high prio already waits • Make use of event objects!

15. How to overcome the priority inversion issue? • Priority inheritance • An extension to the Mutex structure • Lifts priority of the thread that owns the mutex if another thread with higher priority has to wait • How does it work?

16. Priority Inheritance in action priority m.lock() -> wait m.unlock() Priority boost Priority set back m.lock() -> grab time

17. Windows kernel synchronization implementation – a place to start • DISPATCHER_HEADER is included in Mutexes, (struct _KMUTANT) • You can modify KMUTANT! • What should be stored there? • Where do we need to deal with priorities? Lock/Unlock? Nested priorities? • (base\ntos\inc\ke.h)

18. Represent a thread’s reference to something it’s waiting for (one per handle passed to WaitFor…) All wait blocks from a given wait call are chained to the waiting thread Type indicates wait for “any” or “all” Key denotes argument list position for WaitForMultipleObjects WaitBlockList List entry Thread Object Key Type Next link Size Type State Wait listhead List entry List entry Thread Thread Object Object Key Type Key Type Next link Next link Thread Objects Wait Blocks: WaitBlockList Dispatcher Objects Wait blocks Size Type State Wait listhead Object-type-specific data Object-type-specific data

19. Assignment #3 - Hints • What needs to be done: • A new system call that receives a Mutex HANDLER • Use number: 12E • Kernel modifications for: • Mutex obj, • Lock() • Unlock() operations…

20. Report / Assignment notes • Deadline (for all assignments): • End of January 2010 ! •  Submission Place: • Office of Computer Science DepartmentScience Bldg #7 1F • Report: • Design • Code • Why that way? • What other ways might be possible?