Cooperative Task Management without Manual Stack Management

1. Cooperative Task Managementwithout Manual Stack Management Presentation by David Florey CS533 - Concepts of Operating Systems

2. Overview • Paper serves to solve two main goals: • Introduce the two models and the baggage that they carry: • Event-Driven Programming is cooperative concurrency, but you have to use manual stack management • Multi-threaded Programming uses automatic stack management, but it doesn’t lend it self to a simple concurrency style • There is a sweet spot (event-driven/automatic stacks) • Show how code written in disparate styles can work with each other through adapters • Essentially neither side of the debate (between which coding style should prevail) would give and so we get the sweet spot • The Problem • Definition of Terms • The Solution • The Adapters (making the code work together) CS533 - Concepts of Operating Systems

3. The Problem: I want the Best of Both Worlds! • Event-Driven programming • Easy to use when reasoning about concurrency • Code considered is difficult to read • Code considered is difficult to maintain • Good for single-processor (cooperative concurrency) • Multi-Threaded Programming • Code is more readable • Code is easier to maintain • But it is difficult to reason about concurrency with threads • This type of programming typically involves the need for synchronization • Good for multi-processors (true concurrency) CS533 - Concepts of Operating Systems

4. Definition of Terms • Task Management Models • Stack Management Models • I/O Models • Conflict Management Models • Data Partitioning CS533 - Concepts of Operating Systems

5. Cooperative Task Management • Event-Driven: • Typically Single Threaded • Consists of an event loop and queue • Each event handler runs to completion and is not pre-empted (at least not to its knowledge) • If an event handler needs to access I/O: • It can set up notification with the I/O (asynchronous) • Unwinds its stack • Gives up control to the event loop • Provide a continuation during notification setup so that the “conceptual” task can continue • Rips stack, forcing manual stack (state) management • This is typically more natural due to the single-threaded nature of events • It can block (synchronous) • Holding up the entire event-driven thread • Stack and therefore state is maintained, but it can be argued that this is no longer cooperative! CS533 - Concepts of Operating Systems

6. Preemptive Task Management • Multi-threaded • Uses multiple threads to get work done • Uses synchronization primitives to share state • Good for multi-processor programming • If a thread must access I/O: • It can set up notification with the I/O (asynchronous): • Tells I/O which procedure to call • Thread terminates • New thread of execution started with notification from I/O • Less natural due to the nature of multiple threads • Stack is ripped here too! • Still cooperative task management, but… • It can block on the I/O (synchronous): • Thread stops execution until I/O returns • More natural for multiple threads – when thread blocks, another thread may be scheduled • State maintained in stack, since thread never terminates – it can pick up where it left off (assuming the invariant) CS533 - Concepts of Operating Systems

7. Manual Stack Management • Inherent in Cooperative Task Management (Event-Driven Programming) • Typical model includes: • Tear conceptual task “A” into actual task A1 and A2 • A1: Build up state • A1: Set up continuation information for I/O call • A1: Unwind stack and terminate • I/O calls A2: • A2: rebuilds state and completes conceptual task “A” • Results in stack ripping • If task “A” was in a single function, it would have only one stack and we would not need to maintain the stack ourselves across the I/O call • Even bigger problem in a loop CS533 - Concepts of Operating Systems

8. Automatic Stack Management • Inherent in Pre-emptive Task Management • Typical model includes: • Start a thread to handle conceptual task “A” • Thread “A” builds up state, calls I/O – perhaps blocks on I/O and then completes the job of conceptual task “A” • No need to rip stack in two (or more pieces) CS533 - Concepts of Operating Systems

9. Automatic Stack Management • But there is a down side: • Hidden concurrency assumptions • Yields in manual stack management are clearly defined • This is not true in automatic stack management (if local state depends on shared state, state may need re-evaluation after the call which forced this thread to yield) • Solution • Static check (declarative programming): • Code marked Atomic may not call code marked Yeilding • Dynamic check (function calls marking beginning and end): • Code contained within the calls startAtomic()…endAtomic() cannot call code that blocks on a yield() call. CS533 - Concepts of Operating Systems

10. Other Management Models • Conflict Management • Concerned with avoiding resource conflicts • Serial tasks are fully atomic – no resource conflicts • Cooperative tasks assume code in between yield points are fully atomic • Pre-emptive tasks must use synchronization primitives to ensure the safe access of shared data – these primitives are the only guaranteed atomic functions • Data Partitioning • The more you break up your shared state, the fewer conflicts you should have CS533 - Concepts of Operating Systems

11. Review • Cooperative tasks are the most desirable when reasoning about concurrency • usually associated with event-driven programming • Automatic stack management is the most desirable when reading/maintaining code • Usually associated with threaded or serial programming • Use dynamic checking to remove hidden concurrency assumptions • Therefore the advantage of manual stack management is absorbed by automatic stack management CS533 - Concepts of Operating Systems

12. Solution • Build a system that uses Cooperative Task Management without Manual Stack Management • Pulls together the best of both worlds • Ease of reasoning about concurrency • Readable/maintainable code CS533 - Concepts of Operating Systems

13. Implementation DetailsHow do we get that sweet spot? • Use fibers • User level thread package • Many-to-many threading solution for Windows Platform • Non-preemptive • Scheduled cooperatively – one fiber hands control off to another fiber • Fibers run in the context of a thread • System can still preempt that thread • Solution includes use of two fibers • MainFiber: schedules manual and automatic tasks • Potentially blocking automatic tasks always run on secondary fiber so that MainFiber can continue to schedule tasks CS533 - Concepts of Operating Systems

14. Quick Review • Cooperative Task Management with Manual Stack Management • Tx is ripped into T1 and T2 • T1 saves state in continuation along with which method (or event) to call: T2 • Control is yielded back to Event Driver • I/O calls T2 event, event driver passes control to conceptual Task Tx: actual task T2 • T2 completes task • Single Threaded, Event-Driven, Cooperative CS533 - Concepts of Operating Systems

15. Goal • Cooperative Task Management with Automatic Stack Management • Scheduler calls task Tx (which is expecting to block) on Secondary Fiber • Main Fiber hands control over to secondary fiber • Tx calls I/O (presumably through some manual mechanism) and blocks by yielding control back to Main Fiber • Scheduler on main fiber is free to execute compute only tasks: Ty, Tz • I/O returns, scheduler switches to secondary fiber to complete Tx • Tx’s state remains with it throughout the call (automatic stack management) • Tasks are allowed to execute cooperatively • Note all I/O must be Asynchronous to work in this model! CS533 - Concepts of Operating Systems

16. What does this give us? • By executing an “automatic” function on a secondary fiber, we can return to the main fiber to continue working • No blocking • No stack ripping • All I/O calls must be asynchronous now • We need to be able to schedule the I/O request and then let the MainFiber continue CS533 - Concepts of Operating Systems

17. The Devil’s in the Details • To make this work, we need adapters • Manual Stack Code calling Automatic Stack Code • Automatic Stack Code calling Manual Stack Code • Adapters are code generated to reduce complexity for the common developer CS533 - Concepts of Operating Systems

18. Manual Calling Automatic • Uses a Continuation to Fiber Adapter: • Glues the notion of a continuation to the notion of a blocking call • Handles the details of the continuation: rips itself into two, schedules the second half within the first and calls the blocking function on the second fiber • Second half of CFA uses continuation to call the original task’s continuation CS533 - Concepts of Operating Systems

19. Manual Calling Automatic Part 1:Initiating the Task CS533 - Concepts of Operating Systems

20. Manual Calling Automatic Part 2:Completing the Task CS533 - Concepts of Operating Systems

21. Automatic Calling Manual • Uses a Fiber to Continuation Adapter • FCA executes on secondary thread • May short-circuit if no blocking occurred • Glues the notion of a blocking call to one that uses a continuation • Handles the details of the transformation: creates a new “fiberContinuation” which causes MainFiber to switch back to the secondary fiber, executes the manual stack function and switches back to the main thread • FiberContinuation simply resumes the suspended fiber CS533 - Concepts of Operating Systems

22. Automatic Calling Manual CS533 - Concepts of Operating Systems

23. Conclusion • Gives the best of both worlds • Automatic Stack Management with Event-Driven Concurrency • Makes the code easier to read and understand while allowing one to reason over concurrency concepts with ease • The former solved with automatic stack management • The latter solved with Cooperative Task Management • Shows that the concepts behind event driven programming are not the opposite of those behind threaded programming • Finally, once we accept this concept, we can evolve our code with either style and use adaptors to tie the two together CS533 - Concepts of Operating Systems