A. Adya, J. Howell, M. Theimer, W. Bolosky and J. Douceur Microsoft Research, Redmond

1. Cooperative Task Management without Manual Stack Managementor Event-driven programming is not the opposite of thread programming A. Adya, J. Howell, M. Theimer, W. Bolosky and J. Douceur Microsoft Research, Redmond Appears in USENIX ATC 2002 Presented by: Fabián E. Bustamante* * Some slides from Adya et al. presentation at Usenix ATC 2002

2. Task Management • Work of a program • A set of tasks, each encapsulating a control flow • All tasks access some common shared state • Strategies • Preemptive task management • Execution of tasks can interleave/overlap on uni/multiprocessors • Serial task management • Each task runs to completion before starting next • No conflict on shared state • Problems if you want to exploit multiprocessor parallelism

3. Cooperative Task Management • Executing task has “lock” on shared state • Concurrency considered only at well defined points, commonly when needing to wait for I/O • Task must re-validate state after resuming • May need to be done even with multi-threading, e.g., mutex released before calling high-latency opn • Allows I/O concurrency but not CPU concurrency • Need to wrap I/O calls to yield instead of blocking Task A I/O1 Task B I/O1 done I/O2

4. Stack Management • Manual Stack Management (MSM) • Common approach to achieve cooperative task mgmt • Forces programmer to abandon basic programming language features • Control flow for a single conceptual task * • Task specific state are broken across several language procedures • Automatic Stack Management (ASM) • Allows natural code structure

5. Issues we’re NOT talking about • I/O Management • Synchronous vs. asynchronous • Concurrent I/O does not affect shared state • Conflict Management • Serial task mgmt – task is an atomic operation on shared state • Preemptive task mgmt – need synchronization primitives • Pessimistic (mutexes/locks) vs. optimistic (abort/retry) • Cooperative task mgmt – large atomic operations – between yields • Data Partitioning • Monolithic vs. partitioned • Each partition independently sets task mgmt strategy

6. Stack Management automatic manual cooperative preemptive serial Task Management Contributions • Separate out concerns of task and stack mgmt • Argue for not discarding automatic stack mgmt • Allow interactions between manual and automatic stack mgmt code styles Conventional concurrent programming This paper “multi- threaded” “sweet spot” Traditional alternative “event-driven”

7. Stack Management – pros & cons • Automatic stack management (ASM) • Each complete task as a single procedure • Procedures may call functions that block on I/O • While blocked, state is kept on procedure’ program stack • Manual stack management (MSM) • Programmer must rip code for a task into event handlers that run to completion w/o blocking • To initiate I/O, event handler E1 registers a continuation w/ scheduler • Continuation bundles • State indicating where E1 left off working on the task & • Reference to another handler E2 to work once I/O is completed

8. Info* GetInfoBlocking(ID id) { Info* GetInfoBlocking(ID id) { Info *info = LookupTable(id); Info *info = LookupTable(id); return info; if (info != NULL) { } return info; } info = new Info(); DiskRead(id, info); InsertTable(id, info); return info; Class Continuation { } void (*function) (Continuation cont); void *returnValue; void *arg1, *arg2, …’ } Function call when continuation is scheduled to run Return value set by I/O operation and passed to continuation Bundled up state Automatic to Manual Stack Mgmt • From ASM in-memory to on-disk To MSM …

9. Manual Stack Mgmt (MSM) Info* GetInfoBlocking(ID id) { void GetInfo1(ID id, Cont *c) { void GetInfo1(ID id) { void GetInfo2(Frame *f) { ID id = (ID) farg1; Info *info = (Info*) farg2; Info *info = LookupTable(id); if (info != NULL) { (cfunc)(cframe, info); return; return info; InsertTable(id, info); } Cont *c =(Cont*) farg3; SchedDiskReadAndYield(id, info); Frame *f = new Frame(id, info, c); Frame *f = new Frame(id, info); return info; (cfunc)(cframe, info); InsertTable(id, info); Cont *cont = new Cont(&GetInfo2, f); SchedDiskRead(id, info, cont); return info; SchedDiskReadAndYield(id, info); } } • Stack frame manually maintained by programmer • Task yields by unrolling stack to the scheduler • Result returned to caller via a continuation function call

10. Related Work • “Event-driven” to reduce concurrency bugs (Ousterhout 96) • Cooperative task management conflated with MSM • “Event-driven” model for performance • Popular for web servers [Flash, Jaws, StagedServer, Seda] • Inter-stage: each task reads as in MSM • Equivalence of “procedure-oriented” and“message-oriented” systems [Lauer & Needham 79] • Different equivalence than ASM and MSM • Cooperative task management not considered

11. F1() I/O F2() MSM: Poor Software Structure Function scoping: 2+ language function for one conceptual ASM Style MSM Style F() I/O done

12. I/O I/O MSM: Poor Software Structure Control structures: Every basic block reachable from a continuation must be a separate function e.g., while loops ASM Style F() F1() F2() F3() MSM Style

13. F1() I/O F2() MSM: Poor Software Structure Automatic variables gone - stack frames on the heap ASM Style MSM Style Heap F() a = 17 Use a Frame I/O a: 17 … I/O done I/O done

14. K1() H1() F1() F2() MSM: Poor Software Structure Loss of debugging stack: • Debugger not aware of stack frames on the heap • Some frames optimized way for convenience MSM Style F1’s cont frame I/O H1’s cont frame I/O done K1’s cont frame Heap

15. F1() H1() Software Evolution F() Proc Call GetInfo1() H() Sched I/O GetInfo() Yield control I/O done Schedule I/O GetInfo2() I/O done H2() Proc Return F2() ASM Style MSM Style Stack Ripping:Software evolution exacerbates MSM code structure problems

16. F1() H1() Detecting I/O Yields with MSM F() Proc Call GetInfo1(…, Cont * …) H() Sched I/O I/O done GetInfo(…) GetInfo2() H2() Proc Return F2() Signature change guarantees programmer aware of yielding

17. Detecting I/O Yields with ASM F() H() Schedule I/O GetInfo() Yield control I/O done • Must verify changed semantics but no stack ripping • Static check allows same benefits as MSM

18. MSM code calls ASM code F1() Does not expect continuation Expects to call GetInfo(id, contF2) and unroll stack GetInfo(Id id) Schedule I/O Yield control Process I/O Expects to be scheduled when GetInfo done I/O done Expects to return Info F2(Frame *f) Extract Info from f ASM Code MSM Code One stack (fiber) for MSM code One stack (fiber) per task for ASM code

19. MSM code calls ASM code F1() M2A-Adapter() FiberStart(Id …) Start new fiber GetInfo(Id …) Schedule I/O Yield Process I/O F2 scheduled I/O done F2(Frame *f) Returns Info Schedules F2 with Info Extract Info from f MSM (MainFiber) ASM (Fiber 1) One code style unaware of other style

20. Stack Management automatic manual cooperative preemptive serial Task Management Conclusions • Separate concerns of task & stack mgmt • MSM leads to poor code structure • Code evolution exacerbates problem • ASM: natural code structure • MSM and ASM code can co-exist • For legacy code or != programmer preferences “Multi- threaded” “Coroutines” “Event-driven”