Events Can Make Sense

1. Events Can Make Sense Maxwell Krohn, Eddie Kohler, and M. Frans Kaashoek

2. Motivation • Maintain flexibility and performance of events • Add programmability of threads • Manage closures and local variables during “stack ripping” • Manage memory efficiently

3. Stack Ripping void wait_print() { sleep(10); printf(“Done!”); }

4. void wait_print() { timer(10, WAIT_PRINT2); /* Send event WAIT_PRINT2 after 10 seconds */ } void wait_print2() { printf("Done!"); } while (event = getNextEvent()) { switch(event) { case WAIT_PRINT: wait_print(); case WAIT_PRINT2: wait_print2(); } } Stack Ripping

5. Stack Ripping with parameters void wait_print(char *msg) { sleep(10); printf(“%s”, msg); }

6. Stack Ripping with parameters void wait_print(char *msg) { Continuation *c = new Continuation( &wait_print2, msg); EventHandle eh = InitAsyncSleep(10); // sleep(10) RegisterContinuation(eh, c); } void wait_print2(Continuation *cont) { char *msg = (char *) cont->arg1; printf(“%s”, msg); } Adya, J. Howell, M. Theimer, W.J. Bolosky, J.R. Douceur Cooperative task management without manual stack management. In Proc. 2002 USENIX Annual Tech. Conference, June 2002.

7. Stack Ripping Overview • Manually managing a stack frame across multiple functions is very tedious and if not careful can result in large memory leaks

8. TAME • Expands the event system • Eliminates manual stack ripping • Allows events and threads to coexist • Implemented in C++ using portable libraries and a source-to-source translator • Automated event memory management scheme with garbage collection included

9. Threading void wait_print() { sleep(10); printf(“Done!”); } Tame tamed wait_print () { twait { timer(10, mkevent()); } printf(“Done!”); } Sleep and Print

10. Tame Abstractions • Events • Represents a future occurrence • Calling mkevent(s) or mkevent(r, i, s) creates an event of type event<T>, where T is a sequence of zero or more types that match the types of slot variables s • When trigger(v) is called, 0 or more results, v, are passed. These are trigger values that are stored in slots defined by the event.

11. Tame Abstractions • Event Example rendezvous<> r; int i = 0; int j = 0; event<int, int> e = mkevent(r, i, j); e.trigger(100, 87); assert(i == 100); // pass assert(j == 87); // pass

12. Tame Abstractions • Wait Points • Represented by twait{ statements; } • Blocks calling function until all events created in statements are triggered • Functions with wait points must have return type of “tamed” • Causes function to return to caller, but function remains incomplete • Execution point and variables are stored in memory • twait { timer(10, mkevent()); }

13. Tame Abstractions • Safe Local Variables • Variables whose values are preserved across wait points. • Must be defined inside of tvars {} • Allocates them in a heap-allocated closure

14. Example: Multiple DNS Lookups tamed gethost_ev(dnsname name, event<ipaddr> e); tamed multidns_tame(dnsname name[], ipaddr a[], int n, event<> done) { int i; for (i = 0; i < n; i++) gethost_ev(name[i], mkevent(a[i])); done.trigger(); } tvars { } twait { twait { } }

15. Tame Abstractions • Rendezvous objects • Collection of events relevant to a wait point • twait(r) unblocks when any event in r triggers. That event is then consumed. • twait(r, i) returns information about which event triggered in the optional i variable • rendezvous<I> will accept events with event IDs of type I.

16. Example: DNS Lookup with Timeout tvars { ipaddr a; rendezvous<bool> r; bool ok; } timer(10, mkevent(r, false)); gethost_ev(name, mkevent(r, true, a)); twait(r, ok); if(!ok) printf(“Timeout”); r.cancel();

17. tamed A::f(int x) { tvars { rendezvous<> r; } a(mkevent(r)); twait(r); b(); } 01 void A::f(int __tame_x, A_f_closure *__cls) { 02 if (__cls == 0) 03 __cls = new A__f__closure(this, &A::f, __tame_x); 04 assert(this == __cls->this_A); 05 int &x = __cls->x; 06 rendezvous<> &r = __cls->r; 07 switch(__cls->entry_point) { 08 case 0: // original entry 09 goto __A__f__entry__0; 10 case 1: // reentry after first twait 11 goto __A__f__entry__1; 12 __A__f__entry__0; 13 a(_mkevent(__cls,r)); 14 if(!r.has_queued_trigger()) { 15 __cls->entry_point = 1; 16 r.set_reenter_closure(__cls); 17 return; } 18 __A__f__entry__1; 19 b(); 20 } Translated Code

18. Threading • TAME can interoperate with thread packages • Useful for third-party functions that block • Threaded functions that use twait don’t use tamed return type • twait checks its events, if no events are triggered it asks for a wake up notification when any event is triggered, and yields to another thread

19. Threading • Can start a new thread with tfork • Uses Gnu PTH library • tfork(rendezvous<I> r, I i, threadfunc<V> f, V &v); • e = mkevent(r, i) • Fork a new thread • Call f, store return value in v • Trigger e • Exit thread

20. Threading Example tamed gethost_ev(const char *name, event<struct hostent *> e) { tvars { struct hostent *h; } twait { tfork(wrap(gethostbyname, name), h); } e.trigger(h); }

21. Memory Management • Uses reference counting to eliminate memory leaks • Key invariants that must be enforced I1. Function’s closure lives at least until control exits the function for the last time I2. Some trigger slots in an event may be safe local variables. Thus a function’s closure must live until events created in the function have been triggered I3. Events associated with a rendezvous r must be triggered once before r is deallocated. The programmer must manage r’s lifetime correctly and trigger each event exactly once.

22. Reference Counting • Incremented when functions called or pointers are created • Decremented when exiting a function or deallocating pointers • When an object’s reference count hits zero, object is deallocated • Weak references are not counted in the reference count. • Only allow access to object if it is still allocated • Used between events and rendezvous. If nothing references the event, it should be deallocated. But if the rendezvous is about to be deallocated, it needs to cancel all its events

23. Memory Management • Reference counts R1. Entering a tamed function the first time creates a strong reference to the corresponding closure, which is removed only when the function exits the last time R2. Each event created inside a closure holds a strong reference to that closure. The reference is dropped once the event is triggered R3. A rendezvous and its events keep weak references to each other. This allows the cancellation of any events that did not trigger before deallocation. Also, any triggered events can update its rendezvous

24. Memory Management • Extra reference count R4. Exiting a tamed function for the last time cancels any rendezvous allocated in that function’s closure. Cancelling a rendezvous will cancel any events associated with it.

25. Production use of TAME • OKWS - a lightweight web server • OkCupid.com - a dating web site • Used TAME to easily parallelize DB queries • NFS Server - written by MIT Grad students • Source code was cleaner and shorter than other students that didn’t use TAME.

26. Performance - Tame vs. Capriccio Knot web server benchmark

27. Benchmarked Functions static tamed nullfn() { tvars { int i(0); } i++; }

28. Performance - Tame vs libasync Cycle count of Tame and libasync operations

29. static tamed benchfn (int niter, event<> done) { tvars { int i; } for (i = 0; i < niter; i++) twait { timer(0, mkevent()); } done.trigger(); } static void noop_tame() {} void benchfn_tame_thr(int niter) { for (int i = 0; i < niter; i++) twait { tfork(wrap(noop_tame)); } } static void noop() { pthread_exit(NULL); } void benchfn_thr(int niter) { for (int i = 0; i < niter; i++) { pthread_t t; pthread_create(&t, NULL, noop, NULL); pthread_join(t, NULL); } } Benchmarked Functions

30. Performance - Tame vs. pthreads Using niter = 100

31. Conclusions • Tame offers a simple event-based concurrency system • Greatly improves readability but preserves the flexibility of events • Model is still up to programmer’s choice • Tame is ideal for networked and distributed systems • Excellent results in real event-based systems • Fewer lines of code, simple memory management and simple code maintenance

32. Evaluation • Very easy to use tool with negligible cost • Reduced code is always a good thing • Some of the performance tests could have been clearer • Newest version of TAME is called Tamer and is located at http://www.read.cs.ucla.edu/tamer/