Multithreading vs. Event Driven in Code Development of High Performance Servers

1. Multithreading vs. Event Driven in Code Development of High Performance Servers

2. Goals • Maximize server utilization and decrease its latency. • Server process doesn’t block! • Robust under heavy loads. • Relatively easy to develop.

3. Key Idea Multi-threaded servers Event driven servers Programming level: Multi-Threaded Event Driven Implementation level: Multi-Threaded Event Driven

4. Lazy threaded servers Key Idea • Get the best of the two worlds. Multi-threaded servers Event driven servers Programming level: Multi-Threaded Event Driven Implementation level: Multi-Threaded Event Driven

5. Backgound: AIO • Asynchronous I/O mainly used for disk reads/writes. • Issue disk request and return immediately without blocking for results. • On completion an event is raised.

6. Servers Architectures:a- Multi-threaded Servers • Kernel thread per request executing the request handler. • A thread blocking on I/O doesn’t block the server. • Ex: Apache, MySql. Request Handler A B Function Calls C

7. Multi-threaded Servers (Cont.) • Advantages: + Relatively easy to develop. • Disadvantages: - Not good performance. • Thread scheduling. • Context switching. - Doesn’t scale. • Under heavy load , many threads would be created exhausting kernel’s memory, crashing the server.

8. Needed for Event-Driven servers Asynchronous I/O • Kernel event notification mechanism. • Supported by FreeBSD Kernel. • Events queued in a kernel queue, and delivered to user by kernel calls. • Events usually correspond to AIO completions, or socket reads/writes.

9. b- Event Driven Servers • Single thread (event loop). • AIO operations. • I/O completions / Requests (events) put in Event Queue. • Continuations. • Ex:Flash web server (slightly different architecture). + High performance. + Scalable/Robust. - Hard to write. Event Loop Events From Kernel A1 A2 B2 B1 Event Queue (KQueue) C2 C1 “Blocking” operation

10. Similar design to event driven but no continuations. Threads yield control on fixed points (I/O). If request handler doesn’t “block” Run handler to completion then use same thread to handle next event. Else Suspend current thread, create another user thread to handle next event. Lazy Threading

11. IO_Wrapper(..) { uc = save_user_context(..); /* save the current user thread context to be able to resume it later */ aio_func(..); /* non-blocking I/O call */ stack=allocate_stack(); switch_thread(event_loop,stack); /* suspend current thread and create a new thread running the event loop using the newly allocated stack */ . . } Wrappers around “blocking” operations to create the blocking illusion. Uses AIO. Saves current stack instead of unwinding it. Creates a user thread by jumping to the event loop and using a new stack. Wrappers / Thread Suspension

12. Event_loop() { . . . e = get_event(..); /* read new event from the kernel’s kqueue */ if e is completion of aio_func(..) run_thread(uc); /* dispose current thread and resume the IO_Wrapper from where it has stopped */ . . } Restores registers and jumps back to wrapper as if blocking has finished. Current thread is disposed as it’s useless. Wrapper’s thread eventually returns to the event loop after handling its request. Thread Resumption

13. Summary Multi-Threaded Event Driven Lazy-Threaded Scalable No Yes Yes High Performance No Yes Yes “Easy” to build Yes No Yes >1 thread Yes(kernel) No Yes(user) Thread / request? Yes No No Arbitrary thread yield Yes N/A No