Lazy Asynchronous I/O For Event-Driven Servers

1. Lazy Asynchronous I/O For Event-Driven Servers Khaled Elmeleegy, Anupam Chanda and Alan L. Cox Department of Computer Science Rice University, Houston, Texas. Willy Zwaenepoel School of Computer and Communication Sciences EPFL, Lausanne, Switzerland.

2. Event-Driven Architecture • Event-driven architecture is widely used for servers • Performance • Scalability

3. Problem • Developing event-driven servers is hard for many reasons • We focus on problems with non-blocking I/O: • Incomplete coverage • Burden of state maintenance

4. Lazy Asynchronous I/O (LAIO) • Addresses problems with non-blocking I/O • Universality • Covers all I/O operations • Simplicity • Requires less code • High performance for event-driven servers • Meets or exceeds alternatives

5. Outline • Background • Lazy Asynchronous I/O (LAIO) • Evaluation • Conclusions

6. Outline • Background • Event-driven servers • Lazy Asynchronous I/O (LAIO) • Evaluation • Conclusions

7. Event-Driven Servers • Event loop processes incoming events • For each incoming event, it dispatches its handler • Single thread of execution Event Loop Handler #1 Handler #2 Handler #k

8. Event Handler I/O operation (Network/Disk) To event loop Complete handling event

9. Event Handler • If the I/O operation blocks • The server stalls I/O operation (Network/Disk) Block To event loop Complete handling event

10. Outline • Background • Lazy Asynchronous I/O (LAIO) • API • Example of usage • Implementation • Evaluation • Conclusions

11. LAIO API

12. laio_syscall() • Lazily converts any system call into an asynchronous call • If the system call doesn’t block • laio_syscall() returns immediately • With return value of system call • Else if it blocks: • laio_syscall() returns immediately • With return value -1 • errno set to EINPROGRESS • Background LAIO operation

13. laio_gethandle() • Returns a handle representing the last issued LAIO operation • If operation didn’t block, NULL is returned

14. laio_poll() • Returns a count of completed background LAIO operations • Fills an array with completion entries • One for each operation • Each completion entry has • Handle • Return value • Error value

15. Event Handler With LAIO • If operation completes • Handler continues execution laio_syscall() Operation completed No To event loop Yes Complete handling event

16. Event Handler With LAIO • If operation doesn’t complete • laio_syscall() returns immediately • Handler records LAIO handle • Returns to event loop • Completion notification arrives later laio_syscall() Operation completed No Handle= laio_gethandle() To event loop Yes Complete handling event

17. LAIO Implementation • LAIO uses scheduler activations • Scheduler activations • The kernel delivers an upcall when an operation • Blocks - laio_syscall() • Unblocks - laio_poll()

18. laio_syscall() – Non-blocking case Application laio_syscall() • Save context • Enable upcalls Issue operation Library System call blocks? No • Disable upcalls • Return retval

19. laio_syscall() – Blocking case Application laio_syscall() Library • Save context • Enable upcalls Issue operation System call blocks? Yes

20. laio_syscall() – Blocking case Application laio_syscall() Library • Save context • Enable upcalls Issue operation System call blocks? Yes Kernel Background laio_syscall Upcall on a new thread

21. laio_syscall() – Blocking case Application laio_syscall() Library • Save context • Enable upcalls Issue operation upcall handler System call blocks? Library Steals old stack using stored context Yes Kernel Background laio operation Upcall on a new thread

22. laio_syscall() – Blocking case Application laio_syscall() • Disable upcalls • errno = EINPROGRESS • Return -1 Library • Save context • Enable upcalls Issue operation upcall handler System call blocks? Library Steals old stack using stored context Yes Kernel Background laio operation Upcall on a new thread

23. List of completions is retrieved by the application using laio_poll() upcall handler() Library • Construct completion structure: • laio operation handle. • System call return value. • Error code. • Add completion to list of • completions. Unblocking case Kernel • Background laio operation completes, thread dies • Upcall on the current thread

24. Outline • Background • Lazy Asynchronous I/O (LAIO) • Evaluation • Methodology • Experiments • LAIO vs. conventional non-blocking I/O • LAIO vs. AMPED • Programming complexity • Conclusions

25. Methodology • Flash web server • Intel Xeon 2.4 GHz with 2 GB memory • Gigabit Ethernet between machines • FreeBSD 5.2-CURRENT • Two web workloads • Rice 1.1 GB footprint • Berkeley 6.4 GB footprint

26. Experiments: LAIO vs. conventional non-blocking I/O Compare performance

27. Performance: Large Workload

28. Performance: Small Workload

29. Why Be Lazy? • Most potentially blocking operations don’t actually block • Experiments: 73% - 86% of such operations don’t block • Lower overhead • Micro-benchmark: AIO is 3.2 times slower than LAIO for non-blocking operations

30. Experiments: LAIO vs. AMPED Compare performance

31. Asymmetric Multi-process Event-Driven (AMPED) Architecture • Event-driven core • Potentially blocking I/O handed off to a helper • Helper can block as it runs on a separate thread of execution Event Loop Helper #1 Handler #1 Helper #2 Handler #2 Helper #n Handler #k

32. Flash-NB-AMPED • Stock version of flash • Two relevant helpers • File read: reads a file from disk • Name conversion: checks for a file’s existence and permissions

33. Performance of LAIO vs. AMPED

34. Performance of LAIO vs. AMPED

35. Programming Complexity • Where performance is comparable: Flash-LAIO-LAIO vs. Flash-NB-AMPED • Flash-LAIO-LAIO is simpler • No helpers • No state maintenance

36. State Maintenance With Non-blocking I/O • If operation does not complete • Handler returns to event loop handler I/O operation Operation completed No To event loop Yes Complete handling event

37. State Maintenance With Non-blocking I/O • If operation does not complete • Handler returns to event loop • Operation is continued later handler I/O operation Operation completed No To event loop Yes Complete handling event

38. State Maintenance With Non-blocking I/O • If operation does not complete: • Handler returns to event loop • Operation is continued later • This may require storing state handler I/O operation Operation completed To event loop No Store state Yes Complete handling event

39. Lines of Code Comparison 9.5% reduction in lines of code

40. Conclusions • LAIO is universal • Supports all system calls • LAIO is simpler • Used uniformly • No state maintenance • No helpers • Less lines of code • LAIO meets or exceeds the performance of other alternatives

41. LAIO source can be found at: http://www.cs.rice.edu/~kdiaa/laio