PA3: Improving Performance with I/O Multiplexing

1. PA3: Improving Performance with I/O Multiplexing Part 1-1: Nov. 7, Part 1-2: Nov. 10 Part 2-1: Nov. 17, Part 2-2: Nov.20

2. Dealing with Concurrency • Creating new child or new thread • Expensive • Only handling 1 connection in a thread • I/O multiplexing with event-driven programming • Multiplex input/output between multiple files (or sockets) • Select() • When event comes, select returns • Buffered I/O (POSIX AIO) • I/O asynchronously done in background • Libevent • When event comes, pre-designated function is called

3. Select() • Select returns when there’s new event • After select returns, it checks whether the fd is set. • If fd is set, process the event (e.g., handle HTTP request) // server example ret = select(n, &fds, NULL, NULL, NULL); if (ret < 0) { perror(“select”); } else if (ret) { for (i = 0; i < n; i++) { if (FD_ISSET(fd_set[i], &fds)) { /* if listening fd, accept* / /* otherwise, process request */ if (fd_set[i] == listenfd) { accept(listenfd, …); } else { read(fd_set[i], …); write(fd_set[i], …); } } } }

4. Part 1: POSIX AIO • POSIX asynchronous I/O • aio_read(), aio_write() immediately returns • The actual read and write are processed in the background • The application can do another work without being blocked by read() or write() • The completion of I/O can be notified by a signal

5. Server Example using AIO structaiocbaiocbList[MAX_FD]; sa.sa_flags= SA_RESTART | SA_SIGINFO; sa.sa_sigaction= aioSigHandler; if (sigaction(IO_SIGNAL, &sa, NULL) == -1) exit(1); while (1) { fd = accept(listenfd, …); if (fd < 0) continue; /* build aio control block */ aiocbList[fd].aio_fildes = fd; aiocbList[fd].aio_buf= malloc(BUF_SIZE); aiocbList[fd].aio_nbytes= BUF_SIZE; aiocbList[fd].aio_reqprio= 0; aiocbList[fd].aio_offset= 0; aiocbList[fd].aio_sigevent.sigev_notify= SIGEV_SIGNAL; aiocbList[fd].aio_sigevent.sigev_signo= IO_SIGNAL; aiocbList[fd].aio_sigevent.sigev_value.sival_ptr= &aiocbList[fd]; s = aio_read(&aiocbList[fd]); if (s == -1) errExit("aio_read"); }

6. Server Example using AIO /* Handler for I/O completion signal */ static void aioSigHandler(intsig, siginfo_t *si, void *ucontext) { structaiocb *acb = si->value.sival_ptr; intfd = acb->aio_fildes; /* process the request stored in acb->aio_buf /* write a response using aio_write() */ /* if aio_write() completes, close the connection */ } For more detail, refer to the man page of aio (in shell, $ man aio)

7. Part 2: Libevent • Event notification library • Execute a callback function when a specified event occurs on a file descriptor • Availability-based • If incoming connection available, it notifies • If incoming request available, it notifies • If sending buffer available, it notifies • Calls pre-registered callback function when there is event • Use nonblocking for I/O operations • Set socket fds to nonblocking by fcntl()

8. Server Example using libevent #include <event.h> /* for libevent */ int main() { struct event ev_accept; … /* create listening socket and set it nonblock */ … event_init(); /* register the event */ event_set(&ev_accept, listenfd, EV_READ | EV_PERSIST, OnAcceptEvent, NULL); event_add(&ev_accept, NULL); /* start event loop */ event_dispatch(); … close(listenfd); return 0; }

9. Server Example using libevent static void OnAcceptEvent(intfd, short event, void *arg) { struct event ev_read; intnew_fd = accept(fd, NULL, NULL); /* set new_fdnonblocking */ event_set(&ev_read, new_fd, EV_READ, OnReadEvent, NULL); event_add(&ev_read, NULL); } static void OnReadEvent(intfd, short event, void *arg) { struct event ev_write; char buf[BUFSIZE]; read(fd, buf, BUFSIZE); /* process the request in buffer */ /* write the response */ write(fd, response, len); }

10. Buffer Management • What if send buffer is full? • write() will return -1 • errno will be set to EAGAIN or EWOULDBLOCK • What you need to do? • You need to send the response again later • Remember the “state” • Until where it was sent • Register a write event • Retry later when write event comes • When send buffer becomes available, write event will come

11. Remembering the State struct context { char w_buf[BUFSIZE]; /* write buffer */ intfd; struct event ev_read; struct event ev_write; int off; /* buffer offset */ intrem_len; /* remaining length */ }; static void OnReadEvent(intfd, short event, void *arg) { struct context *ctx= (struct context *)arg; char buf[BUFSIZE]; rd = read(fd, buf, BUFSIZE); /* process the request in buffer */ /* start writing the response */ /* if not sent fully, register an event */ wr = write(fd, ctx->w_buf, ctx->rem_len); ctx->off += wr; ctx->rem_len-= wr; event_del(ctx->ev_read, NULL); event_set(ctx->ev_write, fd, EV_WRITE, OnWriteEvent, ctx); event_add(ctx->ev_write, NULL); }

12. Remembering the State static void OnWriteEvent(intfd, short event, void *arg) { struct context *ctx = (struct context *)arg; wr = write(fd, ctx->w_buf + ctx->off, ctx->rem_len); ctx->off += wr; ctx->rem_len -= wr; /* if done writing */ if (ctx->rem_len == 0) { event_del(ctx->ev_write, NULL); close(fd); free(ctx); } }

13. Evaluation • 50 pt for each part • 20 pt for basic functionality testing • Ina.kaist.ac.kr, yahoo, naver • 20 pt for performance and robustness testing • Testing with ab (ApachBench) • Many consecutive requests • Many concurrent connections • 10 pt for report • Performance comparison • Threaded vs. AIO vs. libevent