Concurrent Programming

1. Concurrent Programming Lecture 5

2. Last lecture • TCP • I/O • RIO: buffered I/O. • Unix IO • Question: difference between send() and write()? • MSG_DONTWAIT, MSG_NOSIGNAL

3. Concurrent Programming is Hard! • The human mind tends to be sequential • The notion of time is often misleading • Thinking about all possible sequences of events in a computer system is at least error prone and frequently impossible

4. Concurrent Programming is Hard! • Classical problem classes of concurrent programs: • Races: outcome depends on arbitrary scheduling decisions elsewhere in the system • Example: who gets the last seat on the airplane? • Deadlock: improper resource allocation prevents forward progress • Example: traffic gridlock • Livelock / Starvation / Fairness: external events and/or system scheduling decisions can prevent sub-task progress • Example: people always jump in front of you in line

5. rio_writen rio_readlineb rio_readlineb rio_writen EOF rio_readlineb close close Iterative Echo Server Client Server socket socket bind open_listenfd open_clientfd listen Connection request connect accept Client / Server Session Await connection request from next client

6. Iterative Servers • Iterative servers process one request at a time client 1 server client 2 connect connect accept read write write call read call read write ret read close close Wait for Client 1 accept read write ret read

7. Where Does Second Client Block? • Second client attempts to connect to iterative server • Call to connect returns • Even though connection not yet accepted • Server side TCP manager queues request • Feature known as “TCP listen backlog” • Call to rio_writen returns • Server side TCP manager buffers input data • Call to rio_readlineb blocks • Server hasn’t written anything for it to read yet. Client socket open_clientfd Connection request connect rio_writen rio_readlineb

8. Fundamental Flaw of Iterative Servers client 1 server client 2 connect • Solution: use concurrent servers instead • Concurrent servers use multiple concurrent flows to serve multiple clients at the same time connect accept read write write call read call read write ret read User goes out to lunch Client 1 blocks waiting for user to type in data Server blocks waiting for data from Client 1 Client 2 blocks waiting to read from server

9. Creating Concurrent Flows • Allow server to handle multiple clients simultaneously • 1. Processes • Kernel automatically interleaves multiple logical flows • Each flow has its own private address space • 2. Threads • Kernel automatically interleaves multiple logical flows • Each flow shares the same address space • 3. I/O multiplexing with select() • Programmer manually interleaves multiple logical flows • All flows share the same address space • Relies on lower-level system abstractions

10. Concurrent Servers: Multiple Processes • Spawn separate process for each client client 1 server client 2 call accept call connect call connect ret connect ret accept call fgets fork child 1 call accept call read User goes out to lunch Client 1 blocks waiting for user to type in data ret connect call fgets ret accept fork write child 2 call read call read ... write end read close close

11. Review: Iterative Echo Server int main(int argc, char **argv) { int listenfd, connfd; int port = atoi(argv[1]); struct sockaddr_in clientaddr; int clientlen = sizeof(clientaddr); listenfd = Open_listenfd(port); while (1) { connfd = Accept(listenfd, (SA *)&clientaddr, &clientlen); echo(connfd); Close(connfd); } exit(0); } • Accept a connection request • Handle echo requests until client terminates

12. Process-Based Concurrent Server int main(int argc, char **argv) { int listenfd, connfd; int port = atoi(argv[1]); struct sockaddr_in clientaddr; int clientlen=sizeof(clientaddr); Signal(SIGCHLD, sigchld_handler); listenfd = Open_listenfd(port); while (1) { connfd = Accept(listenfd, (SA *) &clientaddr, &clientlen); if (Fork() == 0) { Close(listenfd); /* Child closes its listening socket */ echo(connfd); /* Child services client */ Close(connfd); /* Child closes connection with client */ exit(0); /* Child exits */ } Close(connfd); /* Parent closes connected socket (important!) */ } } Fork separate process for each client Does not allow any communication between different client handlers

13. Process-Based Concurrent Server(cont) • Reap all zombie children void sigchld_handler(int sig) { while (waitpid(-1, 0, WNOHANG) > 0) ; return; }

14. Process Execution Model • Each client handled by independent process • No shared state between them • Both parent & child have copies of listenfd and connfd • Parent must close connfd • Child must close listenfd Connection Requests Listening Server Process Client 2 Server Process Client 1 Server Process Client 1 data Client 2 data

15. Concurrent Server: accept Illustrated listenfd(3) 1. Server blocks in accept, waiting for connection request on listening descriptor listenfd Client Server clientfd Connection request listenfd(3) 2. Client makes connection request by calling and blocking in connect Client Server clientfd listenfd(3) 3. Server returns connfd from accept. Forks child to handle client. Clientreturns from connect. Connection is now established between clientfd and connfd Server Server Child Client clientfd connfd(4)

16. Implementation Must-dos With Process-Based Designs • Listening server process must reap zombie children • to avoid fatal memory leak • Listening server process must close its copy of connfd • Kernel keeps reference for each socket/open file • After fork, refcnt(connfd) = 2 • Connection will not be closed until refcnt(connfd) == 0

17. View from Server’s TCP Manager Client 1 Client 2 Server srv> ./echoserverp 15213 cl1> ./echoclient greatwhite.ics.cs.cmu.edu 15213 srv> connected to (128.2.192.34), port 50437 cl2> ./echoclient greatwhite.ics.cs.cmu.edu 15213 srv> connected to (128.2.205.225), port 41656

18. Pros and Cons of Process-Based Designs • + Handle multiple connections concurrently • + Clean sharing model • file tables (yes) • global variables (no) • + Simple and straightforward • –Additional overhead for process control • –Nontrivial to share data between processes • Requires IPC (interprocess communication) mechanisms • FIFO’s (named pipes), System V shared memory and semaphores

19. #2)Event-Based Concurrent Servers Using I/O Multiplexing • Use library functions to construct scheduler within single process • Server maintains set of active connections • Array of connfd’s • Repeat: • Determine which connections have pending inputs • If listenfd has input, then accept connection • Add new connfd to array • Service all connfd’s with pending inputs

20. Adding Concurency: Step One • Start with allowing address re-use • Then we set the socket to non-blocking int sock, opts; sock = socket(…); // getting the current options setsockopt(sock, SOL_SOCKET, SO_REUSEADDR, &opts, sizeof(opts)); // getting current options if (0 > (opts = fcntl(sock, F_GETFL))) printf(“Error…\n”); // modifying and applying opts = (opts | O_NONBLOCK); if (fcntl(sock, F_SETFL, opts)) printf(“Error…\n”); bind(…);

21. Adding Concurrency: Step Two • Monitor sockets with select() • int select(intmaxfd, fd_set *readfds, fd_set *writefds, fd_set *exceptfds, const structtimespec *timeout); • So what’s an fd_set? • Bit vector with FD_SETSIZE bits • maxfd – Max file descriptor + 1 • readfs – Bit vector of read descriptors to monitor • writefds – Bit vector of write descriptors to monitor • exceptfds – Read the manpage, set to NULL • timeout – How long to wait with no activity before returning, NULL for eternity

22. How does the code change? if (listen(sockfd, 5) < 0) { // listen for incoming connections printf(“Error listening\n”); init_pool(sockfd, &pool); while (1) { pool.ready_set = &pool.read_set; pool.nready = select(pool.maxfd+1, &pool.ready_set, &pool.write_set, NULL, NULL); if (FD_ISSET(sockfd, &pool.ready_set)) { if (0 > (isock = accept(sockfd,…))) printf(“Error accepting\n”); add_client(isock, &caddr, &pool); } check_clients(&pool); } // close it up down here

23. What was pool? • A struct something like this: typedefstructs_pool { intmaxfd; // largest descriptor in sets fd_setread_set; // all active read descriptors fd_setwrite_set; // all active write descriptors fd_setready_set; // descriptors ready for reading intnready; // return of select() intmaxi; /* highwater index into client array */ intclientfd[FD_SETSIZE]; /* set of active descriptors */rio_tclientrio[FD_SETSIZE]; /* set of active read buffers */ } pool;

24. void init_pool(intsockfd, pool * p); { inti; p->maxfd = -1; for (i=0;i<FD_SETSIZE;i++) p->clientfd[i]=-1; p->maxfd = sockfd; FD_ZERO(&p->read_set); FD_SET(listenfd, &p->read_set); } // close it up down here

25. void add_client(intconnfd, pool *p) { inti; p->nready--; for (i = 0; i < FD_SETSIZE; i++) /* Find an available slot */ if (p->clientfd[i] < 0) { /* Add connected descriptor to the pool */ p->clientfd[i] = connfd; Rio_readinitb(&p->clientrio[i], connfd); /* Add the descriptor to descriptor set */ FD_SET(connfd, &p->read_set); /* Update max descriptor and pool highwater mark */ if (connfd > p->maxfd) p->maxfd = connfd; if (i > p->maxi) p->maxi = i; break; } if (i == FD_SETSIZE) /* Couldn't find an empty slot */ app_error("add_client error: Too many clients"); }// close it up down here

26. void check_clients(pool *p) { inti, connfd, n; char buf[MAXLINE]; rio_trio; for (i = 0; (i <= p->maxi) && (p->nready > 0); i++) { connfd = p->clientfd[i]; rio = p->clientrio[i]; /* If the descriptor is ready, echo a text line from it */ if ((connfd > 0) && (FD_ISSET(connfd, &p->ready_set))) { p->nready--; if ((n = Rio_readlineb(&rio, buf, MAXLINE)) != 0) { byte_cnt += n; //line:conc:echoservers:beginecho printf("Server received %d (%d total) bytes on fd %d\n", n, byte_cnt, connfd); Rio_writen(connfd, buf, n); //line:conc:echoservers:endecho } /* EOF detected, remove descriptor from pool */ else { Close(connfd); //line:conc:echoservers:closeconnfd FD_CLR(connfd, &p->read_set); //line:conc:echoservers:beginremove p->clientfd[i] = -1; //line:conc:echoservers:endremove } } } }

27. So what about bit vectors? • void FD_ZERO(fd_set *fdset); • Clears all the bits • void FD_SET(intfd, fd_set *fdset); • Sets the bit for fd • void FD_CLR(intfd, fd_set *fdset); • Clears the bit for fd • int FD_ISSET(intfd, fd_set *fdset); • Checks whether fd’s bit is set

28. What about checking clients? • The code only tests for new incoming connections • But we have many more to test! • Store all your client file descriptors • In pool is a good idea! • Several scenarios • Clients are sending us data • We may have pending data to write in a buffer • Keep the while(1) thin • Delegate specifics to functions that access the appropriate data • Keep it orthogonal!

29. Back to that lifecycle… Client(s) Server socket() socket() bind() listen() select() FD_ISSET(sfd) connect() Connection Request accept() write() read() Client / Server Session(s) read() write() close() EOF read() check_clients() main loop close()

30. I/O Multiplexed Event Processing Read Active Descriptors Pending Inputs listenfd = 3 listenfd = 3 clientfd clientfd 10 0 10 7 1 Active 7 2 4 4 3 -1 -1 Inactive 4 -1 -1 5 12 12 Active 6 5 5 7 -1 -1 8 -1 -1 9 Never Used -1 -1

31. Time out? int select(intnfds, fd_set *readfds, fd_set *writefds, fd_set *exceptfds, structtimeval *timeout);

32. Pros and Cons of I/O Multiplexing • + One logical control flow. • + Can single-step with a debugger. • + No process or thread control overhead. • Design of choice for high-performance Web servers and search engines. • – Significantly more complex to code than process- or thread-based designs. • – Hard to provide fine-grained concurrency • E.g., our example will hang up with partial lines. • – Cannot take advantage of multi-core • Single thread of control