Simple Web Server I/O Models: select() and Thread-Based Blocking

1. Project Assignment 1 Ying Zhang EECS 489 W07

2. Clarification • Questions for each test • Expected output for each test

3. Project • a simple Web server that only serves files. • two different I/O models: • select()-based I/O multiplexing (45 points) • use select() to serve all clients in a single thread • thread-based blocking I/O. (45 points) • pthreads serving each client with its own thread. • documentation (10 points) • one to two pages documentation • README file

4. Clarification • HTTP file directory: chroot problem The problem is because in many systems usually only the super-user can use it. Because most of you cannot use as super-user, you cannot use chroot. However, the spec requires: disallow clients from access files above the working directory where the server is run So, besides using chroot, you can parse the command line to check whether there are "../file" such character exists. • checksum issue First of all, the checksum is hard-coded in the python code. And the calcuation is platform dependent. I have modified the HttpTest.py to set the checksum based on platform. So, you don't need to worry about that. Please download the latest HttpTest.py from the web page. • File size In the HTTP spec, you can use "HTTP/1.%d 200 OK\r\nContent-Type: text/html\r\nContent-Length: 1000" to set the file size. However, the filesize is hard-coded in HttpTest.py. So, you don't need to consider how to set the file size for this project because it is not required. • connection for Test 3 The problem is because that there is an inconsistency between the HttpTest.py and the project specification. I have changed the connection value for Test3 to be 1000 to make it consistent with the spec.

5. Questions for each test

6. select • can use blocking socket • select has three sets read_sets, write_sets, exception_sets • FD_ZERO(&rfds); • FD_ZERO(&efds); • FD_ZERO(&wfds); • FD_SET(listen_Sockfd,&rfds); • FD_SET(listen_Sockfd,&efds); • if (socks[i].state==StateWaitingForRequest) • FD_SET(socks[i].sock, &rfds); • FD_SET(socks[i].sock, &efds); • if (socks[i].state==StateSendingFile) • FD_SET(socks[i].sock, &wfds); • FD_SET(socks[i].sock, &efds); select(MAX_FDSIZE, &rfds, &wfds, &efds, &to); • protect recv() and send() by using select • if (FD_ISSET(socks[i].sock, &rfds)) • recv() • if (FD_ISSET(socks[i].sock, &wfds)) • send()

7. Data ready • Number of bytes is greater than low-water mark for socket receive buffer • Read half of the connection is closed(FIN,EOF) • Listen socket, incoming connection; • Socket error pending

8. Timeout • Acutally, you need to implete the checking for inactive connections. To do that, you need to remember a time for the client's latest activity and use that to compare with your current time • Broken pipe • When sending a file, our code will spontaneously stop, then display "Broken pipe". The python test (#1) ends up throwing a checksum error: signal(SIGPIPE, SIG_IGN);

9. Test 2 • Resource temporally unavailable? • client sequentially send request for a small file, recv file, and close the socket (Phython code GetFile()) • server finish sends • need to check whether the connection is closed by the client • select(MAX_FDSIZE, &rfds, &wfds, &efds, &to); • for (int i=0;i<ALL_CONNECTIONS;i++) • if (FD_ISSET(socks[i].sock, &rfds))– ready to read because of FIN_received • If(recv() == 0) close_connection() • socket number increased by 2, reaching max 1024 quickly? • limit the number of file descriptor used for opening file at a time

10. Test 3 • Connect to the server until the server cannot handle more connections • Set maximum number of connections • //check if can accept another connection • if (nconn<MAX_CONNECTIONS) • Or • Select(MAX_CONNECTIONS, &rfds, &wfds, &efds, &to); • Keep persistent connections • //get current time • time(&curTime); • If(curTime-socks[i].timestamp)>=TIMEOUT) • close_connection • Rev request: • time(&socks[i].timestamp);

11. Test 3 • avoid blocking in send() • each time send a chunk of file for each connection • select(FD_SETSIZE, &rfds, &wfds, &efds, &to); • for (int i=0;i<CONNECTIONS_WAIT_SNED;i++) • if (FD_ISSET(socks[i].sock, &wfds)) • Send size • if we are sending an error, file_pos=-1 • file and are not done with the file send chuck size • whatever remains in the buffer

12. Test 5 • Maximum number of concurrent slow downloads • client send request without recv() • In Python code, DownloadThread() function, check __MAX_IDLE_SECONDS__ (10sec), close

13. Expected output for each test

14. Test 1 • ############################################################### • Test #1: Get a single file from the web server (Simple Correctness) • Calculating checksum • HttpTest.py:194: FutureWarning: %u/%o/%x/%X of negative int will return a signed string in Python 2.4 and up • print "Received file with checksum: %08X, elapsed time: %f seconds" % \ • Received file with checksum: 83021ED8, elapsed time: 0.170263 seconds • Checksum is correct (PASSED) • #Wed Feb 7 16:27:38 2007 0 Accepting new connection from client 141.212.110.75:43036. • ##Wed Feb 7 16:27:39 2007 0 client 141.212.110.75:43036 prematurely disconnected.

15. Test 2 • ############################################################### • Test #2: Get a small file 2000 times from the web server (Resource de-allocation and latency) • Received 2000 files successfully, average request time: 1.936741 milliseconds • #Wed Feb 7 16:28:12 2007 0 Accepting new connection from client 141.212.110.75:43037. • ##Wed Feb 7 16:28:12 2007 0 client 141.212.110.75:43037 prematurely disconnected. • …… • ##Wed Feb 7 16:28:12 2007 0 Accepting new connection from client 141.212.110.75:43038. • ##Wed Feb 7 16:28:12 2007 0 client 141.212.110.75:43038 prematurely disconnected.

16. Test 3 • ############################################################### • Test #3: Connect to the server until the server cannot handle more connections (Scalability) • Server handled all 1000 connections without failing • ##Wed Feb 7 16:29:08 2007 0 Accepting new connection from client 141.212.110.75:45037. • ##Wed Feb 7 16:29:08 2007 0 Accepting new connection from client 141.212.110.75:45038. • ##Wed Feb 7 16:29:08 2007 0 Accepting new connection from client 141.212.110.75:45039. • ##Wed Feb 7 16:29:08 2007 0 Accepting new connection from client 141.212.110.75:45040. • ##Wed Feb 7 16:29:08 2007 0 Accepting new connection from client 141.212.110.75:45041. • ##Wed Feb 7 16:29:08 2007 0 Accepting new connection from client 141.212.110.75:45042. • ##Wed Feb 7 16:29:08 2007 0 Accepting new connection from client 141.212.110.75:45043. • ##Wed Feb 7 16:29:08 2007 0 Accepting new connection from client 141.212.110.75:45044. • ….. • ##Wed Feb 7 16:29:32 2007 104 client 141.212.110.75:45037 prematurely disconnected.

17. Test 4 • ############################################################### • Test #4: Maximum bandwidth of 100 concurrent downloads (Throughput) • Waiting for downloads to complete........................................................................................................................................................................................................ • Test case #4 finished successfully -- Average bandwidth for ~50 MByte transfer: 39.0049286575 MBits/Second

18. Test 5 • ############################################################### • Test #5: Maximum number of concurrent slow downloads (Fairness and scalability) • Starting concurrent slow downloads......................................................................................................................... • 1210 connections in progress when terminated, 0 finished

19. Test 6 • ############################################################### • Test #6: Repeatedly connect, send request, then close connection (Resource de-allocation and error handling) • Successfully handled 5000 subsequent failed connections (PASSED)