1 / 49

More Network Programming

Learn about HTTP push server, request framing, useful APIs, select/poll and advanced sockets tidbits with code walk-throughs and flow charts from the UIUC CS/ECE 438 course in Fall 2006.

mmaud
Download Presentation

More Network Programming

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. More Network Programming

  2. More Network Programming • HTTP push server • Request framing and server push concepts • Demo • Useful API’s • select/poll and advanced sockets tidbits • threads • HTTP push server code • Components • Flow charts • Code walk-through (code is online) UIUC - CS/ECE 438, Fall 2006

  3. HTTP Request Framing • Characteristics • ASCII-based (human readable) • Framed by text lines • First line is command • Remaining lines are additional data • Blank line ends request frame GET /surf/too/much.html HTTP/1.0 Date: 28 February 2006 11:25:53 CST Host: www.surfanon.org <blank line> UIUC - CS/ECE 438, Fall 2006

  4. HTTP Server Push • Idea • Connection remains open • Server pushes down new data as needed • Termination • Any time by server • Stop loading (or reload) by client • Components • Header indicating multiple parts • New part replaces old part • New part sent any time • Wrappers for each part UIUC - CS/ECE 438, Fall 2006

  5. the data component HTTP Server Push HTTP/1.0 200 OK Content-type: multipart/x-mixed-replace;\ boundary=---never_in_document--- ---never_in_document--- Content-type: text/html (actual data) ---never_in_document--- UIUC - CS/ECE 438, Fall 2006

  6. More Network Programming • Useful Application Programming Interfaces • Topics • More advanced sockets • Unix file functionality • Multithreaded programming (Posix Threads) • Specific APIs • select/poll and advanced sockets • Threads • Attributes • Creation • Thread-specific data • Minor synchronization UIUC - CS/ECE 438, Fall 2006

  7. Select and Poll • Building timeouts with select/poll • Similar functions • Parameters • Set of file descriptors • Set of events for each descriptor • Timeout length • Return value • Set of file descriptors • Events for each descriptor • Notes • Select is somewhat simpler • Poll supports more events UIUC - CS/ECE 438, Fall 2006

  8. Select and Poll: Prototypes • Select: • Wait for readable/writable file descriptors #include <sys/time.h> int select (int num_fds, fd_set* read_set, fd_set* write_set, fd_set* except_set, struct timeval* timeout); • Poll: • Poll file descriptors for events #include <poll.h> int poll (struct pollfd* pfds, nfds_t nfds, int timeout); UIUC - CS/ECE 438, Fall 2006

  9. Select int select (int num_fds, fd_set* read_set, fd_set* write_set, fd_set* except_set, struct timeval* timeout); • Wait for readable/writable file descriptors. • Return: • Number of descriptors ready • -1 on error, sets errno • Parameters: • num_fds: • number of file descriptors to check, numbered from 0 • read_set, write_set, except_set: • Sets (bit vectors) of file descriptors to check for the specific condition • timeout: • Time to wait for a descriptor to become ready UIUC - CS/ECE 438, Fall 2006

  10. File Descriptor Sets • Bit vectors • Often 1024 bits (FD_SETSIZE) • Only first num_fdschecked • Macros to create and check sets fds_set myset; void FD_ZERO (&myset); /* clear all bits */ void FD_SET (n, &myset); /* set bits n to 1 */ void FD_CLEAR (n, &myset); /* clear bit n */ int FD_ISSET (n, &myset); /* is bit n set? */ UIUC - CS/ECE 438, Fall 2006

  11. File Descriptor Sets • Three conditions to check for • Readable: • Data available for reading • Writable: • Buffer space available for writing • Exception: • Out-of-band data available (TCP) UIUC - CS/ECE 438, Fall 2006

  12. Timeout • Structure struct timeval { long tv_sec; /* seconds */ long tv_usec; /* microseconds */ }; UIUC - CS/ECE 438, Fall 2006

  13. Select • High-resolution sleep function • All descriptor sets NULL • Positive timeout • Wait until descriptor(s) become ready • At least one descriptor in set • timeoutNULL • Wait until descriptor(s) become ready or timeout occurs • At least one descriptor in set • Positive timeout • Check descriptors immediately (poll) • At least one descriptor in set • 0timeout UIUC - CS/ECE 438, Fall 2006

  14. Select: Example fd_set my_read; FD_ZERO(&my_read); FD_SET(0, &my_read); if (select(1, &my_read, NULL, NULL) == 1) { ASSERT(FD_ISSET(0, &my_read); /* data ready on stdin */ UIUC - CS/ECE 438, Fall 2006

  15. Poll #include <poll.h> int poll (struct pollfd* pfds, nfds_t nfds, int timeout); • Poll file descriptors for events. • Return: • Number of descriptors with events • -1 on error, sets errno • Parameters: • pfds: • An array of descriptor structures. File descriptors, desired events and returned events • nfds: • Length of the pfds array • timeout: • Timeout value in milliseconds UIUC - CS/ECE 438, Fall 2006

  16. Descriptors • Structure struct pollfd { int fd; /* file descriptor */ short events; /* queried event bit mask */ short revents; /* returned event mask */ • Note: • Any structure with fd < 0 is skipped UIUC - CS/ECE 438, Fall 2006

  17. Event Flags • POLLIN: • data available for reading • POLLOUT: • Buffer space available for writing • POLLERR: • Descriptor has error to report • POLLHUP: • Descriptor hung up (connection closed) • POLLVAL: • Descriptor invalid UIUC - CS/ECE 438, Fall 2006

  18. Poll • High-resolution sleep function • 0 nfds • Positive timeout • Wait until descriptor(s) become ready • nfds> 0 • timeoutINFTIMor -1 • Wait until descriptor(s) become ready or timeout occurs • nfds> 0 • Positive timeout • Check descriptors immediately (poll) • nfds> 0 • 0 timeout UIUC - CS/ECE 438, Fall 2006

  19. Poll: Example struct pollfd my_pfds[1]; my_pfds[0].fd = 0; my_pfds[0].events = POLLIN; if (poll(&my_pfds, 1, INFTIM) == 1) { ASSERT (my_pfds[0].revents & POLLIN); /* data ready on stdin */ UIUC - CS/ECE 438, Fall 2006

  20. Advanced Sockets signal (SIGPIPE, SIG_IGN); • Call at start of main in server • Allows you to ignore broken pipe signals which are degenerated when you write to a socket that has already been closed on the other side • Default handler exits (terminates process) UIUC - CS/ECE 438, Fall 2006

  21. Advanced Sockets int yes = 1; setsockopt (fd, SOL_SOCKET, SO_REUSEADDR, (char *) &yes, sizeof (yes)); • Call just before bind • Allows bind to succeed despite the existence of existing connections in the requested TCP port • Connections in limbo (e.g. lost final ACK) will cause bind to fail UIUC - CS/ECE 438, Fall 2006

  22. Posix Threads (pthreads) • When coding • Include <pthread.h> first in all source files • When compiling • Use compiler flag –D_REENTRANT • When linking • Link library -lpthread UIUC - CS/ECE 438, Fall 2006

  23. pthreads Attributes • Attributes • Data structure pthread_attr_t • Set of choices for a thread • Passed in thread creation routine • Choices • Inherit scheduling specifics from parent thread? • Compete at process or thread level? • Scheduling policy • Thread priority • Detached state • Join operation required at termination? • Only useful if thread returns a value UIUC - CS/ECE 438, Fall 2006

  24. pthreads Attributes • Initialize an attributes structure to the default values • int pthread_attr_init (pthread_attr_t* attr); • Set the detached state value in an attributes structure • int pthread_attr_setdetachedstate (pthread_attr_t* attr, int value); • Value • PTHREAD_CREATE_DETACHED • PTHREAD_CREATE_JOINABLE UIUC - CS/ECE 438, Fall 2006

  25. pthread Error Handling • pthreads functions do not follow the usual Unix conventions • Similarity: • Returns 0 on success • Differences: • Return error code on failure • Does not set errno • What about errno? • Each thread has its own • Define _REENTRANT (-D_REENTRANT switch to compiler) when using pthreads UIUC - CS/ECE 438, Fall 2006

  26. pthread Creation int pthread_create (pthread_t* tid, pthread_attr_t* attr, void*(child_main), void* arg); • Spawn a new posix thread • Parameters: • tid: • Unique thread identifier returned from call • attr: • Attributes structure used to define new thread • Use NULL for default values • child_main: • Main routine for child thread • Takes a pointer (void*), returns a pointer (void*) • arg: • Argument passed to child thread UIUC - CS/ECE 438, Fall 2006

  27. pthread Cleanup • Problem • A thread can be terminated by an external source (i.e. by other threads) • Deallocation an other cleanup must still be performed • Solution • Push and pop cleanup routines • Defer external cancellation until after perfroming cleanup to avoid race conditions UIUC - CS/ECE 438, Fall 2006

  28. pthread Cleanup { pthread_cleanup_push (my_cleanup, heap_data); … { … } pthread_setcancel_type(PTHREAD_CANCEL_DEFERRED, &old); pthread_cleanup_pop(1); } UIUC - CS/ECE 438, Fall 2006

  29. pthread Cleanup Pitfalls • your code... if (i_need_a_cleanup) { pthread_cleanup_push (my_cleanup, my_arg); } /* some code outside the cleanup logic */ if (i_need_a_cleanup) { pthread_cleanup_pop (1); } UIUC - CS/ECE 438, Fall 2006

  30. pthread Cleanup Pitfalls • what your code becomes... if (i_need_a_cleanup) { { /* code to push cleanup */ /* PUSH translation */ } /* THIS CODE IS NOW IN THE CLEANUP LOGIC! */ /* some code outside the cleanup logic */ if (i_need_a_cleanup) { /* code to exec cleanup */ }/* POP translation */ } UIUC - CS/ECE 438, Fall 2006

  31. Unix Files #include <sys/stat.h> int stat(const char* name, struct stat* buf); • Get information about a file • Returns: • 0 on success • -1 on error, sets errno • Parameters: • name: Path to file. Absolute paths begin with “/”, relative paths do not • buf: Statistics structure • off_t st_size: Size in bytes • time_t st_mtime: Date of last modification. Seconds since January 1, 1970 UIUC - CS/ECE 438, Fall 2006

  32. Unix File Operations • Open (and/or create) a file for reading, writing or both int open (const char* name, in flags); • Read data from one buffer to file descriptor size_t read (int fd, void* buf, size_t cnt); • Write data from file descriptor into buffer size_t write (int fd, void* buf, size_t cnt); • Close a file int close(int fd); UIUC - CS/ECE 438, Fall 2006

  33. File: Open #include <sys/types.h> #include <sys/stat.h> #include <fcntl.h> int open (const char* name, int flags); • Open (and/or create) a file for reading, writing or both • Returns: • New file descriptor on success ( 0) • -1 on error, sets errno • Parameters: • name: Path to file, absolute or relative • flags: • O_RDONLY: read only, O_WRONLY: write only, O_RDWR: read and write, O_CREAT: create file if it doesn’t exist, O_EXCL: prevent creation if it already exists UIUC - CS/ECE 438, Fall 2006

  34. File: Read #include <unistd.h> size_t read (int fd, void* buf, size_t cnt); • Read data from one buffer to file descriptor • Return: • Number of bytes read on success • -1 on error, sets errno • -1 on signal interrupt, sets errno to EINTR • Parameters: • fd: file descriptor • buf: buffer to read data from • cnt: length of buffer • Can return < cnt for sockets without reaching end of files • Returns 0 on end of file UIUC - CS/ECE 438, Fall 2006

  35. File: Write #include <unistd.h> size_t write (int fd, void* buf, size_t cnt); • Write data from file descriptor into buffer • Return: • Number of bytes written on success • -1 on error, sets errno • -1 on signal interrupt, sets errno to EINTR • Parameters: • fd: file descriptor • buf: buffer to write data to • cnt: length of buffer • Can return < cnt for sockets without reaching end of files • Returns 0 on end of file UIUC - CS/ECE 438, Fall 2006

  36. File: Close #include <unistd.h> int close(int fd); • Close a file • Return: • 0 on success • -1 on error, sets errno • Parameters: • fd: file descriptor UIUC - CS/ECE 438, Fall 2006

  37. Multithreading-Safe Functions • Functions in original libraries • Use static storage • Not compatible with multiple threads • Reentrant versions • Dubbed <name>_r • Pass state back and forth to routine • Example: • Tokenizing a string using state local to the current thread char* strtok_r (char* s, const char* delim, char** last_p); UIUC - CS/ECE 438, Fall 2006

  38. Multithreading-Safe Functions char* strtok_r (char* s, const char* delim, char** last_p); • Tokenizing a string using state local to the current thread • Used to split up words in English text, for example, or commands and arguments given in a command line • Parameters: • s: The string on the first call, NULL afterwards • delim: Characters that separate the tokens • last_p: Storage for a pointer to the end of the last token UIUC - CS/ECE 438, Fall 2006

  39. Example • Server similar to last MP • Push server • Client-server connection remains open • Server pushes new data • Use pthreads • Main thread • Accepts new client connections • Spawns child thread for each client • Child threads • Parses client requests • Constructs response • Checks for file modification • Pushes file when necessary UIUC - CS/ECE 438, Fall 2006

  40. Set up server TCP socket Initialize thread attributes structure Spawn a child thread to handle new connection Wait for a connection Create a thread specific data structure Example: Server Thread Flow Chart Start UIUC - CS/ECE 438, Fall 2006

  41. Push thread cleanup function Wait for 1 second or until client send request n Need to send? Got client request? Send file to client y n y Success in send? Request valid? Store file name and reset send time y y n n Done Pop and execute thread cleanup function Example: Client Thread Flow Chart Start UIUC - CS/ECE 438, Fall 2006

  42. Makefile CFLAGS = -g -D_REENTRANT –Wall OFILES = pushy.o push_thr_code.o read_line.o \\ print_error.o all: pushy pushy: ${OFILES} gcc -g -o $@ ${OFILES} – lpthread -lxnet %.o: %.c gcc -c ${CFLAGS} -O9 -o $@ $< clean: rm -f pushy query *.o *~ UIUC - CS/ECE 438, Fall 2006

  43. set_up_server_socket static int set_up_server_socket (u_short port) { int fd; /* server socket file descriptor */ int yes = 1; /* used for setting socket options */ struct sockaddr_in addr; /* server socket address */ /* Create a TCP socket. */ if ((fd = socket (PF_INET, SOCK_STREAM, 0)) == -1) { perror ("set_up_server_socket/socket"); return -1; } /* Allow port reuse with the bind below. */ if (setsockopt (fd, SOL_SOCKET, SO_REUSEADDR, (char*)&yes, sizeof (yes)) == -1) { perror ("set_up_server_socket/setsockopt"); return -1; } UIUC - CS/ECE 438, Fall 2006

  44. set_up_server_socket /* Set up the address. */ bzero (&addr, sizeof (addr)); addr.sin_family = AF_INET; /* Internet address */ addr.sin_addr.s_addr = INADDR_ANY; /* fill in local IP address */ addr.sin_port = htons (port); /* port specified by caller*/ /* Bind the socket to the port. */ if (bind (fd, (struct sockaddr*)&addr, sizeof (addr)) == -1) { perror ("set_up_server_socket/bind"); return -1; } /* Listen for incoming connections (socket into passive state). */ if (listen (fd, BACKLOG) == -1) { perror ("set_up_server_socket/listen"); return -1; } /* The server socket is now ready. */ return fd; } UIUC - CS/ECE 438, Fall 2006

  45. wait_for_connections static void wait_for_connections (int fd){ pthread_attr_t attr; /* initial thread attributes */ thread_info_t* info; /* thread-specific connection information */ int len; /* value-result argument to accept */ pthread_t thread_id; /* child thread identifier */ /* Signal a bug for invalid descriptors. */ ASSERT (fd > 0); /* Initialize the POSIX threads attribute structure. */ if (pthread_attr_init (&attr) != 0) { fputs ("failed to initialize pthread attributes\n", stderr); return; } /* The main thread never joins with the children. */ if (pthread_attr_setdetachstate (&attr, PTHREAD_CREATE_DETACHED) != 0) { fputs ("failed to set detached state attribute\n", stderr); return; } UIUC - CS/ECE 438, Fall 2006

  46. wait_for_connections /* Use an infinite loop to wait for connections. For each connection, create a structure with the thread-specific data, then spawn a child thread and pass it the data. The child is responsible for deallocating the memory before it terminates. */ while (1) { /* Create a thread information structure and initialize fields that can be filled in before a client contacts the server. */ if ((info = calloc (1, sizeof (*info))) == NULL) { perror ("wait_for_connections/calloc"); return; } info->fname = NULL; info->last_sent = (time_t)0; UIUC - CS/ECE 438, Fall 2006

  47. wait_for_connections /* Wait for a client to contact the server. */ len = sizeof (info->addr); if ((info->fd = accept (fd, (struct sockaddr*)&info->addr, &len)) == -1) { perror ("accept"); return; } /* Create a thread to handle the client. */ if (pthread_create (&thread_id, &attr, (void* (*) (void*))client_thread, info) != 0) { fputs ("failed to create thread\n", stderr); /* The child does not exist, the main thread must clean up. */ close (info->fd); free (info); return; } } } UIUC - CS/ECE 438, Fall 2006

  48. client_thread void client_thread (thread_info_t* info) { /* Check argument. */ ASSERT (info != NULL); /* Free the thread info block whenever the thread terminates. Note that pushing this cleanup function races with external termination. If external termination wins, the memory is never released. */ pthread_cleanup_push ((void (*)(void*))release_thread_info, info); /* Loop between waiting for a request and sending a new copy of the current file of interest. */ while (read_client_request (info) == 0 && send_file_to_client (info) == 0); /* Defer cancellations to avoid re-entering deallocation routine (release_thread_info) in the middle, then pop (and execute) the deallocation routine.*/ pthread_setcanceltype (PTHREAD_CANCEL_DEFERRED, NULL); pthread_cleanup_pop (1); } UIUC - CS/ECE 438, Fall 2006

  49. client_has_data static int client_has_data (int fd) { fd_set read_set; struct timeval timeout; /* Check argument. */ ASSERT (fd > 0); /* Set timeout for select. */ timeout.tv_sec = CHECK_PERIOD; timeout.tv_usec = 0; /* Set read mask for select. */ FD_ZERO (&read_set); FD_SET (fd, &read_set); /* Call select. Possible return values are {-1, 0, 1}. */ if (select (fd + 1, &read_set, NULL, NULL, &timeout) < 1) { /* We can't check errno in a thread--assume nothing bad has happened. */ return 0; } /* Select returned 1 file descriptor ready for reading. */ return 1; } UIUC - CS/ECE 438, Fall 2006

More Related