1 / 57

Network Programming Intro to Distributed systems Fall 2013

Network Programming Intro to Distributed systems Fall 2013. Some material taken from publicly available lecture slides including Srini Seshan’s and David Anderson’s. L3-socket. Dongsu Han. Today’s Lecture. Lecture Transport protocols: UDP and TCP Version control using SVN

kueng
Download Presentation

Network Programming Intro to Distributed systems Fall 2013

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Network ProgrammingIntro to Distributed systemsFall 2013 Some material taken from publicly available lecture slides including SriniSeshan’s and David Anderson’s L3-socket Dongsu Han

  2. Today’s Lecture • Lecture • Transport protocols: UDP and TCP • Version control using SVN • TA-led programming session • Editor, socket programming • Debugging with gdb • Announcement • Programming Assignment 1 is out. (Due in two weeks--추석)

  3. Review of Previous Lectures • What is a distributed system? (L1) • Definition • Example • What are some of the important decisions made in the Internet architecture? (L2) • Narrow waist: Network provides minimal functionality (best-effort) • Why?: to connect existing networks that are heterogeneous • Stateless architecture: no per-flow state inside network • Why? • For survivability

  4. Transport Protocols • Lowest level end-to-end protocol. • Header generated by sender is interpreted only by the destination • Routers view transport header as part of the payload; they are not supposed to see the transport header. 7 7 6 6 5 5 Transport Transport IP IP IP Datalink 2 2 Datalink Physical 1 1 Physical router

  5. What does the transport layer do? • No per-flow (connection) state in the network • Transport layer maintains the connection state. Transport layer identifies an end-to-end connection. • How does the transport layer identify an end-point (application)?

  6. De-multiplexing: Port numbers • Network layer gets you to the host. • A port identifies the sending or receiving socket (application). • A socket is one endpoint of a two-way communication between two programs running on the network. A socket is bound to a port number so that the transport layer can identify the application that data is destined to be sent. • Identifying two end-points: An endpoint = <IP address, Port #> • Source endpoint = <SrcIPAddr, SrcPort> • destination endponit = <DestIPAddr, DestPort>

  7. Protocol and Port Demultiplexing • Multiple choices at each layer • Applications open a socket (identified by the port number) Port 80 Port 22 Port 80 Port 22 TCP UDP Network IP TCP/UDP IP Type Field =IP Protocol Field = TCP Port Number = 80 NET1 NET2 … NETn

  8. TCP/UDP TCP/UDP IP IP Server Server and Client Server and Client exchange messages over the network through a common Socket API Clients user space ports Socket API kernel space Ethernet Adapter Ethernet Adapter hardware

  9. Transport protocols • UDP • Provides only integrity and de-multiplexing • TCP adds…

  10. “No frills,” “bare bones” Internet transport protocol, “Best effort” service UDP segments may be: Lost, duplicated (bug, spurious retransmission at L2) Delivered out of order to app Connectionless: No handshaking between UDP sender, receiver Each UDP segment handled independently UDP: User Datagram Protocol [RFC 768] • Why is there a UDP? • No connection establishment (which can add delay) • Simple: no connection state at sender, receiver • Small header • No congestion control: UDP can blast away as fast as desired

  11. Often used for streaming multimedia apps (e.g., Skype) Loss tolerant Rate sensitive Other UDP uses (why?): DNS Reliable transfer over UDP Must be at application layer Application-specific error recovery UDP, cont. 32 bits Source port # Dest port # Length, in bytes of UDP segment, including header Checksum Length Application data (message) UDP segment format

  12. Transport protocols • UDP • Provides only integrity and de-multiplexing • TCP adds… • Connection-oriented • Reliable (error control) • Ordered byte stream (in-order delivery) • Bi-directional byte-stream • Flow and congestion controlled

  13. TCP • Protocol implemented entirely at the ends • Fate sharing • Abstraction provided • Connection oriented protocol (hand-shake or connection establishment) • In-order byte-stream • Implemented functionality (next lecture) • Flow control • Reliability (error control) • Reassembly (in-order delivery) • Congestion control

  14. Difference in abstraction • UDP • Recv(len) call gives you a packet (the payload part) • TCP • Connection establishment (hand-shake)  Will see in the TA led session. • Recv(len) call give you a sequence of bytes len len UDP segment (packet) UDP segment (packet) Application send() len len Application Recv() Byte stream Byte stream Incoming stream

  15. TCP– What goes on underneath Packet (9~11) Packet (1~7) Packet (8~10) Byte stream Receive buffer Packet (8~10) I need seq# 8.

  16. Example UDP applications Multimedia, voice over IP • Postal Mail • Single mailbox to receive letters • Unreliable  • Not necessarily in-order delivery • Letters sent independently • Must address each reply User Datagram Protocol(UDP): An Analogy • UDP • Single socket to receive messages • No guarantee of delivery • Not necessarily in-order delivery • Datagram – independent packets • Must address each packet

  17. Example TCP applications Web, Email, Telnet Transmission Control Protocol (TCP): An Analogy TCP • Reliable – guarantee delivery • Byte stream – in-order delivery • Connection-oriented – single socket per connection • Setup connection followed by data transfer • Telephone Call • Guaranteed delivery • In-order delivery • Connection-oriented • Setup connection followed by conversation

  18. Revision control & make

  19. Overview • Revision control systems • Motivation • Features • Subversion primer • Make • Simple gcc • Make basics and variables • Testing • Useful Unix commands

  20. Dealing with large codebases • Complications • Code synchronization • Backups • Concurrent modifications • Solution: Revision Control System (RCS) • Store all of your code in a repository on a server • Metadata to track changes, revisions, etc… • Also useful for writing books, papers, etc…

  21. Basic RCS features (1/2) • Infinite undo • go back to previous revisions • Automatic backups • all your code ever, forever saved • Changes tracking • see differences between revisions

  22. Basic RCS features (2/2) • Concurrency • Multiple people working at the same time • Snapshots • Save stable versions on the side (i.e. handin) • Branching • Create diverging code paths for experimentation

  23. Typical RCS workflow • Create repository on RCS server • Checkout the repository to your local machine • Work locally: create, delete and modify files • Update the repository to check for changes other people might have made • Resolve any existing conflicts • Commit your changes to the repository

  24. RCS implementations • Revision Control System (RCS) • Early system, no concurrency or conflict resolution • Concurrent Versions System (CVS) • Concurrency, versioning of single files • Subversion (SVN) • File and directory moving and renaming, atomic commits

  25. Concurrent edits (1/2) Carnegie File v1 Mellon Both check out file Edits (lines 30-40) Edits (lines 1-20) commit File v2 Update Merged automatically!

  26. Concurrent edits (2/2) Carnegie File v1 Mellon Both check out file Edits (lines 10-20) Edits (lines 1-20) commit File v2 Update Conflict needing manual intervention!

  27. Resolving conflicts • When changes can’t be merged automatically • SVN gives you 3 files: • file.mine : your file • file.rx : the remote file • file : original file with marked conflicts • You can • Discard your changes • Discard others’ changes • Go over each conflict and arbitrate as appropriate

  28. Interacting with SVN • Command line • Readily available in andrew machines • Graphical tools • Easier to use • Subclipse for Eclipse gives IDE/SVN integration • tortoiseSVN

  29. 실습 server • IP: 143.248.141.52 ~ 143.248.141.59 (8대) • ID: your student id (e.g., 20110111) • Password: ee324EE#@$

  30. Svn password • 20060584 28482 • 20071122 6534 • 20081275 51085 • 20090058 50346 • 20090276 54586 • 20100120 23860 • 20100137 29297 • 20100441 17889 • 20100827 30690 • 20100967 7532 • 20110343 4967 • 20110456 9139 • 20110773 8574 • 20111047 36671

  31. Command line SVN example (1/2) $ svn co –-username=“<<student id>>” https://ina.kaist.ac.kr/svn/test $ cd test/trunk $ vim <<student id>>.txt $ vim <<student id>>. $ svn add <<student id>>.txt $ svn commit -m 'adding a text file with my student id‘ $ svn update $ cd ../ $ svncp trunk tags/<<student id>>_submission $ svn commit -m 'making a tag of the trunk for submission!‘ $ echo -e ”<<student id>>" >> student.txt $ svn commit –m “added my student id”

  32. Command line SVN example (2/2) Revision control lets you note (and then see) what you changed: > svn log student.txt r986 | ntolia | 2006-08-01 17:13:38 -0400 (Tue, 01 Aug 2006) | 6 lines This allows the sp to get rid of chunks early before a transfer is complete. Useful when a file is requested in-order and the file size > mem cache size > svn diff -r 1:2 student.txt Index: file =================================================================== --- file (revision 1) +++ file (revision 2) @@-1,2+1,3@@ This isatestfile -It startedwithtwolines +It nolongerhastwolines +it hasthree

  33. General SVN tips • Update, make, test, only then commit • Merge often (svn up) • Comment commits • Avoid commit races • Modular design avoids conflicts

  34. Know more • subversion.tigris.org for SVN software & info • svnbook.red-bean.com for SVN book

  35. Makefile • You are on your own.

  36. Make • Utility for executable building automation • Saves you time and frustration • Helps you test more and better

  37. Simple gcc If we have files: • prog.c: The main program file • lib.c: Library .c file • lib.h: Library header file % gcc -c prog.c -o prog.o % gcc -c lib.c -o lib.o % gcc lib.o prog.o -o binary

  38. gcc flags • Useful flags • -g: debugging hook • -Wall: show all warnings • -Werror: treat warning as errors • -O0, -O1, -O2, -O3: optimization level • -DDEBUG: macro for DEBUG (#define DEBUG) • Avoid using dangerous optimizations that could affect correctness

  39. More gcc • gcc -g -Wall -Werror -c prog.c -o prog.o • gcc -g -Wall -Werror -c lib.c -o lib.o • gcc -g -Wall -Werror lib.o prog.o -o binary • This gets boring, fast!

  40. Makefile basics • Build targets • target: dependency1 dependency2 ... • unix command (start line with TAB)‏ • unix command

  41. Makefile example binary: lib.o prog.o gcc -g -Wall lib.o prog.o -o binary lib.o: lib.c gcc -g -Wall -c lib.c -o lib.o prog.o: prog.c gcc -g -Wall -c prog.c -o prog.o clean: rm *.o binary

  42. Makefile variables (1/7) • Variables • CC = gcc • CFLAGS = -g -Wall -Werror • OUTPUT = binary

  43. Makefile variables (2/7) binary: lib.o prog.o gcc-g -Wall lib.o prog.o -o binary lib.o: lib.c gcc-g -Wall -c lib.c -o lib.o prog.o: prog.c gcc-g -Wall -c prog.c -o prog.o clean: rm *.o binary

  44. Makefile variables (3/7) CC = gcc CFLAGS = -g -Wall OUTPUT = binary $(OUTPUT): lib.o prog.o $(CC) $(CFLAGS) lib.o prog.o -o binary lib.o: lib.c $(CC) $(CFLAGS) -c lib.c -o lib.o prog.o: prog.c $(CC) $(CFLAGS) -c prog.c -o prog.o clean: rm *.o $(OUTPUT)‏

  45. Makefile variables (4/7) CC = gcc CFLAGS = -g -Wall OUTPUT = binary $(OUTPUT): lib.o prog.o $(CC) $(CFLAGS) lib.o prog.o -o binary lib.o: lib.c $(CC) $(CFLAGS) -c lib.c -o lib.o prog.o: prog.c $(CC) $(CFLAGS) -c prog.c -o prog.o clean: rm *.o $(OUTPUT)‏

  46. Makefile variables (5/7) CC = gcc CFLAGS = -g -Wall OUTPUT = binary OBJFILES = lib.o prog.o $(OUTPUT): $(OBJFILES)‏ $(CC) $(CFLAGS) $(OBJFILES) -o binary lib.o: lib.c $(CC) $(CFLAGS) -c lib.c -o lib.o prog.o: prog.c $(CC) $(CFLAGS) -c prog.c -o prog.o clean: rm *.o $(OUTPUT)‏

  47. Makefile variables (6/7) CC = gcc CFLAGS = -g -Wall OUTPUT = binary OBJFILES = lib.o prog.o $(OUTPUT): $(OBJFILES)‏ $(CC) $(CFLAGS) $(OBJFILES)-o binary lib.o: lib.c $(CC) $(CFLAGS) -c lib.c -o lib.o prog.o: prog.c $(CC) $(CFLAGS) -c prog.c -o prog.o clean: rm *.o $(OUTPUT)‏

  48. Makefile variables (7/7) CC = gcc CFLAGS = -g -Wall OUTPUT = binary OBJFILES = lib.o prog.o $(OUTPUT): $(OBJFILES)‏ $(CC) $(CFLAGS) $(OBJFILES)-o binary %.o: %.c # $<: dependency (%.c)‏ # $@: target (%.o)‏ $(CC) $(CFLAGS) -c $< -o $@ clean: rm *.o $(OUTPUT)‏

  49. Simple Test Script (1/2) % ./server 6667 & % cat testfile.01 | ./testscript.py % cat testfile.02 | ./testscript.py % killall -9 server

  50. Simple Test Script (2/2) #/bin/sh echo “Starting server on port 6667.” ./server 6667 & SERVERPID = $! echo “Running test files.” cat testfile.01 | ./testscript.py cat testfile.02 | ./testscript.py echo “Killing server process.” kill $(SERVERPID)‏

More Related