The Application Layer – HTTP and FTP

1. The Application Layer – HTTP and FTP Tahir Azim

2. Application Layer Protocols • QoS lectures postponed to next week • This week: Application Layer Protocols • FTP, HTTP, Bittorrent Materials reused from Philip Levis, David Mazieres (Stanford) and Vern Paxson (Berkeley)

3. File Transfer Protocol (FTP) • Allows a user to copy files to/from remote hosts • Client program connects to FTP server • … provides a login id and password • … allows the user to explore the directories • … and download and upload files with the server • A predecessor of the Web (RFC 959 in 1985) • Requires user to know the name of the server machine • … and have an account on the machine • … and find the directory where the files are stored • … and know whether the file is text or binary • … and know what tool to run to render and edit the file • That is, no URL, hypertext, and helper applications

4. How Do You Transfer Files Today? • HTTP - the usual Web transfer mechanism (http://) • FTP • ftp:// links in web pages (e.g.: in www.kernel.org) • sftp and scp • E.g.: to upload your project files to cms.niit.edu.pk • BitTorrent and other file-sharing software • Any others?

5. Why Study FTP? • Helps cement familiarity with text/status-code based protocols • Illustrates use of multiple concurrent connections • One for control (commands & replies) • Depending on command, can be additional one for data • Illustrates reversal of roles • For data connection, FTP user’s process can play the server role, FTP server can play the client role

6. Example commands • Authentication • USER: specify the user name to log in as • PASS: specify the user’s password • Exploring the files • LIST: list the files for the given file specification • CWD: change to the given directory • Downloading and uploading files • TYPE: set type to ASCII (A) or binary image (I) • RETR: retrieve the given file • STOR: upload the given file • Closing the connection • QUIT: close the FTP connection

7. Server Response Codes • 1xx: positive preliminary reply • The action is being started, but expect another reply before sending the next command. • 2xx: positive completion reply • The action succeeded and a new command can be sent. • 3xx: positive intermediate reply • The command was accepted but another command is now required. • 4xx: transient negative completion reply • The command failed and should be retried later. • 5xx: permanent negative completion reply • The command failed and should not be retried.

8. FTP Data Transfer • Separate data connection • To send lists of files (LIST) • To retrieve a file (RETR) • To upload a file (STOR) control data

9. Creating the Data Connection • Client acts like a server • Creates a socket • Assigned an ephemeral port number by the kernel • Listens on socket • Waits to hear from FTP server socket

10. Creating Data Connection (cont.) • But, the server doesn’t know the port number • So after starting to listen, client tells it to the server • Using the PORT command on the control connection • Server can tell the client a port to connect to using PASV or EPSV PORT <IP address, port #>

11. Why Out-of-Band Control? • Avoids need to mark the end of the data transfer • Data transfer ends by closing of data connection • Yet, the control connection stays up • Aborting a data transfer • Can abort a transfer without killing the control connection • … which avoids requiring the user to log in again • Done with an ABOR on the control connection • Third-party file transfer between two hosts • Data connection could go to a different host • … by sending a different client IP address to the server • e.g., a user can coordinate a transfer between two servers • But: this is rarely needed, and presents security issues

12. Example • Collected using: • tcpdump -s 0 -w ftp.trace host www.niit.edu.pk • ftp www.niit.edu.pk • Ethereal

13. Example blade1:~ # ftp www.niit.edu.pk Connected to www.niit.edu.pk. Server sends back the following: 220---------- Welcome to Pure-FTPd [privsep] [TLS] ---------- 220-You are user number 1 of 50 allowed. 220-Local time is now 18:54. Server port: 21. 220-This is a private system - No anonymous login 220 You will be disconnected after 15 minutes of inactivity.

14. Example continued… Name (www.niit.edu.pk:root): tahir USER tahir 331 User tahir OK. Password required Password: PASS tahirazim 230-User tahir has group access to: tahir 230 OK. Current restricted directory is / SYST 215 UNIX Type: L8 FEAT 211-Extensions supported: EPRT IDLE MDTM SIZE REST STREAM MLST type*;size*;sizd*;modify*;UNIX.mode*;UNIX.uid*;UNIX.gid*;unique*; MLSD ESTP PASV EPSV SPSV ESTA AUTH TLS PBSZ PROT 211 End.

15. Example continued… • ftp> pwd PWD 257 "/" is your current location • ftp> cd public_html/tinyos CWD public_html/tinyos 250 OK. Current directory is /public_html/tinyos

16. Example continued… • ftp> ls EPSV 229 Extended Passive mode OK (|||22710|) LIST 150 Accepted data connection 226-Options: -l 226 5 matches total Received from port 22710 of the server: -rw-r--r-- 1 tahir tahir 1143243 Feb 1 14:23 BVR-T2-0.1.tar.gz -rw-r--r-- 1 tahir tahir 535 Jan 18 20:03 Makefile -rw-r--r-- 1 tahir tahir 2788 Jan 18 20:03 RadioCountToLedsAppC.nc -rw-r--r-- 1 tahir tahir 5219 Jan 18 20:03 RadioCountToLedsC.nc -rw-r--r-- 1 tahir tahir 429056 Jan 18 20:03 nesC-wksp.ppt

17. Example continued… • ftp> get Makefile TYPE I 200 TYPE is now 8-bit binary SIZE Makefile 213 535 EPSV 229 Extended Passive mode OK (|||9442|) RETR Makefile 150 Accepted data connection 226-File successfully transferred 226 0.000 seconds (measured here), 1.11 Mbytes per second MDTM Makefile 213 20080118150308 From port 9442 of the server: COMPONENT=RadioCountToLedsAppC BUILD_EXTRA_DEPS = RadioCountMsg.py RadioCountMsg.class RadioCountMsg.py: RadioCountToLeds.h .mig python -target=$(PLATFORM) $(CFLAGS) -python-classname=RadioCountMsg RadioCountToLeds.h radio_count_msg -o $@ RadioCountMsg.class: RadioCountMsg.java .javac RadioCountMsg.java RadioCountMsg.java: RadioCountToLeds.h .mig java -target=$(PLATFORM) $(CFLAGS) -java-classname=RadioCountMsg RadioCountToLeds.h radio_count_msg -o $@ #CFLAGS += -I/home/Tahir/T2SerialT1. include $(MAKERULES)

18. Example continued… • ftp> exit QUIT 221-Goodbye. You uploaded 0 and downloaded 1 kbytes. 221 Logout.

19. HTTP • Server listens on a port (by default, 80) • On connection, waits for a request • Protocol (but not data) is in ASCII • Sends response, maybe closes connection (client can ask it to stay open)

20. Parsing a URL http://www.niit.edu.pk/~tahir/tcpip/index.html Protocol Host File path on host

21. HTTP Request Format • Request types: GET, PUT, POST, HEAD, DELETE • A trivial browser request: http://localhost:8000

22. A trivial browser request GET / HTTP/1.1 Host: localhost:8000 User-Agent: Mozilla/5.0 (Macinto ... Accept: text/xml,application/xm ... Accept-Language: en-us,en;q=0.5 Accept-Encoding: gzip,deflate Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 Keep-Alive: 300 Connection: keep-alive

23. Other useful header fields • Range: Request a partial range of data • Authorization: Present authorization credentials to a server (not HTTPS) • Proxy-Authorization: Present proxy credentials to a proxy server • Referer: URL of the web page the user was on, when the HTTP request was made

24. HTTP Response Format • 1xx codes: Informational • 2xx codes: Successes • 3xx codes: Redirection • 4xx codes: Client Error, 5xx codes: Server Error

25. Response from thenews.com.pk HTTP/1.0 200 OK Date: Fri, 14 Mar 2008 10:13:05 GMT Server: Microsoft-IIS/6.0 X-Powered-By: ASP.NET Content-Length: 54063 Content-Type: text/html Set-Cookie: ASPSESSIONIDAADTQQAR=PFKAADCBBDGBDDJLMKLFJNMI; path=/ Cache-Control: private X-Cache: MISS from micronet-proxy.niit.edu.pk X-Cache-Lookup: MISS from micronet-proxy.niit.edu.pk:8080 X-Cache: MISS from proxy-ptcl.niit.edu.pk X-Cache-Lookup: MISS from proxy-ptcl.niit.edu.pk:8080 Via: 1.0 micronet-proxy.niit.edu.pk:8080 (squid/2.6.STABLE5), 1.0 proxy-ptcl.niit.edu.pk:8080 (squid/2.6.STABLE5) Connection: keep-alive <html> <head> <title>The News - International - Friday, March 14, 2008</title> …

26. HTTP Performance • What matters most? • Different kinds of requests • Lots of small requests (loading a web page) • Big request (fetching a download) • Require different solutions

27. Small requests • Latency matters • Governed by RTT between hosts • Two major causes of delay: • Opening a TCP connection • Data response-request • Solutions: • Persistent connections • Pre-fetching • Others??

28. Browser request, revisited GET / HTTP/1.1 Host: localhost:8000 User-Agent: Mozilla/5.0 (Macinto ... Accept: text/xml,application/xm ... Accept-Language: en-us,en;q=0.5 Accept-Encoding: gzip,deflate Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 Keep-Alive: 300 Connection: keep-alive

29. Big requests • Problem is throughput on bottleneck links (usually edge links) • Use an HTTP proxy cache or mirror • Can also improve latency!

30. Stale caches • Items in the cache can go stale (you don’t want to read yesterday’s paper) • Cache needs a way to conditionally ask for a document • Cache can issue a conditional GET (with an If-modified-since header) • Server can reply with a 304 Not Modified GET / HTTP/1.1 Host: www.niit.edu.pk If-modified-since: Wed, 2 April 2008 08:00:00

31. Client-server vs Peer-to-peer • Server can be a bottleneck • Download time can scale down O(n) with n clients • Scaling up server bandwidth can be expensive (CDNs) • Slashdotting/flash crowds • Peer-to-peer: get a bunch of end-hosts to collaboratively distribute content • A common peer-to-peer challenge is finding whom to collaborate with

32. Bittorrent