1 / 148

Chapter 2

The Application Layer. Chapter 2. Goals of this Chapter. To understand common application protocols work Web (http) Email (smtp) FTP DNS P2P DHT (distributed hash table) To understand how the design alternatives for application layer networking protocols

andra
Download Presentation

Chapter 2

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. The Application Layer Chapter 2

  2. Goals of this Chapter • To understand common application protocols work • Web (http) • Email (smtp) • FTP • DNS • P2P • DHT (distributed hash table) • To understand how the design alternatives for application layer networking protocols • A network application runs on many hosts, it is a distributed application • This chapter discusses several designs of distributed applications

  3. Road Map • Application networking basics • Web • Email • FTP • DNS • P2P • DHP

  4. Road Map • Application networking basics • Web • Email • FTP • DNS • P2P • DHT

  5. Creating a network app write programs that run on (different) end systems communicate over network e.g., web server software communicates with browser software No need to write software for network-core devices Network-core devices do not run user applications applications on end systems allows for rapid app development, propagation application transport network data link physical application transport network data link physical application transport network data link physical

  6. An app-layer networking protocol defines Types of messages exchanged, e.g., request, response Message syntax: what fields in messages & how fields are delineated Message semantics meaning of information in fields Rules for when and how processes send & respond to messages Public-domain protocols: defined in RFCs allows for interoperability e.g., HTTP, SMTP Proprietary protocols: e.g., Skype

  7. Which application gets a newly arriving packet? IP: 74.125.115.99 Applications Operating System Transport Network Link Layer Web server SSH server SSH client Web browser Skype Link Layer Physical Layer IM Dest IP: 74.125.115.99

  8. Transport layer multiplexing: TCP I would like to communicate with 74.125.115.99 port 80 I would like to accept communication on port 80 IP: 128.174.13.63 IP: 74.125.115.99 Operating System Operating System Web server app Web browser app socket TCP port 23421 TCP port 80 Dest IP: 74.125.115.99 Source IP: 128.174.13.63 Dest port: 80 Source port: 23421 Network Network Link layer Link layer • An application is identified by the hosts IP addresses, transport protocol, and ports • A TCP connection is identified by the pair of IPs, the pair of ports, and the transport protocol

  9. Transport layer multiplexing: TCP I would like to communicate with 74.125.115.99 port 80 IP: 128.174.13.63 IP: 74.125.115.99 Operating System Operating System Web server app Web browser app socket socket Dest IP: 128.174.13.63 Source IP: 74.125.115.99 Dest port: 23421 Source port:80 TCP port 23421 TCP port 80 Network Network Link layer Link layer • An application is identified by the hosts IP addresses, transport protocol, and ports • A TCP connection is identified by the pair of IPs, the pair of ports, and the transport protocol

  10. Transport layer multiplexing: TCP IP: 128.174.13.63 IP: 74.125.115.99 Operating System Operating System Web server app Web browser app socket socket TCP port 23421 TCP port 80 Dest IP: 74.125.115.99 Source IP: 128.174.13.63 Dest port: 80 Source port: 23421 Network Network Link layer Link layer socket • An application is identified by the hosts IP addresses, transport protocol, and ports • A TCP connection is identified by the pair of IPs, the pair of ports, and the transport protocol

  11. Transport layer multiplexing: TCP IP: 128.174.13.63 IP: 74.125.115.99 I would like to send data: XXSFGFEWRV Operating System Operating System Web server app Web browser app socket socket Dest IP: 74.125.115.99 Source IP: 128.174.13.63 Protocol: TCP Dest port: 80 Source port: 23421 Data: XXSFGFEWRV Dest IP: 74.125.115.99 Source IP: 128.174.13.63 Protocol: TCP Dest port: 80 Source port: 23421 Data: XXSFGFEWRV TCP port 23421 TCP port 80 TCP port 80 Dest IP: 74.125.115.99 Source IP: 128.174.13.63 Protocol: TCP Dest port: 80 Source port: 23421 Data: XXSFGFEWRV Network Network Network Data: XXSFGFEWRV Link layer Link layer Dest IP: 74.125.115.99 Source IP: 128.174.13.63 Protocol: TCP Dest port: 80 Source port: 23421 Data: XXSFGFEWRV socket • An application is identified by the hosts IP address, transport protocols, and port • A TCP connection is identified by the pair of IPs, the pair of ports, and the transport protocol

  12. Transport layer multiplexing: UDP I would like to send/receive data over UDP port 23421 I would like to receive any data on UDP port 1401 IP: 128.174.13.63 IP: 74.125.115.99 Operating System Operating System IM server app IM client app socket socket UDP port 23421 UDP port 1401 Network Network Link layer Link layer • An application is identified by the hosts IP address, transport protocols, and port

  13. Transport layer multiplexing: UDP Send data to 74.125.115.99 port 1401 Data: xxadre IP: 128.174.13.63 IP: 74.125.115.99 Operating System Operating System IM Server app IM client app socket socket UDP port 23421 UDP port 1401 Dest IP: 74.125.115.99 Source IP: 128.174.13.63 Protocol: UDP Dest port: 1401 Source port: 23421 Data: xxadre Network Network Data: xxadre Link layer Link layer • An application is identified by the hosts IP address, transport protocols, and port

  14. Transport layer multiplexing TCP UDP The application is identified by the port on which the application is listening. The application can use the source IP and port to further multiplex • Applications accept new connections based on the destination port • An client app that would like to communicate a server app must know • the IP of the host that is running the server app and • the port on which the server app is listening • When a connection is created, a socket is made. • Data is sent and received over this socket • This socket is identified by two pairs of IP-port, and the transport layer protocol, i.e., the tuple (IP, port, IP, port, transport layer)

  15. Project 1 – Send a Message via TCP and UDP • Make client program to send message to server and then wait for message from server • Make server program to wait for message from client and then respond with a message • UDP • Client • Set up socket • Send message • Wait for reply • If no reply comes, give up • If reply comes, print it • Server • Set up socket • Wait for message • When message arrives • Print message • Send reply • TCP • Client • Set up socket • Send message • Wait for reply • If no reply comes, give up • If reply comes, print it • Server • Set up socket • Wait for connection • When connect arrives, get socket for connection • Wait for message over connection socket • When message arrives • Print message • Send reply

  16. Steps in Visual Studio • Open visual studio • File -> new ->project • Win32 console application • Select name and directory • Leave defaults • Console app • Uncheck empty project • Checked precompiled header • Paste code from web page into source code • Build->build solution • The output window is in the lower middle frame • Might need to adjust the frames to see the output window • There will be many warnings, but, hopefully, no errors • the bottom of the output gives the directory where the program is located • Open two command windows • Change to the directory where the program is located • Run the program in each command windows • Testudp 1testudp 0 • Start the server first (the one with 1 as an argument

  17. What transport service does an app need? Data reliability some apps (e.g., audio) can tolerate some loss other apps (e.g., file transfer, telnet) require 100% reliable data transfer Timing some apps (e.g., Internet telephony, interactive games) require low delay to be “effective” • Throughput • some apps (e.g., multimedia) require minimum amount of throughput to be “useful” (i.e., in order for the user to gain utility) • other apps (“elastic apps”) make use of whatever throughput they get • Security • Encryption, data integrity, …

  18. Transport service requirements of common apps Application file transfer e-mail Web documents real-time audio/video stored audio/video interactive games instant messaging Time Sensitive no no not really yes, 100’s msec yes, few secs yes, 100’s msec yes and no Throughput elastic elastic some what elastic audio: 5kbps-1Mbps video:10kbps-5Mbps same as above few kbps up elastic Data loss no loss no loss no loss loss-tolerant loss-tolerant loss-tolerant no loss

  19. Internet transport protocols services TCP service: connection-oriented:setup required between client and server processes reliable transport between sending and receiving process does not provide: timing, minimum throughput guarantees, or even when packets are transmitted flow control: sender won’t overwhelm receiver congestion control: throttle sender when network overloaded UDP service: No connection set-up needed unreliable data transfer between sending and receiving process Packets can be sent at any rate/time desired (but this might be cause considerable congestion) does not provide: flow control, congestion control, timing, throughput guarantee, or security

  20. Internet apps: application, transport protocols Application layer protocol SMTP [RFC 2821] Telnet [RFC 854] HTTP [RFC 2616] FTP [RFC 959] HTTP (eg Youtube), RTP [RFC 1889] SIP, RTP, proprietary (e.g., Skype) Underlying transport protocol TCP TCP TCP TCP TCP UDP typically UDP Application e-mail remote terminal access Web file transfer streaming multimedia Internet telephony

  21. Road Map • Application basics • Web • Email • FTP • DNS • P2P • DHT

  22. Web and HTTP www.someschool.edu/someDept/pic.gif path name host name • Web page consists of objects • Object can be HTML file, JPEG image, Java applet, audio file,… • Web page consists of base HTML-file which includes several referenced objects • The browser first requests the base file • The base file specifies text and URLs of objects • The browser requests these objects, where ever they are (not always on the same server) • HTTP is used to request the base file and all the other files • Note, that HTTP can be used for other applications besides web • Each object is addressable by a URL • Example URL:

  23. HTTP overview HTTP: hypertext transfer protocol Web’s application layer protocol client/server model client: browser that requests, receives, “displays” Web objects server: Web server sends objects in response to requests HTTP request PC running Explorer HTTP response HTTP request Server running Apache Web server HTTP response Mac running Navigator

  24. HTTP overview (continued) Uses TCP: client initiates TCP connection (creates socket) to server, port 80 server accepts TCP connection from client HTTP messages (application-layer protocol messages) exchanged between browser (HTTP client) and Web server (HTTP server) TCP connection closed HTTP is “stateless” server maintains no information about past client requests aside • Protocols that maintain “state” are complex! • past history (state) must be maintained • if server/client crashes, their views of “state” may be inconsistent, must be reconciled

  25. HTTP connections Nonpersistent HTTP At most one object is sent over a TCP connection. Persistent HTTP Multiple objects can be sent over single TCP connection between client and server.

  26. Nonpersistent HTTP Suppose user enters URL www.someSchool.edu/someDepartment/home.index 1a. HTTP client initiates TCP connection to HTTP server (process) at www.someSchool.edu on port 80 (contains text, references to 10 jpeg images) 1b. HTTP server at host www.someSchool.edu waiting for TCP connection at port 80. “accepts” connection, notifying client 2. HTTP client sends HTTP request message (containing URL) into TCP connection socket. Message indicates that client wants object someDepartment/home.index 3. HTTP server receives request message, forms response message containing requested object, and sends message into its socket 5. HTTP client receives response message containing html file, displays html. Parsing html file, finds 10 referenced jpeg objects 4. HTTP server closes TCP connection. time 6.Steps 1-5 repeated for each of 10 jpeg objects

  27. Non-Persistent HTTP: Response time time to transmit file Definition of RTT: time for a small packet to travel from client to server and back. Response time: • one RTT to initiate TCP connection • one RTT for HTTP request and first few bytes of HTTP response to return • file transmission time total = 2RTT + data transmit time 10 objects require 20RTT+10 data transmit times initiate TCP connection RTT request file RTT file received time time

  28. Persistent HTTP Suppose user enters URL www.someSchool.edu/someDepartment/home.index 1a. HTTP client initiates TCP connection to HTTP server (process) at www.someSchool.edu on port 80 (contains text, references to 10 jpeg images) 1b. HTTP server at host www.someSchool.edu waiting for TCP connection at port 80. “accepts” connection, notifying client 2. HTTP client sends HTTP request message (containing URL) into TCP connection socket. Message indicates that client wants object someDepartment/home.index 3. HTTP server receives request message, forms response message containing requested object, and sends message into its socket 5. HTTP client receives response message containing html file, displays html. Parsing html file, finds 10 referenced jpeg objects, requests these 10 references 4. HTTP server sends each of the 10 objects. time

  29. (Non) Persistent HTTP: Response time time to transmit file time to transmit file time to transmit file time to transmit file Non-persistent Persistent initiate TCP connection initiate TCP connection RTT RTT request file request file RTT RTT base received base received initiate TCP connection Request files RTT RTT request file Object 1 received Object 2 received RTT Object 1 received 10 objects require 20RTT+10 data transmit times 10 objects require 3RTT+10 data transmit times

  30. (non) Persistent HTTP Advantages of persistent HTTP are only valid if the objects are the same server. Usually some objects are on the server, but many are not Instead of using a single persistent HTTP connection, a browser could use many non-persistent connections in parallel This is a bit unfair. As far as I know, persistent HTTP is usually not supported by the server or browser time to transmit file time to transmit file Non-persistent initiate TCP connection RTT request file RTT base received Initiate 5 TCP connection RTT request 5 files RTT Object 1 received Object 2 received Object 3 received Object 4 received Object 5 received

  31. HTTP request message • two types of HTTP messages: request, response • HTTP request message: • ASCII (human-readable format) request line (GET, POST, HEAD commands) GET /somedir/page.html HTTP/1.1 Host: www.someschool.edu User-agent: Mozilla/4.0 Connection: close Accept-language:fr (extra carriage return, line feed) header lines Carriage return, line feed indicates end of message

  32. HTTP request message: general format

  33. HTTP response message status line (protocol status code status phrase) HTTP/1.1 200 OK Connection close Date: Thu, 06 Aug 1998 12:00:15 GMT Server: Apache/1.3.0 (Unix) Last-Modified: Mon, 22 Jun 1998 …... Content-Length: 6821 Content-Type: text/html data data data data data ... header lines data, e.g., requested HTML file

  34. HTTP response status codes 200 OK request succeeded, requested object later in this message 301 Moved Permanently requested object moved, new location specified later in this message (Location:) 400 Bad Request request message not understood by server 404 Not Found requested document not found on this server 505 HTTP Version Not Supported In first line in server->client response message. A few sample codes:

  35. Trying out HTTP (client side) for yourself 1. Telnet to your favorite Web server: Opens TCP connection to port 80 (default HTTP server port) at www.eecis.udel.edu. Anything typed in sent to port 80 at www.eecis.udel.edu telnet www.eecis.udel.edu 80 • 2. Type in a GET HTTP request: By typing this in (hit carriage return twice), you send this minimal (but complete) GET request to HTTP server GET / HTTP/1.1 Host: www.eecis.udel.edu 3. Look at response message sent by HTTP server!

  36. Wireshark (ethereal) • Wireshark captures all packets that pass through the hosts interface • To run Wireshark , libpcap (linux) or winpcap (windows) must be installed. It comes with wireshark package • Then, run wireshark • Select Capture • Find the active interface • E.g., not generic dialup, nor vnp, nor packet scheduler, but wireless …. With IP address • Then select prepare • Let’s watch TCP packets on port 80 • Next to capture filter, enter TCP port 80 • Select update in realtime and autoscroll • Might need to enable or disable “capture in promiscuous mode” • Press start • Press close • Load www.eecis.udel.edu page in browser • Press stop in Wireshark • Find http request to 128.4.40.10. • Right click and select follow TCP stream

  37. User-server state: cookies Many major Web sites use cookies Four components: 1) cookie header line of HTTP response message 2) cookie header line in HTTP request message 3) cookie file kept on user’s host, managed by user’s browser 4) back-end database at Web site Example: Susan always access Internet always from PC visits specific e-commerce site for first time when initial HTTP requests arrives at site, site creates: unique ID entry in backend database for ID

  38. Cookies: keeping “state” (cont.) ebay 8734 usual http request msg Amazon server creates ID 1678 for user usual http response Set-cookie: 1678 create entry ebay 8734 amazon 1678 ebay 8734 amazon 1678 usual http request msg cookie: 1678 access cookie- specific action usual http response msg usual http response msg access usual http request msg cookie: 1678 cookie- spectific action client server cookie file backend database one week later:

  39. Cookies (continued) What cookies can bring: authorization shopping carts recommendations user session state (Web e-mail) aside • Cookies and privacy: • cookies permit sites to learn a lot about you • you may supply name and e-mail to sites • How to keep “state”: • protocol endpoints: maintain state at sender/receiver over multiple transactions • cookies: http messages carry state

  40. Web Serving Systems • LAMP Stack – very popular • Linux (OS) • Apache (web sever) • Receives http request and generates http response • The generation of response can involve many steps • Other servers are also popular: • nginx – open source, reverse proxy, load balancer, popularity: apache, microsoft, nginx, google • lighttpd – open source, small and fast. Good for high load. E.g., youtube, meeboo. Can handle 1000 hits per sec • MySQL or MariaDB (mySQL is oracle. MariaDB is a branch of mySQL, but not under oracle’s control) • Open source database • Very popular • PostgreSQL is also very popular • New, noSQL (not only SQL), databases are also playing a role • Casandra • mongoDB • Php • Php scripts make the html that is delivered by the web server to the client • Other application • Perl • Python • Java (with a tomcat server) • Facebook converts php to C and save 30% in speed (which means 30% in servers, which is a huge amount of money)

  41. Simple LAMP Topology

  42. Faster Topology Web cache: holds recently requested pages and generate response if the desired page is in cache. Otherwise, the request is forward to the web server Squid APC is a cache for compiled php code Apache+PHP+APC Cache for sql queries. A giant hash table. Give it a string (key) and if the response is in cache, it gets it. Otherwise, the request is sent to database and the result is also saved the response in cache The key can be anything. The programmer decides. memcached MySQL

  43. MySQL Replication MySQL master slave slave • All writes are to the master • Reads are from the master or the slaves • There is a slight delay from when the master is updated and the update is reflected by all slaves Where’d my edit go???

  44. Data Sharding MySQL MySQL group s1 English-language Wikipedia MySQL group s2 Next 19 biggest wikis MySQL group s3 Next 764 wikis

  45. Load Balancer Load Balancer • 1/3 of HTTP request to each server • Might keep request with the same cookie to the same server (sticky sessions) • Might decrypt SSL Apache Apache Apache Also load balances tomcat tomcat tomcat tomcat tomcat tomcat Load balancers also check if machines are healthy and will stop sending requests if they seem unhealthy

  46. Content distribution networks Even with a very fast server architecture, RTT is still large to some users • CDNs • allow you to put parts of your web page (e.g., logo, javascripts, audio, video, multicast live video) on servers that are close to the client • Act as web proxy. See next slides • If the content is always unique and changing (facebook), then the design of the CDN is more complicated Locations of Amazon’s “cloudFront” servers

  47. CDN example: AWS CloudFront • cloudFront machines are scattered around the world and are close to most people • Close in terms of RTT • Documents and media can be cached in a cloud • If the document is not in the CloudFront machine, the request is forwarded to the permanent storage/original server and given to the user and saved in cache • E.g., • Create a distribution for the image in cloudfront, and get a new url, e.g., http://mydomain.cloudfront.net • If image is at http://mydomain.com/images/pic1.jpg • Your web pages should refer to http://mydomain.cloudfront.net/images/pic1.jpg not to http://mydomain.com/images/pic1.jpg • A request to mydomain.cloudfront.com should go to the nearest cloudfront server farm. • How? • If the cache document changes, then different cloudfront machines will have different versions. Eventually they will be updated, usually within a few minutes, depending on the timeout value set

  48. Web caches (proxy server) user sets browser: Web accesses via cache browser sends all HTTP requests to cache object in cache: cache returns object else cache requests object from origin server, then returns object to client HTTP request HTTP request HTTP response HTTP response HTTP request HTTP response Goal: reduce network utilization by satisfying client request without involving original server origin server Proxy server client client origin server

  49. More about Web caching cache acts as both client and server typically cache is installed by ISP (university, company, residential ISP, e.g., satellite-based ISP) or as part of a CDN Why Web caching? reduce response time for client request reduce traffic on an institution’s access link. Internet dense with caches: enables “poor” content providers to effectively deliver content (similar objective as P2P file sharing)

  50. Caching example Assumptions average object size = 100,000 bits avg. request rate from institution’s browsers to origin servers = 15/sec delay from institutional router to any origin server and back to router = 2 sec Consequences utilization on LAN = 15% utilization on access link = 100% total delay = Internet delay + access delay + LAN delay = 2 sec + minutes + milliseconds origin servers public Internet 1.5 Mbps access link institutional network 10 Mbps LAN institutional cache

More Related