Proxies

1. Proxies Herng-Yow Chen

2. Outline • Explain HTTP proxies, contrasting them to web gateways and illustrating how proxies are deployed. • Show some of the ways proxies are helpful. • How proxies are deployed in real networks and how traffic is directed to proxy servers. • How to configure your browser to use a proxy. • Demonstrate HTTP proxy requests, how they differ from server requests, and how proxies can subtly change the behavior of browsers.

3. Outline (cont.) • Explain how you can record the path of your messages through chains of proxy servers, using Via headers and the TRACE method. • Describe proxy-based HTTP access control. • Explain how proxies can interoperate between clients and servers, each of which may support different features and versions.

4. Web intermediaries • Web proxy servers are middlemen that fulfill transactions on the client’s behalf. • Without a web proxy, HTTP clients (e.g., a browser) talk directly to HTTP servers. • HTTP proxy servers are both web servers and web clients.

5. A proxy must be both a server and a client Proxies act like CLIENTto web servers. Proxies act like SERVERSto web clients. Request Request Response Response client server Proxy

6. Private and Shared Proxies • Public proxies (Shared proxies) • A proxy server can be shared among numerous clients. E.g., caching servers. • Private proxies • A proxy server can be dedicated to a single client. • E.g., some browser assistant products, as well as some ISP services, run small proxies directly on the user’s PC in order to extend browser features, improve performance, or host advertising for free ISP services.

7. Proxies Versus Gateways • Proxies connect two or more applications that speak the same protocol. • A gateway acts as a “protocol converter,” allowing a client to complete a transaction with a server, even when the client and server speak different protocols.

8. Proxies Versus Gateways (a)HTTP/HTTP Proxy HTTP HTTP Web proxy Web server Browser (b)HTTP/POP gateway HTTP POP Web/email gateway Browser Email server

9. Why Use Proxies? • Child filter • Document access controller • Security firewall • Web cache • Surrogate • Content router • Transcoder • Anonymizer

10. Child-safe Internet filter ok server Internet Child user DENY server Site contains adult content Child user School’s filtering proxy

11. Document access controller • Proxy servers can be used to implement a uniform access-control strategy across a large set of web servers and web resources and to create an audit trail. • All the access controls can be configured on the centralized proxy server, without requiring the access controls to be updated frequently on numerous web servers. • Maintain “blacklists” in order to identity and restrict access to objectionable content.

12. Centralized document access control General news General news Access control proxy Server A Client 1 To the Internet Internet Local area network Client 2 Secret financial data Intended request to server B blocked Client 3 Server B What is the password for the financial data?

13. Security firewall • Network security engineers often use proxy servers to enhance security. • Proxy servers restrict which application-level protocols flow in and out of an organization, at a single secure point in the network. • They also can provide hooks to examine the traffic, as used by virus-eliminating web and email proxies.

14. Security firewall Internet Server Client Filtering router Filtering router Server Client Virus Firewall proxy Server Client Firewall Firewall

15. Web cache • Proxy caches maintain local copies of popular documents and serve them on demand, reducing slow and costly Internet communication.

16. Web cache

17. Surrogate • Proxies can masquerade as web servers. • These so-called surrogates or reverse proxies receive real web server requests, but, unlike web servers, they many initiate communicate with other servers to locate the requested content on demand. • Surrogate (server accelerator) may be used to improve the performance of slow web servers for common content. • Surrogates also can be used in conjunction with content-routing functionality to create distributed networks of on-demand replicated content.

18. Surrogate Internet Surrogate client (also know as a reverse proxy or a server accelerator) server

19. Content router • Proxy servers can act as “content routers,” directing requests to particular web servers based on Internet traffic conditions and type of content. • Content routers also can be used to implement various service-level offerings. • For example, content routers forward requests to nearby replica caches (if the user has paid for higher performance), or route HTTP requests through filtering proxies (if the user has signed up for a filtering service).

20. Content routing

21. Transcoder • Proxy servers can modify the body format of content before delivering it to clients. This transparent translation between data representation is called transcoding. • For example, • convert GIF images into JPEG images, • compress files, • summarize web content as a compact form, • Language translation

22. Content transcoder • Players de Verano • Obtendra mchas sonrisas yguinios cuando use nuestras players de verano. • Blanco • Negro • Naranja amanecer • Summer Beach Shirts • You’ll get lots of smiles and winks when you wear out summer beach shirt. • White • Black • Sunrise orange Spanish- speaking client Summer Beach Shirts You’ll get lots of smiles and winks when you wear out summer beach shirt. White Black Sunrise orange Origin server Transcoding proxy Web-enabled mobile phone

23. Anonymizer • Anonymizer proxies provide heightened privacy and anonymity, by actively removing identifying information from HTTP messages. • Removed information, e.g., client IP, From header, Referer header, cookies, URI session IDs. • However, because identifying information is removed, the quality of the user’s browsing experience may be diminished, and some web sites may not function properly.

24. Anonymizer GET /something/file.html HTTP/1.0 Date: Thu, 25 Sep 2003 12:55:23 GMT User-Agent: Mozilla/4.0 (Windows NT 5.0) From: hychen@csie.ncnu.edu.tw Referer: http://www.csie.ncnu.edu.tw/tax-audits.html Cookie: profile="fotbal,litte beer" Cookie: income-braket="30k-45k" Anonymized message doesn't contain the common identifying information headers GET /something/file.html HTTP/1.0 Date: Thu, 25 Sep 2003 12:55:23 GMT User-Agent: Mozilla/4.0 Anonymizing proxy client server

25. Proxy server deployment • Egress proxy • Located at the exist points of local networks to control the traffic flow between LAN and the greater Internet. • E.g. Firewall protection, to reduce bandwidth charges and improve performance of Internet traffic. • Access (ingress) proxy • placed at ISP access points, processing the aggregate requests from the customers. E.g., ISPs use caching proxies to improve access performance. • Surrogates • Located at the edge of the network, in front of web servers, where they can field all of the requests directed at the web server and ask the web server for resources only when necessary. • Add security features to web servers, improve slower web server’s performance. • Network exchange proxy • Placed in the Internet peering exchanging points between networks, to alleviate congestion at Internet junctions through caching and to monitor traffic flows. (e.g. for national security concerns).

26. Private LAN egress proxy (a)Private Lan egress proxy Local network Internet client Proxy server client

27. ISP access proxy (b)ISP access proxy Internet client Proxy server client

28. Surrogate (c)Surrogate Local network Internet client Proxy server client

29. Network exchange proxy (d)Network exchange proxy Network 2 Network 1 Router Router client Proxy server

30. Proxy Hierarchies (e.g. 3-level) Proxies can be cascaded in chains called proxy hierarchies. This hierarchy is static. Proxy 1 Proxy 2 Proxy 3 client (Child of proxy 2) (Child of proxy 3 and parent of proxy 1) (parent of proxy 2) server

31. Dynamic hierarchy, changing for each request Dedicated cache server for specially-subscribed objects Caching proxy Access proxy client Internet Compressor proxy Web servers around the globe

32. Examples of dynamic parent selection • Load balancing • Geographic proximity routing • Protocol/type routing • Subscription-based routing

33. How Proxies Get Traffic (b) Network intercepts and redirects traffic to proxy (a)Client configured to use proxy Router client server client server proxy proxy (c) Surrogate stands in for web server (d) Server redirects HTTP requests to proxy proxy client client server server (Assuming the web server’s name) proxy

34. Client Proxy Settings • Manual configuration • Explicitly set a proxy to use. • Browser preconfiguration • The browser vendor manually preconfigures the proxy setting of the browser before delivering it to customers. • Proxy auto-configuration (PAC) • Provide a URI to a JavaScript proxy auto-configuration (PAC) files. • The browser fetches the JavaScript file and runs it to decide which proxy to use. • WPAD proxy discovery • Some browser support the Web Proxy Autodiscovery Protocol (WPAD), which automatically detects a “configuration server” from which the browser can download an auto-configuration file. (e.g. in I.E.)

35. PAC files • get http://proxy.ncnu.edu.tw/ncnu.pac • .pac suffix and the MIME type “application/x-ns-proxy-autoconfig.” • Each PAC file must define a function calledFindProxyForURL (url, host) that computes the proper proxy server to use for accessing the URI. • DIRECT // connections should be made directly • PROXY host:port // the specified proxy should be used

36. Web Proxy Autodiscovery Protocol (WPAD) • A client that implements the WPAD will: • Use WPAD to find the PAC URI. • Fetch the PAC file given in the URI. • Execute the PAC file to determine the proxy server. • Use the proxy server for requests.

37. WPAD (cont.) • WPAD uses a series of resource-discovery techniques, one by one until it succeeds, to determine the proper PAC file. • Multiple discovery techniques are used, because not all organizations can use all techniques. • Dynamic Host Discovery Protocol (DHCP) • Service Location Protocol (SLP) • DNS well-known hostnames • DNS SRV records • DNS service URIs in TXT records.

38. Proxy URLs Differ from Server URLs (a)Server request GET /index.html HTTP/1.0 User-agent: SuperBrowser v1.3 client Origin server (b)Explicit proxy request GET http://www.ncnu.edu.tw/index.html HTTP/1.0 User-agent: SuperBrowser v1.3 Proxy Server (Proxy explicitly configured) client Origin server

39. Proxy URLs Differ from Server URLs (c)Surrogate(reverse proxy) request GET /index.html HTTP/1.0 User-agent: SuperBrowser v1.3 Surrogate client (Server hostname points to the surrogate proxy) Origin server (d) Intercepting proxy request GET /index.html HTTP/1.0 User-agent: SuperBrowser v1.3 client Origin server Intercepting proxy

40. URL Resolution Without a Proxy (2a)Browser looks up host “ncnu” via DNS (2b)Failed , host unknown (3b)Browser looks up host “www.ncnu.edu.tw” via DNS (3c)Success!Get IP addresses back DNS server (1)User types”ncnu” into browser’s URI location window (4a)Browser tries to connect to IP addresses, one by one until connect successful (4b)Success;connection established (5a)Browser sends HTTP request (5b)Browser gets HTTP response (3a)The browser does auto-expansion, converting ”ncnu” into “www.ncnu.edu.tw” www.ncnu.edu.tw

41. URL Resolution with an Explicit Proxy GET http://ncnu/ HTTP/1.0 Proxy-connection: keep-Alive User-Agent: Mozilla/4.0 Host: ncnu Accept: */* Accept-encoding: gzip Accept-language: en Accept-charset: iso-8859-1,*,utf-8 (2a)Proxy is explicitly configured, so the browser looks up the address of the proxy server using DNS (2b)Success!Get proxy server IP addresses DNS server (4b)Proxy gets a partial hostname in the request, because the client did not auto-expand it. (1)User types ”ncnu” into browser’s URI location window (3a)Browser tries to connect to proxy (3b)Success;connection established proxy (3a)The browser does auto-expansion, converting”ncnu” into “www.ncnu.edu.tw” (4a)Browser sends HTTP request www.ncnu.edu.tw

42. URL Resolution with an Intercepting Proxy (2a) (2b) (3b) (5a) proxy (3c) DNS server (4a) (1) Interceptor (4b) (5a) www.ncnu.edu.tw (3a) Client

43. Tracing Messages Today, it’s not uncommon for web requests to go through a chain of two or more proxies on their way from the client to the server. It’s important to trace the flow of messages across proxies and to detect any problems. Surrogate cache bank ISP proxy Internet Web server client

44. The Via Header • Is used to track the forwarding of messages, diagnose message routing loops, and identify the protocol capabilities of all senders along the request/response chain. • Lists information about each intermediate node (proxy or gateway) through which a message passes. • Each time a message goes through another node, the intermediate node must be added to the end of the Via list.

45. The Via Header Request message (as received by server) GET /index.html HTTP/1.0 Accept: text/html Host: www.csie.ncnu.edu.tw Via: 1.1 proxy1.ncnu.edu.tw, 1.0 proxy2.ncnu.edu.tw proxy1.ncnu.edu.tw (HTTP/1.1) proxy2.ncnu.edu.tw (HTTP/1.0) client server

46. The response Via is usually the reverse of the request Via Request Via header Via: 1.1 A, 1.1 B, 1.1 C A B C client server Response Via header Via: 1.1 C, 1.1 B, 1.1 A

47. Via and gateways • Some proxies provide gateway functionality to servers that speak non-HTTP protocols. • The Via header records these protocol conversions, so HTTP applications can be aware of protocol capabilities and conversions along the proxy chain.

48. Via and gateways HTTP request message sent to proxy GET ftp://www.ncnu.edu.tw/pub/welcome.txt HTTP/1.0 FTP request FTP response proxy1.ncnu.edu.tw (HTTP/1.1) client HTTP response message HTTP/1.0 200 OK Date: Sun, 12 Dec 2003 21:01:59 GMT Via: FTP/1.0 proxy.ncnu.edu.tw (Traffic-Server/5.0.1-17882[cMsf]) Last-modified: sun, 12 Dec 2003 21:05:24 GMT Content-type: text/plain Hi there. This is an FTP server. www.ncnu.edu.tw

49. The Server and Via headers • The Server response header field describes the software used by the origin server. • Server: Apache/1.3.14 (UNIX) PHP/4.0.4 • Server: Netscape-Enterprise/4.1 • Server: Microsoft-IIS/5.0 • If a response message is being forwarded through a proxy, make sure the proxy does not modify the Server header. • The Server header is meant for the origin server. Instead, the proxy should add a Via entry.

50. Privacy and security implications of Via • There are some cases when we don’t want exact hostnames in the Via string. • For example, when a proxy server is part of a network firewall it should not forward the names and ports of hosts behind the firewall, because knowledge of network architecture behind a firewall might be of use to a malicious party. • Proxy can disable the Via node-name forwarding, replacing the hostname with an appropriate pseudonym. • For strong privacy requirements, a proxy may combine an ordered sequence of Via waypoint entries (with same protocol version) into a single, joined entry. • Via: 1.0 foo, 1.1 devirus.com, 1.1 access-logger.com • Via: 1.0 foo, 1.1 concealed-stuff