The State of the Art in Locally Distributed Web-server Systems

1. V. Cardellini, E. Casalicchio, M. Colajanni, P.S. Yu IBM Research Report The State of the Art in Locally Distributed Web-server Systems

2. Contents • Introduction • Request routing mechanisms for cluster-based Web systems • Request routing mechanisms for distributed Web systems • Dispatching algorithms for cluster-based Web systems • Integrated dispatching algorithms • Placement of Web content and services • Summary • Research perspectives

3. Introduction (1/3) • Needs for scalable Web server systems • Web site is the only component to be controlled by a single organization • In interne, other components are not controllable by a single organization • The complexity of Web applications and services is growing • Web servers contribute for 40% of the delay in a Web transaction • As network speed improves faster than CPU, server will be bottleneck in Web • Approaches for scalable Web server systems • Scale-up • Hardware scale-up: more H/W resources • Software scale-up: improving Web server and O/S performance • Scale-out • Local scale-out: server nodes reside at a single network location • Global scale-out: server nodes are located at different geographical locations

4. Introduction (2/3) • Basic architecture of locally distributed Web systems • Appear as a single host (transparency) • Request routing mechanisms • At the client / network / DNS / server level

5. Introduction (3/3) • Classification of locally distributed architectures • Cluster-based Web systems: the server nodes are only visible by virtual IP • Also known as “Web cluster” or “Web farm” • Web switch (dispatcher) distributes the requests to the servers in LAN • Distributed Web systems: IP address of the server nodes are visible • One-to-many mapping in authoritative DNS server Cluster-based web system Distributed web system

6. Request Routing Mechanisms forCluster-based Web Systems (1/9) • Layer-4 vs. Layer-7 switching • Layer-4 switching (content-blind routing) • Web switch determines the target server when the clients asks for establishing a TCP/IP connection, upon the arrival of the first TCP SYN packet • Efficient and simple, but unaware of the content of the client request • Layer-7 switching (content-aware routing) • Web switch examines the HTTP request at application level and then replays it to the target server • Inefficient, but more sophisticated dispatching policies are possible • Two-way vs. one-way routing • Two-way: the target server returns its response to the Web switch • One-way: the target server responds directly to the client • More complex but efficient

7. Request Routing Mechanisms forCluster-based Web Systems (2/9) • Layer-4 two way architecture • Packet rewriting based on IP Network Address Translation (NAT) • Web switch rewrites inbound packet by changing VIP to the target server IP • Web switch rewrites outbound packet by changing the serve IP to VIP • Web switch recalculates IP and TCP header checksum for both packets

8. Request Routing Mechanisms forCluster-based Web Systems (3/9) • Layer-4 one-way architecture • Packet single-rewriting • Packet tunneling • Packet forwarding

9. Request Routing Mechanisms forCluster-based Web Systems (4/9) • Layer-4 one-way architecture (cont’d) • Packet single-rewriting • Web switch rewrites the destination IP address for each inbound packet • Web server replaces the source IP address with VIP for each outbound packet src 143.248.137.2 dst 202.103.106.5 143.248.137.2 172.16.0.3 202.103.106.5 143.248.137.2

10. Request Routing Mechanisms forCluster-based Web Systems (5/9) • Layer-4 one-way architecture (cont’d) • Packet tunneling (IP encapsulation) • Web switch encapsulates inbound IP packet • Web server strips the IP header off and response directly to the client using VIP • All servers should support IP tunneling and configured with VIP src dst payload client VIP encapsulation at Web switch payload src dst switch server client VIP IP network src dst payload switch server client VIP decapsulation at Web server src dst payload client VIP

11. Request Routing Mechanisms forCluster-based Web Systems (6/9) • Layer-4 one-way architecture (cont’d) • Packet forwarding (MAC address translation) • VIP address is shared by Web switch and all of the servers • Inbound packets reach the Web switch, even all nodes share VIP • All servers disable ARP (Address Resolution Protocol) mechanism • Web switch modifies MAC address of the inbound packet and retransmit it • Web server can response directly to the client since it shares VIP src dst payload client VIP Web switch server MAC … client VIP physical layer server MAC … client VIP Web server client VIP

12. Request Routing Mechanisms forCluster-based Web Systems (7/9) • Layer-7 two-way architecture • TCP gateway • Web switch maintains both connections for the client and the server • TCP splicing • Once the client-to-server binding is determined, the two connections are spliced • Packet rewriting for established connection

13. Request Routing Mechanisms forCluster-based Web Systems (8/9) • Layer-7 one-way architecture • TCP handoff • Web server hands off its endpoint of the TCP connection to the server • Multiple handoff, back-end forwarding • TCP connection hop (by Resonate) • Send encapsulated IP packet to the server • Web switch manages session information

14. Request Routing Mechanisms forCluster-based Web Systems (9/9) • Comparison of routing mechanisms for Web clusters • Two-way vs. one-way • Two-way is less scalable because of the overhead in the Web switch • Layer-4 switching vs. Layer-7 switching • Layer-7 switching is less scalable • Layer-7 switching gains benefits from content-aware dispatching approach routing data flow pros cons Layer-4 two-way content-blind inbound/outbound flexible, portable switching throughput TCP connection grain control one-way content-blind inbound simple, fast routing Network topology, TCP connection grain control Layer-7 two-way content-aware inbound/outbound simple, caching, server specialization, HTTP grain control, content partition, SSL session reuse switching bottleneck, Slowest routing one-way content-aware inbound server specialization, HTTP grain control, content partition, SSL session reuse switching bottleneck, Complex routing

15. Request Routing Mechanisms forDistributed Web Systems (1/2) • DNS routing mechanisms • Authoritative DNS server for the Web site selects a different server for every address resolution request: <IP, TTL> • Larger TTL: address-caching causes load imbalance • Smaller TTL: DNS traffic increases → response time increases • Authoritative DNS server does not work for client-level address caching • Non-cooperative DNS servers would override with their minimum TTL

16. Request Routing Mechanisms forDistributed Web Systems (2/2) • Web server routing mechanisms • Triangulation • The client continues to send packets to the first contacted server • The first server re-routes requests to the other server by IP tunneling • HTTP redirection • HTTP status code 301, 302 • Replication management, content-aware routing • Extra round-trip time • Most browsers does not handle redirection well • Bookmark to the redirected site • URL rewriting • The first contacted server changes dynamicallythe links for the embedded objects • Used by Content Delivery Networks (CDN) • Additional load on the redirecting Web server HTTP redirection

17. Dispatching Algorithms forCluster-based Web Systems (1/5) • Content-blind dispatching policies • Stateless (static) algorithms • Random, RR, static WRR • Client state aware algorithms • Use client IP address and TCP port numbers to assign a request to a server • Server state aware algorithms • Least loaded: least connections, fast response • WRR • Client and server state aware algorithms • Considering client affinity (ex. consecutive SSL connections, FTP) • Considerations for content-blind dispatching • Stateless algorithms: easy and fast dispatching, but poor performance • State-aware algorithms: extra overheads • “herd effect” : temporal saturation caused by long load information interval

18. Dispatching Algorithms forCluster-based Web Systems (2/5) • Content-aware dispatching policies • Cache affinity: improve reference locality in the server caches • Specialized servers: provide differentiated Web services • Load sharing: increase load sharing among the server nodes • Client affinity • HTTP/1.1 persistent connection • HTTP requests in a same TCP connection can be handled by different servers • Content-blind dispatching should control TCP connection granularity content-aware dispatching client-aware client and server aware cache affinity load sharing specialized servers client affinity cache affinity load sharing hash CAP SITA-E service partitioning session identifiers cache manager LARD

19. Dispatching Algorithms forCluster-based Web Systems (3/5) • Content-aware dispatching policies (cont’d) • Client state aware algorithms • Hash: static partitioning of the files • Improves cache hit rate, but can be used only for static contents • Ignores load sharing • Service partitioning: different contents are serviced by different servers • Size Interval Task Assignment with Equal load (SITA-E) • Separating light tasks from heavy tasks (considering content size only) • Client-Aware Policy (CAP) • Classifies Web requests into several classes • Estimates each request’s impact on Web server resources • Session identifier • Layer-7 session identifier is better than Layer-4 when Web proxies squeeze a large number of users into a small number of different IP addresses

20. Dispatching Algorithms forCluster-based Web Systems (4/5) • Content-aware dispatching policies (cont’d) • Client and server state aware algorithms • Locality-Aware Request Distribution (LARD) • Considers both locality and load balancing • Direct all requests for the same object to the same server below a threshold • Cache manager • Cache manager periodically gathers cache information of the servers node A node A node A requests node B node B node B Web switch node C node C node C Cache affinity only Load sharing only LARD

21. Dispatching Algorithms forCluster-based Web Systems (5/5) • Analysis of dispatching algorithms • Content-blind vs. content-aware dispatching • Content-aware dispatching can potentially outperform content-blind dispatching • Content-aware dispatching is expensive • Consider the heterogeneity of Web services • Dynamic contents, secure services, … • Server state information • Load indexes heavily impact on final performance of the dispatching algorithms • Input indexes, server indexes, forward indexes • Input indexes: can be easily computed, but have limited information • Server indexes: provide detailed information, but take longer time to acquire • Updating load information • Long update interval: poor performance because of the old load information • “herd effect” • Short update interval: more overhead

22. Integrated Dispatching Mechanisms • Improve scalability of Web systems • Solutions that improve Web cluster scalability • Multiple Web clusters + authoritative DNS • Web server accelerators • The caches which store frequently accessed Web objects • Web switch routes requests to the cache nodes • Solutions that improve Web switch scalability • Web switch is a potential bottleneck and a single point of failure • Layer-7 switch is more critical • Solution: multi-level switching • Layer-4 switch acts as frontend, and several Layer-7 switches are attached • Layer-4 switch + a distributor in each server node + a single dispatcher (Aron et al., UATC2000) • Multi-Node Load Balancing (MNLB) by CISCO • Separates packet-by-packet processing functions from sever selection

23. Placement of Web Contents and Services(1/2) • Distribution of static information • Distributed file system • Communication overhead, need to be optimized for Web services • Content replication • No extra communication overhead • High storage overhead • Hard to maintain consistency among replicated objects • Especially for highly volatile and frequently updated objects • Service partition • Layer-7 switching • High secondary storage scalability, server specialization • Load imbalance • Partition + partial replication can be a cost-effective solution • Static placement has potential weakness as the access pattern changes • Dynamic placement approaches

24. Placement of Web Contents and Services(2/2) • Distribution of dynamic services • Dynamic services are essential in modern Web sites • Use Web as a standard interface for underlying network services • “Web-based information systems”, or “Web-based enterprise applications” • CGI, JSP, ASP, PHP, XML, … • Multi-tier architecture for a Web cluster • Request routing and dispatching can beimplemented at different levels • Second-level Web switch can beintegrated into each Web server • Simple mechanisms are usedin the practical products • Due to complexity • RR, least loaded • Application-level caching

25. Summary • Locally distributed Web system architecture • Multi-node scale-out to improve Web system performance and scalability • Cluster-based Web system • Layer-4 vs. Layer-7 switching • Two-way vs. One-way routing • Distributed Web system • DNS based routing / Web server routing • Dispatching algorithms • Content-blind vs. content-aware routing • Use client and server state information to improve performance • Integrated dispatching mechanisms improves Web system scalability • Placement of web contents and services • Static contents: distributed file system, replication, partition • Dynamic services: multi-tier architecture

26. Research Perspectives • Web is becoming the standard interface for network services • Web clusters will be the basic architecture for Web-based info. system • Increasing complexity of Web-based information system will make performance of the Web clusters worse • Heterogeneity, security, middleware complexity, high availability, … • Research directions • End-to-end QoS support by Web server systems • Layer-7 scalability improvement • Optimal resource management in multi-tier architectures • Cooperation of multiple servers and multiple caches