CS6320 – Performance more details

1. CS6320 – Performance more details L. Grewe

2. System Architecture Tier 1 Tier 2 Tier 3 DMS Client Application Server Database Server Web Server

3. Performance Desires and Approaches • Improving performance and reliability to provide • Higher throughput • Lower latency (i.e., response time) • Increase availability • Some Approaches • Scaling/Replication • How performance, redundancy, and reliability are related to scalability • Load balancing • Web caching

4. Where to Apply Scalability • To the network • To individual servers • Make sure the network has capacity before scaling by adding servers

5. An example…but, first Hardware Review • Firewall • Restricts traffic based on rules and can “protect” the internal network from intruders • Router • Directs traffic to a destination based on the “best” path; can communicate between subnets • Switch • Provides a fast connection between multiple servers on the same subnet • Load Balancer • Takes incoming requests for one “virtual” server and redirects them to multiple “real” servers

6. Switch: Conencting More than 2 Machines

7. Case Study: Retail eBusiness This is the initial design PROBLEM: site is growingand too many users- performance is inadequate

8. Solution - Scaling Scaling through Replication of systems

9. Initial Redesign Scaling mostly the webservers. Problem: still have one Entrance through firewall for clients. A bottleneck

10. The Redesign Again: Primary Redundant Last design: still bottleneck Coming in on one path …. Here we split into 2 “connected” Paths.

11. Performance, Redundancy, andScalability • Scale for performance • But what about redundancy? Site going down.

12. How to get rid of Single Points of Failure (SPOF): Problem: Last designif services to the singlegeographical networkgo down…site is down. Answer: Replicate indifferent geographical locations

13. Scaling Servers: Out or Up • Scale Out (Horizontal)..we saw this in previous design • Multiple servers • Add more servers to scale • Most commonly done with web servers • Scale Up (Vertical) • Fewer larger servers to add more internal resources • Add more processors, memory, and disk space • Most commonly done with database servers

14. Some Approaches to Scalability • Approaches • Farming • Cloning • RACS • Partitioning • RAPS • Load balancing • Web caching

15. Farming This is about the HW scaling • Farm - the collection of all the servers, applications, and data at a particular site. • Farms have many specialized services (i.e., directory, security, http, mail, database, etc.)

16. Simple Web Farm

17. Cloning This is about Service / SW replication • A service can be cloned on many replica nodes, each having the same software and data. • Cloning offers both scalability and availability. • If one is overloaded, a load-balancing system can be used to allocate the work among the duplicates. • If one fails, the other can continue to offer service.

18. Two Clone Design Styles • Shared Nothing is simpler to implement and scales IO bandwidth as the site grows. • Shared Disc design is more economical for large or update-intensive databases.

19. Reliable Array of Cloned Services (RACS) • RACS (Reliable Array of Cloned Services) • a collection of clones for a particular service • shared-nothing RACS • each clone duplicates the storage locally • updates should be applied to all clone’ s storage • shared-disk RACS (cluster) • all the clones share a common storage manager • storage server should be fault-tolerant • subtle algorithms need to manage updates (cache invalidation, lock managers, etc.)

20. Clones and RACS • can be used for read-mostly applications with low consistency requirements. • i.e., Web servers, file servers, security servers… • requirements of cloned services: • automatic replication of software and data to new clones • automatic request routing to load balance the work • route around failures • recognize repaired and new nodes

21. Some definitions - Partitions and Packs • Data Objects (mailboxes, database records, business objects,…) are partitioned among storage and server nodes. • For availability, the storage elements may be served by a pack of servers.

22. Partition • grows a service by • duplicating the hardware and software • dividing the data among the nodes (by object), e.g., mail servers by mailboxes • should be transparent to the application • requests to a partitioned service are routed to the partition with the relevant data • does not improve availability • the data is stored in only one place • partitions are implemented as a pack of two or more nodes that provide access to the storage

23. Taxonomy of Scaleability Designs

24. Reliable Array of Partitioned Services RAPS • RAPS (Reliable Array of Partitioned Services) • nodes that support a packed-partitioned service • shared-nothing RAPS, shared-disk RAPS • Update-intensive and large database applications are better served by routing requests to servers dedicated to serving a partition of the data (RAPS).

25. Some Approaches to Scalability • Approaches • Farming • Cloning • RACS • Partitioning • RAPS • Load balancing • Web caching

26. Load Balancing / Sharing

27. Load Management • Balancing loads (load balancer) can operate at different OSI layers • Round-robin DNS • Layer-4 (Transport layer, e.g. TCP) switches • Layer-7 (Application layer) switches

28. The 7 OSI (Open System Interconnection) Layers(a model of a network)

29. Load Balancing Strategies • Flat architecture • DNS rotation, switch based, MagicRouter • Hierarchical architecture • Locality-Aware Request Distribution

30. DNS Rotation - Round Robin Cluster

31. Flat Architecture - DNS Rotation • DNS rotates IP addresses of a Web site • treat all nodes equally • Pros: • A simple clustering strategy • Cons: • Client-side IP caching: load imbalance, connection to down node • Hot-standby machine (failover) • expensive, inefficient • Switching products • Cisco, Foundry Networks, and F5Labs • Cluster servers by one IP • Distribute workload (load balancing) • Failure detection • Problem • Not sufficient for dynamic content

32. Load Balance Idea 2: Switch-based Cluster

33. Flat Architecture - Switch Based • Switching products • Cluster servers by one IP • Distribute workload (load balancing) • i.e. round-robin • Failure detection • Cisco, Foundry Networks, and F5Labs • Problem • Not sufficient for dynamic content

34. Problems with DNS or Switch Load Balancing • Problems • Not sufficient for dynamic content • Adding/Removing nodes can be involved • Manual configuration required • limited load balancing in switch • Simple algorithms do not consider current loads

35. Load Sharing Strategies • Flat architecture • DNS rotation, switch based, MagicRouter • Hierarchical architecture • Locality-Aware Request Distribution

36. Hierarchical Architecture • Master/slave architecture • Two levels • Level I • Master: static and dynamic content • Level II • Slave: only dynamic

37. Hierarchical Architecture M/S Architecture

38. Hierarchical Architecture

39. Hierarchical Architecture • Benefits • Better failover support • Master restarts job if a slave fails • Separate dynamic and static content • resource intensive jobs (CGI scripts) runs by slave • Master can return static results quickly

40. Locality-Aware Request Distribution • Content-based distribution • Improved hit rates • Increased secondary storage • Specialized back end servers • Architecture • Front-end • distributes request • Back-end • process request

41. Load Sharing Strategies • Flat architecture • DNS rotation, switch based, MagicRouter • Hierarchical architecture • Locality-Aware Request Distribution

42. Locality-Aware Request Distribution Naïve Strategy

43. Some Approaches to Scalability • Approaches • Farming • Cloning • RACS • Partitioning • RAPS • Load balancing • Web caching

44. Web Caching

45. Web Proxy • Intermediate between clients and Web servers • It is used to implement firewall • To improve performance, proxy caching Proxy Client (browser) Web server With Caching

46. Web Architecture • Client (browser), Proxy, Web server Web server Proxy Firewall Client (browser)

47. Web Caching not only at Proxy Servers • Caching popular objects is one way to improve Web performance. • Web caching at clients, proxies, and servers. Web server Proxy Client (browser)

48. Advantages of Web Caching • Reduces bandwidth consumption (decrease network traffic) • Reduces access latency in the case of cache hit • Reduces the workload of the Web server • Enhances the robustness of the Web service • Usage history collected by Proxy cache can be used to determine the usage patterns and allow the use of different cache replacement and prefetching policies.

49. Disadvantages of Web Caching • Stale data can be serviced due to the lack of proper updating • Latency may increase in the case of a cache miss • A single proxy cache is always a bottleneck. • A single proxy is a single point of failure • Client-side and proxy cache reduces the hits on the original server.

50. Web Caching Issues • Cache replacement • Prefetching • Cache coherency • Dynamic data caching