Load Balancing in Distributed Systems

1. Load Balancing in Distributed Systems Nalini Venkatasubramanian nalini@ics.uci.edu

2. Motivation • A given computer is overloaded • Must decrease load and maintain certain characteristics • Scalability • Performance • Throughput • Ideally, this should be done transparently • Solution: Load Balancing Global Distributed Systems and Multimedia

3. Introduction to Load Balancing • Distributed resource allocation • Can be thought of as “distributed scheduling” • Deals with distribution of processes among processors connected by a network • Handles issues such as deciding which process should be handled by a given processor • Can also be influenced by “distributed placement” • Important in data intensive environments and applications • Data placement may force process placement Global Distributed Systems and Multimedia

4. Load Balancing Relationships • Load Balancer • Manages resources • Resource assignment depends on policy or policies in effect • Client • Requests resources • Requests services Global Distributed Systems and Multimedia

5. Load Balancing Overhead • Satisfy client resource access without imposing large amounts of overhead • Performance • How well resources are managed • Efficiency • Cost of accessing and using a resource obtained through a load balancer Global Distributed Systems and Multimedia

6. Load Balancing Issues • When to migrate processes or forward requests • Which processor should be chosen to handle a given process or request • Should processes be moved off a computer • How should searching for lightly loaded computer be performed Global Distributed Systems and Multimedia

7. Load Balancing Issues (cont.) • When should load balancing decisions be made • What should be taken into account when making the above decisions • How should old data be handled • Should load balancing data be stored centrally, or in a distributed manner Global Distributed Systems and Multimedia

8. Load Balancing Issues (cont.) • Should computers make decisions together • What is the performance/overhead tradeoff incurred by load balancing • Prevention of overloading a lightly loaded computer Global Distributed Systems and Multimedia

9. Load Balancing Techniques • Basically two ways to perform load balancing • Statically • Resource is allocated once • Dynamically • Resource is allocated and managed (possibly dynamically reallocated) to ensure balanced load Global Distributed Systems and Multimedia

10. Static Load Balancing • Resource allocation is performed once • Once resource is allocated it remains allocated (for what duration??) • Scheduling decisions are made • Deterministically • Probabilistically Global Distributed Systems and Multimedia

11. Static Load Balancing – cont’d • Advantages • State generally need not be stored • Simplifies implementation • Less network traffic due to load balancing related messages • Disadvantages • Poor resource utilization • A given resource may be used much more than others • Does not adjust to fluctuations in the load • Possible for resource to become overloaded Global Distributed Systems and Multimedia

12. Static Load Balancing – cont’d • Example of Static Load Balancing • Forwarding processes/requests to a given computer based on dynamically assigned addresses (e.g. via DNS) • Web servers (e.g. CNN) • UCI host ea.uci.edu is load balanced Global Distributed Systems and Multimedia

13. Dynamic Load Balancing • Attempts to maintain a balanced load by managing resources while a resource is in use • May involve the following • Process migration • Disabling further access to a resource until a later time • Adding new resources “on-the-fly” Global Distributed Systems and Multimedia

14. Dynamic Load Balancing Strategies • Distributed versus non-distributed • Should load information be stored centrally or across several hosts • Simplicity versus overhead and reliability • Cooperative versus non-cooperative • Should decisions be made by a single load balancer or several • Globally managed resources versus locally managed resources Global Distributed Systems and Multimedia

15. Dynamic Load Balancing Strategies - cont’d • Adaptive versus non-adaptive • Should previous data effect scheduling decisions • Preemptive versus non-preemptive • Should a running process be preempted in favor of another process, or for migration to another resource Global Distributed Systems and Multimedia

16. Case Studies • Load Balancing for Web Servers • Load Balancing for Parallel Computers • Load Balancing for Multimedia Applications Global Distributed Systems and Multimedia

17. Multimedia Applications Electronic Commerce Video Servers Global Entertainment Network Web Servers Distance Learning Graphics Processing Global Distributed Systems and Multimedia Tele-medicine Requirements - Availability, Reliability, Quality-of-Service, Cost-effectiveness, Security

18. Multimedia Load Management • Primary focus • resource optimization across streams and resource management across servers. • Quality of Service • continuous delivery requirement • minor violations of performance requirements • Admission Control - resource reservation/negotiation • Media Delivery - Resource scheduling (CPU,Disk) • Resource Mgmt. Implies Admission Control • Caching, VCR Control, Server Selection, Data Placement Global Distributed Systems and Multimedia

19. Distributed MM Servers • Video Server Topology • Partitioned Server • Externally Switched • Fully Switched • Video File Placement • Heterogeneous Workload - large/small, hot/cold • Online Placement • Good placement is important • dynamic replication time-consuming, dynamic load-bal complementary Global Distributed Systems and Multimedia

20. Dynamic Load Balancing • Adapts to statistical fluctuations and changing access patterns • Dynamic Migration • Deals with poor initial placement • Replication • Dynamic Segment Replication • partial replication (quick response, less expensive) • Total Replication • on-demand vs. predictive Global Distributed Systems and Multimedia

21. Load Management of Distributed MM Servers • Adaptive Scheduling • Assigns requests to servers based on demand and load factors. • Invokes replication-on-demand, request migration • Predictive Placement • Invokes dereplication • Optimizations • Eager Replication • Lazy Dereplication Global Distributed Systems and Multimedia

22. A Scalable Video Server Architecture Distribution Network requests data Distribution Controller Data Source Data Source Data Source Tertiery Storage ... control Global Distributed Systems and Multimedia

23. Architectural View of a Networked MM System Qos Broker and Load Management System Node Manager Node Manager Node Manager Node Manager ... Local Data Streaming Global Distributed Systems and Multimedia

24. Resources in a Video Server Client Client Network Processing Module Communication Modules Data Manipulation Modules Storage Modules Global Distributed Systems and Multimedia

25. Load Placement Scenario Data Source S2 Data Source S1 Storage: 2 objects Bandwidth: 8 requests Storage: 8 objects Bandwidth: 3 requests Access Network ... Clients Global Distributed Systems and Multimedia

26. Characterizing Server Resource Usage • Ability to service a request on a server depends on: • resource available • characteristics of a request • Load factor(LF) for a request: • represents how far a server is from request admission threshold. LF (Ri, Sj) = max (Dbi/DBj , Mi/Mj , CPUi/CPUj , Xi/Xj) Global Distributed Systems and Multimedia

27. Adaptive Scheduling • When the broker receives a request Ri for a video object Vi : • Consider only data sources that have a copy of Vi. • Consider only data sources tha have sufficient resources to support Ri. • Chooser server for which LF (Ri, Sj) is a minimum. • If no such server exists • Reject request. • Perform replication-on-demand. • Perform request migration. Global Distributed Systems and Multimedia

28. Predictive Data Placement • Determines when, where and how many replicas of a video object. • Initiated periodically. • Results in an assignment of replicas to data sources. • Greedy algorithm that uses revenue generated as a metric. Global Distributed Systems and Multimedia

29. The Greedy Cost Placement Matrix PM(Vi, Sj) is the maximum revenue that can accrue from allocating Vi to Sj. Greedy heuristic: Map(Vi,Sj) = 1 if PM(Vi,Sj) = a b max(PM(Va,Sb)) Global Distributed Systems and Multimedia

30. Optimizations • To minimize the overhead of replication • Eager replication • Replication of video object in anticipation • Performed when server resources are free • Lazy Dereplication • Critical nature of storage resources • Mark reusable resources, reclaim disk space later • If disk blocks are not overwritten, can be reclaimed Global Distributed Systems and Multimedia

31. Life of a video object Global Distributed Systems and Multimedia

32. Performance Evaluation Policies Global Distributed Systems and Multimedia

33. Performance Evaluation - Startup Latencies Global Distributed Systems and Multimedia

34. Performance of the basic configuration p1 p2 p3 p4 Global Distributed Systems and Multimedia

35. Performance Evaluation - Varying Replication BW Global Distributed Systems and Multimedia

36. Performance Evaluation Summary • P1: entails high startup latency, requires high storage and replication bandwidth. • P2: Unacceptably poor performance. • P3: Similar performance to P4 in many cases. • At low transfer bandwidths, P4 outperforms P3. • P4: Performs well in all cases. Global Distributed Systems and Multimedia