1 / 30

Join-Idle-Queue: A Novel Load Balancing Algorithm for Dynamically Scalable Web Services

Join-Idle-Queue: A Novel Load Balancing Algorithm for Dynamically Scalable Web Services. Li Yu, Qiaomin Xie , Gabriel Kliot , Alan Geller, James R. Larus , Albert Greenberg IFIP Performance 2011 Best Paper Presented by Amir Nahir. Agenda. Queuing terminology and background:

garson
Download Presentation

Join-Idle-Queue: A Novel Load Balancing Algorithm for Dynamically Scalable Web Services

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Join-Idle-Queue: A Novel Load Balancing Algorithm for Dynamically Scalable Web Services Li Yu, QiaominXie, Gabriel Kliot, Alan Geller, James R. Larus, Albert Greenberg IFIP Performance 2011 Best Paper Presented by Amir Nahir

  2. Agenda • Queuing terminology and background: • M/M/N, Processor Sharing • Motivation • The algorithm • Some analysis • Results • Caveat: where has time gone?

  3. M/M/N • Single shared queue • Jobs wait in the queue • Whenever a server completes one job, it gets the next from the queue

  4. M/M/N • Pros: • Jobs arrive to the next server to become available • Cons: • Centralized (single point of failure, bottleneck) • Hidden overhead – the time is takes to access the queue to get the job

  5. Service Disciplines • FIFO (a.k.a FCFS) • Processor sharing

  6. FIFO vs. Processor Sharing • Analysis is very similar • But results are not quite the same • E.g., assume three jobs arrive at the system at time 0 • PS is currently seen as the “realistic” model Avg. time in system = 2 Avg. time in system = 3

  7. Web-Services: The User’s Experience • No one really tries to model the whole process as a single problem • Common component (unchanged by the research) are often neglected Server Scheduler

  8. The Main Motivation for the Paper • Reduce delays on the job’s critical path Server Scheduler Server Scheduler Scheduler

  9. The Join-Idle-Queue Algorithm: System Structure • Two-layer system: dispatchers (front-ends) and processors (back-ends, servers) • The ratio between servers and dispatchers is denoted by r • No assumptions regarding processor discipline (can support PS, FIFO) • Each dispatcher has an I-queue • The I-queue holds servers (not jobs)

  10. The Join-Idle-Queue Algorithm: Dispatcher Behavior • Upon receiving a job from user: • If there are servers in the I-queue, dequeue first server and send job to it • Otherwise – send job to random server • This deteriorates system performance • This is termed primary load balancing

  11. The Join-Idle-Queue Algorithm: Server Behavior • Upon completing all jobs: • Choose a dispatcher • Two techniques are considered: Random and SQ(d) • Register in its I-queue • This is termed secondary load balancing

  12. The Join-Idle-Queue Algorithm at Work 1 2 4 1 2 3 4

  13. The Join-Idle-Queue Algorithm: Corner Case 1 Server 2 is busy processing a job while being registered as “idle” in one of the I-queues 2 1 2 3 4

  14. The Join-Idle-Queue Algorithm: Corner Case 2 2 2 Server 2 is reported as “idle” in more than one dispatcher 1 2 3 4

  15. JIQ Analysis: Some Notations • r – the ratio of servers to dispatchers • When is the algorithm expected to perform better, large r or small r?

  16. JIQ Analysis: Some Notations • pi – the probability that a server holds exactly i jobs • p0 – the probability that a server is idle • λi – the arrival rate of jobs to a server which holds exactly i jobs • λ0– the arrival rate of jobs to idle processors • ρ=λ/μ • Common notation in queuing

  17. Load Balancing Assertions • No matter how your balance the load: • p0 = 1 – λ • a λN dispatchers n servers

  18. Load Balancing: So Where Does the Wisdom Go? • It’s not about: increasing the probability that a server is idle • It’s about increasing the arrival rate to idle (and lightly loaded) servers • And from there,

  19. Theorem 1: Proportion of Occupies I-Queues • There’s a strong connection between idle servers and occupied I-queues • Jobs arrive at the system at rate λn • The proportion of idle servers is (1-λ)n • This proportion is equally distributed among the dispatchers, so the proportion of occupied I-queues is (1-λ)n/m = (1-λ)r

  20. Theorem 1: Proportion of Occupies I-Queues • On the other hand, the authors show that “server arrivals” to the I-queue do behave like a Poisson process (when n→∞) • Servers arrive at I-queues at rate ρ • There are ρmoccupied I-queues (on average) • And so the average I-queue length, under random secondary load balancing, is:

  21. Corollary 2: The Arrival Rate at Idle Servers (1) • Job arrival rate at the specific dispatcher is λn/m • A job has probability ρ to find an occupied I-queue • Average I-queue length is r(1-λ) 2 1 2 3 4

  22. Corollary 2: The Arrival Rate at Idle Servers (2) • Job arrival rate at servers is λ • A job has probability (1-ρ) to find an empty I-queue • Overall arrival rate at idle servers 2 1 2 3 4

  23. Corollary 2: The Arrival Rate at Non-Idle Servers • Job arrival rate at servers is λ • A job has probability (1-ρ) to find an empty I-queue • Arrival rate at busy servers is λ (1-ρ) 2 1 2 3 4 The arrival rate at idle servers is (r+1) times higher than the arrival rate at non-idle servers

  24. Proportion of Empty I-queues

  25. Results (Exponential Job Length)

  26. Job Length Distributions

  27. Sensitivity to Variance (PS)

  28. Affect of r on Performance

  29. Caveat: Scheduling Still Takes Time… • When the decision for the secondary load balancing takes place, the servers is not registered at any I-queue • At this time, performance is expected to degrade…. Scheduler Server Scheduler

More Related