Fundamental Characteristics of Queues with Fluctuating Load

1. Fundamental Characteristics of Queues with Fluctuating Load VARUN GUPTA Joint with:

2. Motivation Requests Clients Server Farm

3. Motivation Requests Clients Server Farm

4. Motivation Requests Clients Server Farm

5. Motivation Requests Clients Server Farm

6. Motivation Requests Clients Server Farm

7. Motivation Requests Clients Server Farm

8. Motivation Requests Clients Server Farm

9. Motivation Requests Real World Fluctuating arrival and service intensities  Clients Server Farm

10. A Simple Model exp(H) High Load L H Low Load exp(L)

11. H H L L A Simple Model exp() High Load H,H   L,L Low Load exp() • Poisson Arrivals • Exponential Job Size Distribution • H/H> L/L • H>Hpossible, only need stability

12. L L 0 1 2 . . . L L       H H 0 1 2 . . . H H The Markov Chain Number of jobs L Phase H Solving the Markov chain provides no behavioral insight

13. H H   L L • N = Number of jobs in the fluctuating load system • Lets try approximating N using (simpler) non-fluctuating systems

14. H H L L Method 1   Nmix

15. H H L L Method 1 ½ + Nmix ½ Q: Is Nmix≈ N? A: Only when   0  ,

16. H H L L Method 2  

17. Method 2 avg(H,L) ≡ Navg avg(H,L) Q: Is Navg ≈ N? A: When    ,

18. Example H=1, H=0.99   L=1, L=0.01 0  E[Navg] = 1 E[Nmix] ≈ 49.5

19. Observations • Fluctuating system can be worse than non-fluctuating •   0 and    asymptotes can be very far apart E[Nmix] > E[Navg] E[Nmix]  E[Navg]

20. Questions • Is fluctuation always bad? • Is E[N] monotonic in ? • Is there a simple closed form approximation for E[N] for intermediate ’s? • How do queue lengths during High Load and Low Load phase compare? How do they compare with Navg? More than 40 years of research has not addressed such fundamental questions!

21. Not covered in this talk Please read paper. Outline • Is E[Nmix] ≥ E[Navg], always? • Is E[N] monotonic in ? • Simple closed form approximation for E[N] • Application: Capacity Planning • Stochastic orderings for the number of jobs seen by an arrival into an H phase, L phase

22. Prior Work ? But cubic equations have a close form solution… Transforms Matrix Analytic & Spectral Analysis Fluid/Diffusion Approximations • - Clarke • - Neuts • Yechiali and Naor • - P. Harrison • Adan and Kulkarni • - Massey • Newell • Abate, Choudhary, Whitt Numerical Approaches Involves solution of cubic Limiting Behavior Involves solution of cubic

23. Good luck understanding this!

24. H=1, H=0.99 E[Nmix] > E[Navg]    L=1, L=0.01 Asymptotics for E[N] (H<H) E[Nmix] E[N] E[Navg] High fluctuation a (switching rate) Low fluctuation

25. E[Navg] E[N] E[Nmix] a Asymptotics for E[N] (H<H) E[Nmix] E[N] E[Navg] a Q: Is this behavior possible? A: Yes • Agrees with our example (H= L) • Ross’s conjecture for systems with constant service rate: • “Fluctuation increases mean delay”

26. E[N] E[N] E[N] a a a Our Results (H-H) > (L-L) (H-H) = (L-L) (H-H) < (L-L) • Define the slacks during L and H as • sL = L - L • sH = H - H

27. Our Results E[N] E[N] E[N] a a a sH > sL sH = sL sH < sL • Define the slacks during L and H as • sL = L - L • sH = H - H • Not load but slacks determine the response times! KEY IDEA

28. Not covered in this talk Please read paper. Outline • Is E[Nmix] ≥ E[Navg], always? • Is E[N] monotonic in ? • Simple closed form approximation for E[N] • Application: Capacity Planning • Stochastic orderings for the number of jobs seen by an arrival into an H phase, L phase

29. Not covered in this talk Please read paper. Outline • Is E[Nmix] ≥ E[Navg], always? No • Is E[N] monotonic in ? • Simple closed form approximation for E[N] • Application: Capacity Planning • Stochastic orderings for the number of jobs seen by an arrival into an H phase, L phase

30. NL NH exp() H,H L,L exp() Notation • NH: Number of jobs in system during H phase • NL: Number of jobs in system during L phase • N = (NH+NL)/2

31. f g NL=f(g(NL)) Analysis of E[N] H,H First steps: • Note that it suffices to look at switching points • Express • NL= f(NH) • NH = g(NL) • The problem reduces to finding Pr{NH=0} and Pr{NL=0} L,L NL NH

32. f g H(L -L)0H+ L(H-H)0L- (L -L)(H-H) A + 2(A -A) A-A (A-A) (A-A) Where 0L = 0H = L(-1)(H-H) H(-1)(L-L) The simple way forward… H,H • Find the root  of a cubic (the characteristic matrix polynomial in the Spectral Expansion method) • Express E[N] in terms of  E[N] = L,L Difficult to even prove the monotonicity of E[N] wrt  using this! NL NH

33. Our approach (contd.) KEY IDEA • Express the first moment as E[N] = f1()r+f0()(1-r) • r is the root of a (different) cubic • r1 as 0 and r0 as 

34. r 1 0  Monotonicity of E[N] • E[N] = f1()r+f0()(1-r) • r is monotonic in  E[N] is monotonic in  • The cubic for r has maximum power of  as 2

35. Monotonicity of E[N] • E[N] = f1()r+f0()(1-r) • r is monotonic in  E[N] is monotonic in  • The cubic for r has maximum power of  as 2 r 1 c1 0  • Need at least 3 roots for  when r=c1 • but  has at most 2 roots

36. Monotonicity of E[N] • E[N] = f1()r+f0()(1-r) • r is monotonic in  E[N] is monotonic in  • The cubic for r has maximum power of  as 2 r c2 1 0  • Need at least 2 positive roots for  when r=c2 • but for r>1 product of roots is negative

37. Monotonicity of E[N] • E[N] = f1()r+f0()(1-r) • r is monotonic in  E[N] is monotonic in  • The cubic for r has maximum power of  as 2 r 1 0  • E[N] is monotonic in !

38. Not covered in this talk Please read paper. Outline • Is E[Nmix] ≥ E[Navg], always? No • Is E[N] monotonic in ? • Simple closed form approximation for E[N] • Application: Capacity Planning • Stochastic orderings for the number of jobs seen by an arrival into an H phase, L phase

39. Not covered in this talk Please read paper. Outline • Is E[Nmix] ≥ E[Navg], always? No • Is E[N] monotonic in ? Yes • Simple closed form approximation for E[N] • Application: Capacity Planning • Stochastic orderings for the number of jobs seen by an arrival into an H phase, L phase

40. Approximating E[N] KEY IDEA • Express the first moment as E[N] = f1()r+f0()(1-r) • r is the root of a (different) cubic • r1 as 0 and r0 as  • Approximate r by the root of a quadratic KEY IDEA

41. 9 Exact Approx. 7 5 3 1 10-5 10-4 10-3 10-2 10-1 100 10 Approximating E[N] E[N] H=L=1, H=0.95, L=0.2 

42. 9 Exact Approx. 7 5 3 1 10-5 10-4 10-3 10-2 10-1 100 10 Approximating E[N] E[N] H=L=1, H=0.95, L=0.2 

43. 18 Exact Approx. 14 10 6 2 10-2 10-1 100 10 Approximating E[N] E[N] H=L=1, H=1.2, L=0.2 

44. 18 Exact Approx. 14 10 6 2 10-2 10-1 100 10 Approximating E[N] E[N] H=L=1, H=1.2, L=0.2 

45. Not covered in this talk Please read paper. Outline • Is E[Nmix] ≥ E[Navg], always? No • Is E[N] monotonic in ? Yes • Simple closed form approximation for E[N] • Application: Capacity Planning • Stochastic orderings for the number of jobs seen by an arrival into an H phase, L phase

46. Not covered in this talk Please read paper. Outline • Is E[Nmix] ≥ E[Navg], always? No • Is E[N] monotonic in ? Yes • Simple closed form approximation for E[N] • Application: Capacity Planning • Stochastic orderings for the number of jobs seen by an arrival into an H phase, L phase

47. 2H H H H L L 2L L Application: Capacity Provisioning Scenario     Aim: To keep the mean response times same

48. 2H H 2H H 2L L 2L L Application: Capacity Provisioning Scenario     Question: What is the effect of doubling the arrival and service rates on the mean response time?

49. What happens to the mean response time when , are doubled in the fluctuating load queue? A: Halves B: Reduces by more than half C: Reduces by less than half D: Remains almost the same

50. What happens to the mean response time when , are doubled in the fluctuating load queue? A: Halves B: Reduces by more than half C: Reduces by less than half D: Remains almost the same