Things I Thought I Knew About Queueing Theory, but was Wrong About: Part 1, Multiserver Queues

1. Things I Thought I Knew About Queueing Theory, but was Wrong About: Part 1, Multiserver Queues Alan Scheller-Wolf Tepper School of Business Carnegie Mellon University Joint work with Karl Sigman, Rein Vesilo, and Michele Dufalla Alan Scheller-Wolf Lunteren, The Netherlands

2. A Great Day As a graduate student, one of the greatest things that can happen is the following: Student: “I can prove that XXX is true!” Advisor: “That can’t be right, everyone knows YYY is true.” Student: “I know, but I can prove it.” Advisor: “There must be a bug, show me.” Time passes... Advisor: “Huh, you are right, XXX is true!” Alan Scheller-Wolf Lunteren, The Netherlands

3. My Goals • Take you through “my great day” as a student • Help you understand why everyone believed YYY • Help you understand, mathematically, why XXX is true • Help you see why, intuitively, XXX “has to be true” • Discuss what general insights this gives Alan Scheller-Wolf Lunteren, The Netherlands

4. Mean Delay in GI/GI/1 Queues Single Server Queue • Jobs are served in a FIFO (FCFS) fashion • Delay: Time customers spend in queue • Question: What are conditions for mean (stationary) delay to be finite? • Kiefer and Wolfowitz (K&W, 1956) • r =l/m < 1 • E[D] finite iff E[S2] finite i.i.d. arrivals ~ T rate l i.i.d. service ~ S rate m S should be “heavy tailed” -- think Pareto distribution Alan Scheller-Wolf Lunteren, The Netherlands

5. Why is E[S2] Necessary (for Single-Server Queues)? • The inspection paradox • A randomly arriving job sees an equilibrium service timeSe (or the excess of a service time) • Length biasing causes the tail of Se[P(Se>x)] to behave as the integrated tail of S – you are more likely to “see” longer service times • E[Se] is finite iff E[S2] is finite • The upshot: If an arriving job “sees” a job in service, this job has mean remaining size on the order of E[S2]. The arriving job can’t enter service until this job is done. Alan Scheller-Wolf Lunteren, The Netherlands

6. Why is E[S2] Sufficient (for Single-Server Queues)? (I) • The Lindley Equation (Lindley, 1952) • Dn+1 = (Dn+Sn-Tn)+, where Snis the service time of customer n and Tnis the interarrival time between customers n and n+1 • Why? • Let’s say customer n arrives at time t • Then customer n leaves service at time t+Dn+Sn • Customer n+1 arrives at time t + Tn • Customer n+1 enters service at time min{t+Dn+Sn,t + Tn} • Customer n+1’s delay is min{t+Dn+Sn,t + Tn} – (t + Tn) This is a reflected random walk Alan Scheller-Wolf Lunteren, The Netherlands

7. Why is E[S2] Sufficient (for Single-Server Queues)? (II) Reflected Random Walks: • For Zn+1 = (Zn+Xn)+, E[Z2] is finite if: • E[(X+)2] is finite • E[Xn] < -d < 0for all n and on every sample path • For our GI/GI/1 queue: • E[(X+)2] = E[S2]is finite • E[Xn] = E[S] - E[T] < -d < 0; for all n and on every sample path, since r<1 Alan Scheller-Wolf Lunteren, The Netherlands

8. Mean Delay in GI/GI/k Queues Multi-Server Queue • Question: What are conditions for mean stationary delay to be finite? • K&W, 1956 • r =l/ km < 1 • E[D] finite if E[S2] finite i.i.d. arrivals ~ T rate l i.i.d. service ~ S rate m Alan Scheller-Wolf Lunteren, The Netherlands

9. Why is E[S2] Sufficient? • Split the GI/GI/k into k GI/GI/1 queues • This performs no better than the original system • Route arriving jobs to queues in cyclic (or round robin) fashion • Each GI/GI/1 queue has finite mean delay • Therefore the original GI/GI/k queue has finite mean delay as well Alan Scheller-Wolf Lunteren, The Netherlands

10. What about Necessity? “Everyone knew” that r =l/ km < 1 E[S2] finite were necessary as well. This had even appeared in the literature. Alan Scheller-Wolf Lunteren, The Netherlands

11. Pictorially: Moments of S Requirement for finite mean delay 2 1 1 2 3 4 ∞ Servers Alan Scheller-Wolf Lunteren, The Netherlands

12. Back to the GI/GI/1 • Why is finite E[S2] Necessary? • The inspection paradox • If an arriving job “sees” a job in service, this job has mean remaining size on the order of E[S2]. The arriving job can’t enter service until this job is done. • Does this logic hold for the GI/GI/k? NO! • Can we capture this effect mathematically? Alan Scheller-Wolf Lunteren, The Netherlands

13. Lindley Equation for GI/GI/1 • Our logic for GI/GI/1 • Let’s say customer n arrives at time t • Then customer n leaves service at time t+Dn+Sn • Customer n+1 arrives at time t + Tn • Customer n+1 enters service at time min{t+Dn+Sn,t + Tn} • Customer n+1’s delay is min{t+Dn+Sn,t + Tn} – (t + Tn) • Dn+1 = (Dn+Sn-Tn)+ Alan Scheller-Wolf Lunteren, The Netherlands

14. Lindley Equation for GI/GI/k • For GI/GI/k • Let’s say customer n arrives at time t • Then customer n leaves service at time t+Dn+Sn • Customer n+1 arrives at time t + Tn • Customer n+1 enters service at time min{t+Dn+Sn,t + Tn} NO! Alan Scheller-Wolf Lunteren, The Netherlands

15. When Does Customer n+1 Enter Service? (I) • Customer n arrives at time t • Customer n leaves service at time t+Dn+Sn n n+1 j Alan Scheller-Wolf Lunteren, The Netherlands

16. When Does Customer n+1 Enter Service? (II) • Whendoes customer j leave service? • Let’s say that when customer n enters service, she looks over at the other server, and observes the amount of time customer j has left • Let’s call this amount of time Un • Then customer j leaves service at time t+Dn+Un If there were multiple other servers, Un would be the minimum remaining time n n+1 j Alan Scheller-Wolf Lunteren, The Netherlands

17. When Does Customer n+1 Enter Service? (III) n n+1 j • Customer n leaves service at time t+Dn+Sn • Customer j leaves service at time t+Dn+Un • So the next server comes free at time t+Dn+min{Sn,Un} = t+Dn+Pn Alan Scheller-Wolf Lunteren, The Netherlands

18. Lindley Equation for GI/GI/k • For GI/GI/k: Dn+1= (Dn+Pn-Tn)+ How big of difference could one letter make? We have replaced SnwithPn Alan Scheller-Wolf Lunteren, The Netherlands

19. Back to Reflected Random Walks • For our GI/GI/k queue, for finite mean delay it is sufficient that: • E[(X+)2] = E[P2]is finite • E[Xn] = E[Pn] - E[Tn] < -d < 0; for all n and on every sample path How does this help? We don’t know the first thing about P! Alan Scheller-Wolf Lunteren, The Netherlands

20. When is E[P2] Finite? E[P2] = E[min{S,U}2] = E[min{S,U}3/2min{S,U}1/2] < E[S3/2U1/2] = E[S3/2] E[U1/2] And E[U1/2]< E[W1/2] And E[W1/2] is finite iff E[S3/2] (K&W, 1956) S is independent of U W is the total work in system So for a GI/GI/2 queue, E[P2] is finite if E[S3/2] is finite! Alan Scheller-Wolf Lunteren, The Netherlands

21. What about Negative Drift? E[Xn] = E[Pn] - E[Tn] = E[min{Sn,Un}] - E[Tn] < E[Sn] - E[Tn] = E[S] - E[T] This seems like a really weak bound, but is in fact the best we can do, since this must hold on all sample paths and for all n; Un could be arbitrarily large on a sample path I have negative drift for all n on every sample path if E[S] < E[T] (r < ½) but not otherwise Alan Scheller-Wolf Lunteren, The Netherlands

22. Finite Mean Delay in GI/GI/2 (I) So it appears that: • Finite E[S3/2] is sufficient for finite mean delay in GI/GI/2 queues if E[S] < E[T] • Finite E[S3/2] is not sufficient for finite mean delay in GI/GI/2 queues if E[T] < E[S] < 2E[T] Does this make sense? YES! Alan Scheller-Wolf Lunteren, The Netherlands

23. Finite Mean Delay in GI/GI/2 (II) • The inspection paradox • If an arriving job “sees” a job in service, this job has mean remaining size on the order of E[S2] • The mean time this job has been in service also is on the order of E[S2] • The key question is what has been going on while this long job has been in service Alan Scheller-Wolf Lunteren, The Netherlands

24. Finite Mean Delay in GI/GI/2 (III) • If E[T] < E[S] < 2E[T]: • While a single extremely large job is in service the remaining server cannot keep up with the workload, and the system “crashes” • If E[S] < E[T]: • A single extremely large job is not typically enough to crash the system – the remaining server can keep up with the workload • It takes two extremely large jobs to crash the system Alan Scheller-Wolf Lunteren, The Netherlands

25. Redefining r • For a k-server queue r is traditionally defined as r =l/ km This is WRONG! • If instead r =l/m • E[S] < E[T] means r<1, and we have a “spare server” to absorb the workload if a long job arrives • E[T] < E[S] < 2E[T] means 1<r<2, and we have no spare capacity • r =l/m denotes the number of servers needed to handle the mean load on the system Alan Scheller-Wolf Lunteren, The Netherlands

26. What about GI/GI/3? • If 2<r<3 (no spare servers): • E[D] is finite if E[S2] is finite • If 1<r<2 (one spare server): • E[D] is finite if E[S3/2] is finite • If 0<r<1 (two spare servers): • E[D] is finite if E[S4/3] is finite Alan Scheller-Wolf Lunteren, The Netherlands

27. What about GI/GI/k? • If k-1<r<k (no spare servers): • E[D] is finite if E[S2] is finite • If k-2<r<k-1 (one spare server): • E[D] is finite if E[S3/2] is finite • If j<r<j+1 (k-j-1 spare servers): • E[D] is finite if E[S(k+1-j)/(k-j)] is finite Alan Scheller-Wolf Lunteren, The Netherlands

28. Pictorially: Fixed Number of Spare Servers Pictorially: For moment conditions, 20-server queue with load 18.5 is the same as a 2-server queue with load 0.5 Moments of S Requirement for finite mean delay 2 Diminishing benefits One spare server Two spare servers No spare servers 4/3 3/2 1 1 2 3 4 ∞ Servers Alan Scheller-Wolf Lunteren, The Netherlands

29. Pictorially: As Spare Servers Grow Moments of S Requirement for finite mean delay As the number of spare servers approaches infinity, the moment conditions approach those of an infinite-server queue, as they should 2 3/2 4/3 1 1 2 3 4 ∞ Servers Alan Scheller-Wolf Lunteren, The Netherlands

30. Other Results (I) • Higher Moments: • If j<r<j+1 (k-j-1 spare servers): • E[Dr] is finite if E[S(k+r-j)/(k-j)] is finite • (r has replaced 1 in the previous formula) • If you have just two spare servers, E[D2] is finite if E[S5/3] is finite – you get finite variance of delaywithout finite variance of service time Alan Scheller-Wolf Lunteren, The Netherlands

31. Other Results (II) • Necessary Conditions: • The sufficient conditions above are also provably necessary for all but a small set of service time distributions Alan Scheller-Wolf Lunteren, The Netherlands

32. Other Results (III) • Integral Load: • Drift has been crucial in our analysis – if r<1 E[S3/2] is sufficient for finite mean delay; if r>1 we require E[S2] • What if r=1? • We have been able to show that E[S3/2] is not sufficient, but have not been able to get close to showing E[S2] is necessary • We are still working Alan Scheller-Wolf Lunteren, The Netherlands

33. Insights • Delay moments in multi-server queues with heavy-tailed service times depend on the moments of Sand the number of spare servers • r is most usefully defined as l/m for multi-server queues • Adding spare serves helps with delay moments, but in diminishing fashion • This works because under heavy-tailed service times queues build up due to large jobs blocking servers Alan Scheller-Wolf Lunteren, The Netherlands

34. Open Problems • Dependencies: • Everything we showed was for GI/GI/k; what if interarrival and/or service times are not i.i.d? • Our “Lindley” equation for the GI/GI/k: Dn+1= (Dn+Pn-Tn)+ also holds for a special type of GI/G/1 queue. (It actually defines a GI/G/1 queue.) So there must be a relationship/duality here. Alan Scheller-Wolf Lunteren, The Netherlands

35. Open Problems (II) • Bounds: • As of now, the best analytical bounds on multi-server delay under heavy-tailed service time are in general very poor • These cannot, in general, be easily simulated • See work by Daley, Scheller-Wolf, Vesilo Alan Scheller-Wolf Lunteren, The Netherlands

36. Open Problems (III) • Tail Asymptotics • Moment conditions for delay are really just a proxy for what we really want to know: How the tail of the delay distribution behaves • See work by Boxma, Foss, Korshunov, Whitt, Zwart Alan Scheller-Wolf Lunteren, The Netherlands

37. Scheller-Wolf, A. (2003). Necessary and Sufficient Conditions for Delay Moments in FIFO Multiserver Queues: Why s Slow Servers are Better than One Fast Server for Heavy-Tailed Systems. Operations Research 51, No. 5, 748-758. Scheller-Wolf, A. and R. Vesilo (2006). Structural Interpretation and Derivation ofNecessary and Sufficient Conditions for Delay Moments in FIFO Multiserver Queues. Queueing Systems, 54, No. 3 221-232. References Alan Scheller-Wolf Lunteren, The Netherlands

38. Any Questions? Dank U! Alan Scheller-Wolf Lunteren, The Netherlands