COMP60611 Fundamentals of Parallel and Distributed Systems

1. COMP60611Fundamentals of Paralleland Distributed Systems Lecture 13 Queueing Theory: Part 1 John Gurd, Graham Riley Centre for Novel Computing School of Computer Science University of Manchester

2. Queueing Theory • Mathematical study of waiting in queues (lines) • Another model of concurrent activity • Applicable to many situations • Everyday situations, such as: • Supermarket check-outs • Bank tellers • Airport landings • Passport control • Computer Science applications: • Scheduling read/write accesses to disk • Scheduling tasks on multiple servers

3. Single-channel Queue

4. Single Queue, Multiple Servers Arrival of Customers

5. Multiple Queues, Multiple Servers Departure of Customers Customer free to join any queue Arrival of Customers

6. Queueing Problems • There are three basic elements to any queueing problem: • The input process – the arrivals rate • The queueing discipline – number of queues, etc. • The service mechanism – the service rate

7. Input Process • Number of potential customers: • Finite or infinite • Number arriving simultaneously: • One or many (with constant or varying batch size(s)) • Intervals between arrivals: • Constant, completely random, other statistical distribution • Average rate of arrivals: • Constant, varying with time, influenced by state of queue • Outside influence: • None, input is output of previous queue, network of queues • Etc.

8. Queueing Discipline • Number of queues: • One, multiple (fixed number or varying numbers) • Queue discipline: • Served in order of arrival (FIFO – “fair” queue) • Served in inverse order of arrival (LIFO – “unfair” queue) • Served at random • Served in priority order (based on “need”?) • Changes from time-to-time • Etc.

9. Service Mechanism • Number of service points: • One, several (fixed number or varying numbers) • Number served: • One-at-a-time, in batches (with fixed or variable batch size(s)) • Etc.

10. Effectiveness of Queues • For given assumptions about the three basic elements of a queueing problem (input, discipline, service), we can set up a mathematical model to predict the behaviour of the system, to answer questions such as: • What is the average queue size? • What is the average waiting time? • What is the probability of waiting longer than a given time? • Etc. • Thereby we can compare different queuing scenarios and plan appropriate resources.

11. Simple Example of Queue Behaviour • Arrivals: let  be a constant that represents the average rate of arrivals. For random arrivals, in any short time interval (t,t+∂t), • The probability that there is no arrival isPr{no arrival} = 1 – ∂t + o(∂t), where lim(o(∂t)/ ∂t) = 0 as ∂t → 0 (i.e., o(∂t) is negligible compared with ∂t). • The probability that there is one arrival isPr{one arrival} = ∂t + o(∂t). • The probability that there is more than one arrival isPr{two or more arrivals} = o(∂t). N.B. The probability of an arrival (or not) in any interval (t,t+∂t) is independent of t and is independent of any other non-overlapping interval. Arrival (or non-arrival) in any given interval is thus not influenced in any way by what happened at earlier times.

12. Poisson Arrivals • The number of arrivals, N, in a finite interval (t,t+x) of length x is a random variable which has a Poissondistribution with parameter x. Pr{N=n} = probability of n arrivals in (t,t+x) =

13. Poisson Arrivals • Proof: Suppose that the interval x is divided into m equal subintervals of length x/m = ∂t, then: Pr{one arrival in any subinterval} = ∂t + o(∂t), Pr{no arrival in any subinterval} = 1 – ∂t + o(∂t). Hence: Pr{exactly n arrivals in (t,t+x)} = (One arrival in each of n subintervals, and no arrivals in the remaining (m – n) subintervals; note that m > n.)

14. Poisson Arrivals Hence: Pr{exactly n arrivals in (t,t+x)} = Thus N has a Poisson distribution with parameter x.

15. Inter-arrival Time • The time that elapses between arrivals is called the inter-arrival time. Denoting this by the continuous random variable T, it can be shown that T has a negative exponential distribution: Pr{T>t} = Pr{no arrival in (0,t)} = Equivalently, Pr{T≤t} = Pr{arrival in (0,t)} = 1 – Pr{T>t} =

16. Pr{no arrival in (0,t)} 0 2 4 6 8 10 t

17. Pr{arrival in (0,t)} 0 2 4 6 8 10 t

18. Equivalence • The following three statements are equivalent: • Pr{no arrivals in (t,t+∂t)} = 1 – ∂t + o(∂t) andPr{one arrival in (t,t+∂t)} = ∂t + o(∂t). • The number of arrivals in any interval of length x is a random variable N which has a Poisson distribution with parameter x. • The inter-arrival time between successive arrivals is a random variable T which has a negative exponential distribution with parameter . • We now turn our attention to the service mechanism.

19. Service Mechanism • We assume a single server and that the service time X is a random variable with a negative exponential distribution with parameter . Thus Pr{X>x} = • Alternatively, the probability that a service that has been in progress for time x terminates in the interval (x,x+∂x) is ∂x + o(∂x). Or, the probability that a service that has been in progress for time x does not terminate in the interval (x,x+∂x) is 1 – ∂x + o(∂x). • Alternatively, the number of customers N that can be serviced during the interval (t,t+x) is a random variable that has a Poisson distribution with parameter x and Pr{N=n} =

20. Shorthand Notation for Queues • Different kinds of queue are denoted by X/Y/Z, where X is the inter-arrival distribution, Y is the service time distribution, and Z is the number of servers. • The permitted distributions are: • M = negative exponential (Markovian) • D = constant (Deterministic) • G = General independent • M/M/1 thus denotes a single server queue with random arrivals (negative exponential inter-arrival times) and random service completions (negative exponential service times); • While D/M/2 denotes a two-server queue with regular arrivals (constant inter-arrival times) and random service completions (negative exponential service times).