1 / 13

Continuous Time Markov Chains and Basic Queueing Theory

Continuous Time Markov Chains and Basic Queueing Theory. EE384X Review 4 Winter 2004. Review: DTMC. p ij is the transition probability from i to j over one time slot The time spent in a state is geometrically distributed Result of the Markov (memoryless) property

vince
Download Presentation

Continuous Time Markov Chains and Basic Queueing Theory

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Continuous Time Markov Chainsand Basic Queueing Theory EE384X Review 4 Winter 2004

  2. Review: DTMC • pij is the transition probability from i to j over one time slot • The time spent in a state is geometrically distributed • Result of the Markov (memoryless) property • When there is a jump from state i, it goes to state j with probability

  3. qij j i qik k Continuous Time Version • qij is the transitionrate from state i to state j

  4. CTMC • Upon entering state i, a random timer Tij»Exp(qij) is started for each potential transition i!j • These timers are independent of each other • Recall that Exponential distribution is memoryless • When the first timer expires, the MC makes the corresponding transition • Let Ti be the time spent in state i, and qi=åj¹i qij, then Ti» Exp(qi) • When there is a transition, the probability of jumping to state j is qij /qi

  5. Definitions • {X(t):t¸0} is a continuous time Markov chain ifP{X(s+t)=j | X(u); u·s} = P{X(s+t)=j | X(s)} • Similar to Discrete Time MCs, Continuous Time MCs have stationary distributionp • Exists when Markov chain is positive recurrent and irreducible

  6. Stationary Distribution • Balance equations: • Transition rates in and out of state i are equal • Define matrix transition rate Q = (qij) with qii= -qi, then p Q = 0, where p is a row vector • Together with åip(i) = 1, can solve for p

  7. Queueing Theory Notation • A/S/s/k • A is the arrival process, e.g., Geometric, Poisson, Deterministic • S is the service distribution, e.g., Geometric, Exponential, Deterministic • s is the number of servers, e.g., 1, N, 1 • k is the buffer size (if k is absent, then k = 1) • E.g., Geom/M/1, M/M/1, M/D/1, M/M/1

  8. l l l l 3 0 2 1 m m m m M/M/1 Queue • Arrivals are Poisson with rate l • Inter-arrival times are exp(l) • Services are exponential with rate m • These are also transition rates for the Markov chain • This looks very similar to Geom/Geom/1 queue, but different

  9. Solving M/M/1 Queue • We have pil = pi+1m • Let r = l/m, then pi = pi-1 r = p0ri • If r < 1, the stationary distribution exists:pi = (1 - r) ri • Average Queue size:

  10. M/M/1 Queue • NQ is the queue size, excluding the one in service:

  11. l l l l 3 0 2 1 m 4m 2m 3m M/M/1 Queue • Customer arrival process is Poisson(l) • All customers are served in parallel »exp(m) • Departure rate proportional to # of customers

  12. Solving M/M/1 Queue • We have pi-1l = pi i m • Let r = l/m, then • Thus

  13. M/M/1 Queue • The queue size distribution of the M/M/1 queue is Poisson(r) • Therefore the average queue size is E(Q)=r • What’s the condition for the queue to be recurrent?

More Related