1 / 23

CS723 - Probability and Stochastic Processes

CS723 - Probability and Stochastic Processes. Lecture No. 45. In Previous Lecture. Finished analysis of Markov chains with finite and countable infinite state space Random walks on discretized spaces of 1, 2, and higher dimensions Started working with continuous-time Markov chains

taite
Download Presentation

CS723 - Probability and Stochastic Processes

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. CS723 - ProbabilityandStochastic Processes

  2. Lecture No. 45

  3. In Previous Lecture • Finished analysis of Markov chains with finite and countable infinite state space • Random walks on discretized spaces of 1, 2, and higher dimensions • Started working with continuous-time Markov chains • Analysis of arrival process using Poisson distribution

  4. Cont-time Markov Chains Continuous-time Markov chains can change their state at any time

  5. Cont-time Markov Chains • Arrival process of customers

  6. Cont-time Markov Chains • Customers arrive one at a time • Average rate of arrival remains constant at  customers/second • Probability of arrival in t, a short interval of time, is t • Customers arrive independent of each other

  7. Cont-time Markov Chains Prob. of k customers in [0,t] Arrival time of k-th customer Inter-arrival times Tk=Yk-Yk-1 are i.i.d.

  8. Cont-time Markov Chains Rate of change of probability value Probability of state change from k-1

  9. Cont-time Markov Chains Incremental time model of a continuous-time Markov chain and the transition probabilities

  10. Cont-time Markov Chains Probability of being in state k at t +t

  11. Cont-time Markov Chains Probability of being in state k at t +t

  12. Cont-time Markov Chains Solving for P0(t) using P0(0)=1 Solving for Pk(t)

  13. Cont-time Markov Chains Birth and death type processes

  14. Cont-time Markov Chains • Population increases or decreases by no more than 1 • Birth rate kmay depend upon the current size of population • Death rate kmay depend upon the current population size • Births and deaths occur independent of each other

  15. Cont-time Markov Chains Incremental time model of a birth and death type process

  16. Cont-time Markov Chains Probability of state change from k

  17. Cont-time Markov Chains Arrival process of customers with n = and n=0

  18. Cont-time Markov Chains Customers in a single queue with n = and n= 

  19. Cont-time Markov Chains General birth and death process with possible extinction 0 = 0 = 0

  20. Cont-time Markov Chains A continuous-time Markov chain is transient if M/M/1 queue is transient if  <  M/M/k queue is transient if k <  M/M/∞ queue is never transient

  21. Cont-time Markov Chains Stationary probability distribution of a continuous-time Markov chain Rate of change of Pn(t) should be 0

  22. Cont-time Markov Chains Recursive solution gives Since (n)=1

  23. Cont-time Markov Chains Stationary probability distribution of M/M/1 queue Expected length of the queue

More Related