ST3236: Stochastic Process Tutorial 6

1. ST3236: Stochastic ProcessTutorial 6 TA: Mar Choong Hock Email: g0301492@nus.edu.sg Exercises: 7

2. Question 1 • Consider the MC with transition probability matrix • Determine the limiting distribution.

3. Question 1 Let = (0, 1, 2) be the limiting distribution, we have deleting one of the first three equations, we have the solution as 0 = 0.4762, 1 = 0.2381, 2 = 0.2857

4. Question 2 • Consider the MC with transition probability matrix • What fraction of time, in the long run, does the • process spend in state 1?

5. Question 2 Let = (0, 1, 2) be the limiting distribution, we have deleting one of the first three equations, we have the solution as 0 = 0.2308, 1 = 0.2308, 2 = 0.5385

6. Question 2 With frequency 1 = 0.2308, in the long run, does the process spend in state 1

7. Question 3 • Consider the MC with transition probability matrix • Every period that the process spends in state 0 • incurs a cost of 2$. Every period that the process • spends in state 1 incurs a cost of 5$. Every period • that the process spends in state 2 incurs a cost of • 3$. What is the long run average cost per period • associated with this Markov chain.

8. Question 3 Let = (0, 1, 2) be the limiting distribution, we have deleting one of the first three equations, we have the solution as 0 = 0.4167, 1 = 0.1818, 2 = 0.4015

9. Question 3 The long run average cost per period associated with this Markov chain is 0.4167 x 2 + 0.1818 x 5 + 0.4015 x 3 = 2.9470$

10. Question 4 • Suppose that the social classes of successive • generations in a family follows a Markov chain with • transition probability matrix given by • What fraction of families are upper class in the long • run?

11. Question 4 Let = (L, M, U) be the limiting distribution, we have deleting one of the first three equations, we have the solution as L = 0.3529, M = 0.4118, U = 0.2353

12. Question 4 The fraction of families are upper class in the long run is U= 0.2353.