1 / 21

ST3236: Stochastic Process Tutorial 5

ST3236: Stochastic Process Tutorial 5. TA: Mar Choong Hock Email: g0301492@nus.edu.sg Exercises: 6. Question 1. Consider the MC with transition probability matrix Starting in state 1, determine the mean time that the process spends in state 1 prior to absorption and

yale
Download Presentation

ST3236: Stochastic Process Tutorial 5

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. ST3236: Stochastic ProcessTutorial 5 TA: Mar Choong Hock Email: g0301492@nus.edu.sg Exercises: 6

  2. Question 1 • Consider the MC with transition probability matrix • Starting in state 1, determine the mean time that the • process spends in state 1 prior to absorption and • the mean time that the process spends in state 2 • prior to absorption. Verify that the sum of these is • the mean time to the absorption.

  3. Question 1 Denote by wi1 the mean time the process spends in state 1 starting in state i prior to the absorption. We have, w01 = 0 w11 = 1 + 0.1w01 + 0.2w11 + 0.5w21 + 0.2w31 w21 = 0 + 0.1w01 + 0.2w11 + 0.6w21 + 0.1w31 w31 = 0 The solution is w01 = 0, w11 = 1.8182, w21 = 0.9091, w31 = 0.

  4. Question 1 Similarly, denote by wi2 the mean time the process spends in state 2 starting in state i prior to the absorption. We have, w02 = 0 w12 = 0 + 0.1w02 + 0.2w12 + 0.5w22 + 0.2w32 w22 = 1 + 0.1w02 + 0.2w12 + 0.6w22 + 0.1w32 w32 = 0 The solution is w02 = 0, w12 = 2.2727, w22 = 3.6364, w32 = 0.

  5. Question 1 Denote by vithe mean time to the absorption starting in state i prior to the absorption. We have, v0 = 0 v1 = 1 + 0.1v0 + 0.2v1 + 0.5v2 + 0.2v3 v2 = 1 + 0.1v0 + 0.2v1 + 0.6v2 + 0.1v3 v3 = 0 The solution is v0 = 0, v1 = 4.0909, v2 = 4.5455, v3 = 0. We have verified that, v1 = w11 + w12

  6. Question 2 • Consider the MC with transition probability matrix • Starting in state 1, determine the mean time that the • process spends in state 1 prior to absorption and • the mean time that the process spends in state 2 • prior to absorption. Verify that the sum of these is • the mean time to the absorption.

  7. Question 2 Denote by wi1 the mean time the process spends in state 1 starting in state i prior to the absorption. We have, w01 = 0 w11 = 1 + 0.5w01 + 0.2w11 + 0.1w21 + 0.2w31 w21 = 0 + 0.2w01 + 0.1w11 + 0.6w21 + 0.1w31 w31 = 0 The solution is, w01 = 0, w11 = 1.290, w21 = 0.3225, w31 = 0.

  8. Question 2 Similarly, denote by wi2 the mean time the process spends in state 2 starting in state i prior to the absorption. We have, w02 = 0 w12 = 0 + 0.5w02 + 0.2w12 + 0.1w22 + 0.2w32 w22 = 1 + 0.2w02 + 0.1w12 + 0.6w22 + 0.1w32 w32 = 0 The solution is w02 = 0, w12 = 0.3230, w22 = 2.5808, w32 = 0.

  9. Question 2 Denote by vithe mean time to the absorption starting in state i prior to the absorption. We have, v0 = 0 v1 = 1 + 0.5v0 + 0.2v1 + 0.1v2 + 0.2v3 v2 = 1 + 0.2v0 + 0.1v1 + 0.6v2 + 0.1v3 v3 = 0 The solution is v0 = 0, v1 = 1.613, v2 = 2.9033, v3 = 0. We have verified that, v1 = w11 + w12

  10. Question 3 Consider the MC in question 1. Starting in state 1, determine the probability that the process is absorbed into state 0. Compare this with the (1,0)th entry in the matrix powers P2, P4, P8 and P16.

  11. Question 3 Denote by uithe probability that the MC is absorbed by 0 starting in state i. We have, u0 = 1 u1 = 0.1u0 + 0.2u1 + 0.5u2 + 0.2u3 u2 = 0.1u0 + 0.2u1 + 0.6u2 + 0.1u3 u3 = 0 The solution is, u0 = 1, u1 = 0.4091, u2 = 0.4545, u3 = 0.

  12. Compare: u1 = 0.4091

  13. Question 3 By definition, Consider a (4 x 4) transition probability matrix,

  14. Question 3 But for our case, p00 = 1, p03 = 0.

  15. Question 3-Optional • Let: • F(t) be the set of t-step first passage paths from state 1 to state 0 • G(n-t) be the set of (n-t)-step paths from state 0 to state 0 • H(t) be the set of paths that is formed jointly by F(t) followed by G(n-t). Note: paths are n-step.

  16. Question 3-Optional

  17. Question 3-Optional Let L(n) be the set of n-step paths from state 1 to state 0. s.t.

  18. Question 3-Optional f1,0(t) is the t-step first passage probability from state 1 to state 0. If state 0 is an absorbing state, Also, trivially,

  19. Question 3-Optional

  20. Question 4 Which of the following MC is regular: a) b)

  21. Question 4 a) YES, because (all entries are greater than 0) b) NO, because it has absorbing states

More Related