230 likes | 544 Views
ST3236: Stochastic Process Tutorial 3. TA: Mar Choong Hock Email: g0301492@nus.edu.sg Exercises: 4. Question 1. A markov chain X 0 , X 1 ,… on state 0 , 1 , 2 has the transition probability matrix and initial distributions, p 0 = P ( X 0 = 0) = 0 . 3 ,
E N D
ST3236: Stochastic ProcessTutorial 3 TA: Mar Choong Hock Email: g0301492@nus.edu.sg Exercises: 4
Question 1 • A markov chain X0,X1,…on state 0, 1, 2 has the transition • probability matrix • and initial distributions, p0 = P(X0 = 0) = 0.3, • p1 = P(X0 = 1) = 0.4 and p2 = P(X0 = 2) = 0.3. • Determine P(X0 = 0, X1 = 1, X2 = 2) and draw the state- • diagram with transition probability.
Question 1 P(X0 = 0,X1 = 1,X2 = 2) = P(X0 = 0)P(X1 = 1 | X0 = 0)P(X2 = 2 | X0 = 0,X1 = 1) = P(X0 = 0)P(X1 = 1 | X0 = 0)P(X2 = 2 | X1 = 1) = p0 x p01 x p12 = 0.3 x 0.2 x 0 = 0.
Question 2 • A markov chain X0,X1,…on state 0, 1, 2 has the transition • probability matrix • Determine the conditional probabilities • P(X2 = 1,X3 = 1|X1 = 0) and P(X1 = 1,X2 = 1|X0 = 0).
Question 2 P(X2 = 1, X3 = 1 | X1 = 0) = P(X2 = 1 | X1 = 1)P(X3 = 1 | X1 = 0, X2 = 1) = P(X2 = 1 | X1 = 0)P(X3 = 1 | X2 = 1) = p01 x p11 = 0.2 x 0.6 = 0.12 Similarly (or by stationarity), P(X1 = 1, X2 = 1 | X0 = 0) = 0.12 In general, P(Xn+1 = 1, Xn+2 = 1 | Xn = 0) = 0.12 for any n. That is, it doesn’t matter when you start.
Question 3 • A markov chain X0,X1,…on state 0, 1, 2 has the transition • probability matrix • If we know that the process starts in state X0 = 1, determine • probability P(X0 = 1,X1 = 0,X2 = 2).
Question 3 P(X0 = 1,X1 = 0,X2 = 2) = P(X0 = 1)P(X1 = 0| X0 = 1)P(X2 = 2 | X0 = 1,X1 = 0) = P(X0 = 1)P(X1 = 0| X0 = 1)P(X2 = 2 | X1 = 0) = p1 x p10 x p02 = 1 x0.3 x 0.1 = 0.03
Question 4 • A markov chain X0,X1,…on state 0, 1, 2 has the transition • probability matrix
Question 4a Compute the two-step transition matrix P(2). Note: Observe that the rows must always sum to one for all transition matrices.
Question 4b What is P(X3 = 1|X1 = 0)? P(X3 = 1|X1 = 0) = 0.13 In general, P(Xn+2 = 1 | Xn = 0) = 0.13 for any n.
Question 4c What is P(X3 = 1|X0 = 0)? Note that: Thus, P(X3 = 1|X0 = 0) = 0.16 In general, P(Xn+3 = 1 | Xn = 0) = 0.16 for any n.
Question 5 • A markov chain X0,X1,…on state 0, 1, 2 has the transition • probability matrix • It is known that the process starts in state X0 = 1, determine • probability P(X2 = 2).
Question 5 • Note that: • P(X2 = 2) = P(X0 = 0) x P(X2 = 2 | X0 = 0) • +P(X0 = 1) x P(X2 = 2 | X0 = 1) • +P(X0 = 2) x P(X2 = 2 | X0 = 2) • = p0p02 + p1p12 + p2p22 • = 1 x p12 = 0.35
Question 6 • Consider a sequence of items from a production process, with each item being graded as good or defective. • Suppose that a good item is followed by another good item with probability and by a defective item with probability 1-. • Similarly, a defective item is followed by another defective item with probability and by a good item with probability 1-. • Specify the transition probability matrix. • If the first item is good, what is the probability that the first defective item to appear is the fifth item?
Question 6 Let Xnbe the grade of the nth product. P(Xn+1 = g | Xn= g) = , P(Xn+1 = d | Xn= g) = 1 - P(Xn+1 = d | Xn= d) = , P(Xn+1 = g | Xn= d) = 1 - Thus, the transition probability matrix is
Question 6 The probability is, P(X5 = d,X4 = g,X3 = g,X2 = g | X1 = g) = P(X2 = g | X1 = g) x P(X3 = g | X2 = g) x P(X4 = g | X3 = g) x P(X5 = d | X4 = g) (why?) = pgg x pgg x pgg x pgd = 3(1 -)
Question 7 The random variables 1, 2, ... are independent and with common probability mass function Set X0 = 0 and let Xn= max {1, 2, ... }. • Determine the transition probability matrix for the MC {Xn}. • Draw the state-diagram associated with transition probability
Question 7 Observe: X0 = 0, X1 = max {X0, 1}, X2 = max {X1, 2}, … Xn= max {Xn-1, n} Hence, Xn recursively compares the previous maximum and the current input to obtain the new maximum.
Question 7 The state space is S = {0, 1, 2, 3} P(Xn+1 = 0 | Xn= 0) = P(n+1 = 0)= 0.1 P(Xn+1 = 1 | Xn= 0) = P(n+1 = 1)= 0.3 P(Xn+1 = 2 | Xn= 0) = 0.2 P(Xn+1 = 3 | Xn= 0) = 0.4 P(Xn+1 = 1 | Xn= 1) = P(n+1 = 0) + P(n+1 = 1) = 0.1 + 0.3 = 0.4 P(Xn+1 = 2 | Xn= 1) = 0.2 … P(Xn+1 = 2 | Xn= 2) = 0.1 + 0.3 + 0.2 = 0.6 … P(Xn+1 = 3 | Xn= 3) = 0.1 + 0.3 + 0.2 + 0.4 = 1 … P(Xn+1 = j | Xn= i) = 0 if j < i. (Cannot Happen!)
Question 7 The transition probability matrix is