1 / 14

Exercise 1

Exercise 1. A Markov process is defined in the state space {0 , 1 , 2} with the state transitions rates q ij collected in the transition matrix Q = ( q ij | i, j = 0 , 1 , 2), where q ii = − q i for all i , as follows : Draw the state transition diagram of X ( t ).

amy-oliver
Download Presentation

Exercise 1

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Exercise 1 A Markov process is defined in the state space {0, 1, 2} with the state transitions rates qijcollected in the transition matrix Q = (qij | i, j = 0, 1, 2), where qii = −qi for all i, as follows: • Draw the state transition diagram of X(t). • Derive the equilibrium distribution of X(t). • Describe the behavior of the system in equilibrium when μ is very large or very small, respectively.

  2. Solution for Exercise 1a. 0 1 1 1 1 2 

  3. Solution for Exercise 1b. • We see from previous slide that the Markov process X(t) is irreducible. • Since the state space is finite, the equilibrium distribution exists, and it can be derived based on the global balance equations (GBE) and the normalization condition (N).

  4. Solution for Exercise 1b. • Let us start with the GBE’s for states 0 and 1: 0= 2 1= 0+ 2μ • The other probabilities are now solved as a function of 0: 1= 0(1 + μ) 2= 0

  5. Solution for Exercise 1b. • The other probabilities are now solved as a function of 0: 1 = 0(1 + μ), 2 = 0 • The remaining probability 0 is determined by (N): 0+ 1+ 2= 0(1 + (1 + μ) + 1) = 0(3 + μ) = 1. • Thus, the equilibrium distribution is

  6. Solution for Exercise 1b. • The remaining probability 0 is determined by (N): 0+ 1+ 2= 0(1 + (1 + μ) + 1) = 0(3 + μ) = 1. • Thus, the equilibrium distribution is

  7. Solution for Exercise 1c. • If μ , then the process is almost always at state 1 (1 1 dani 0 for the other i). • But if μ 0, then the process ultimately visits the states in the order 0 1 2 0 staying in each state on average 1 time unit. This results in the uniform equilibrium distribution (i1/3 for all i).

  8. Exercise 2 A Markov process is defined in the state space {0, 1, 2, 3} with the state transitions rates qij collected in the transition matrix: Q = (qij | i, j = 0, 1, 2, 3), where qii = −qi for all i, as follows: • Draw the state transition diagram of X(t). • Derive the equilibrium distribution of X(t). • Is the process reversible? (In other words, are the local balance equations (LBE) satisfied?)

  9. Solution for Exercise 2a. 1 1 0 2 1 2 1 2 2 3 2 1

  10. Solution for Exercise 2b. • We see from the previous slide that the Markov process X(t) is irreducible. • Since the state space is finite, the equilibrium distribution exists. The figure also tells us that all the states are symmetric (with respect to the state transition rates), which implies that the equilibrium distribution be the uniform distribution with probabilities:

  11. Solution for Exercise 2b. • Just in case, you can easily check that both the global balance equations (GBE) 30= 23+ 1, 31= 20+ 2, 32= 21+ 3, 33= 22+ 0 • And the normalization condition (N): 0+ 1+ 2+ 3= 1 are satisfied for this uniform distribution.

  12. Solution for Exercise 2c. • Consider, for example, the local balance equation between states 0 and 1: 20= 1 • The uniform equilibrium distribution (0= 1 =) does not satisfy this condition. • Thus, the process is not reversible.

  13. Exercise 3 The Ehrenfest model was used, at a time, to shed light on a paradox related to the second law of thermodynamics. The model can be described as follows. A closed system is constructed from N randomly moving gas molecules and two containers which are connected so that each of the molecules changes the container at rate , independently of the other molecules. Let X(t) denote the number of molecules in one of the containers, which clearly is a Markov process. Draw the state transition diagram of X(t)!

  14. Solution for Exercise 3a. (N-1)  2 N 0 1 … N-1 N  N (N-1) 2

More Related