Time Reversibility

1. Time Reversibility G.U. Hwang Next Generation Communication Networks Lab. Division of Applied Mathematics KAIST

2. Time Reversibility in DTMC • Ref: R.W. Wolff, Stochastic modelling and the theory of queues, Prentice-Hall, Inc., 1989, chapter 6. • Consider a stationary DTMC with transition probabilities pij and stationary probabilities  • Suppose that starting at some time point we trace the DTMC reversely in time, i.e., Xn, Xn-1,Xn-1, . Then we see the reversed process becomes a DTMC. Next Generation Communication Networks Lab.

3. Next Generation Communication Networks Lab.

4. Now p*ij denotes the transition probability of the reversed DTMC. Then the previous derivation also shows that Next Generation Communication Networks Lab.

5. Note that, when the original CTMC is irreducible and positive recurrent, • the reversed CTMC is also irreducible and positive recurrent •  is also the stationary probability vector of the reversed DTMC because Next Generation Communication Networks Lab.

6. A time reversible DTMC • A DTMC is said to be time reversible if, for all k¸ 1 and n • Assume that a DTMC satisfies Next Generation Communication Networks Lab.

7. Then the DTMC is time reversible because by noting that we have Next Generation Communication Networks Lab.

8. Theorem for time reversibility • A stationary DTMC is time reversible if and only if, starting in state i, any path back to i has the same probability as the reversed path, for all i. That is, for all i,i1,,in Next Generation Communication Networks Lab.

9. Proof: If the DTMC is time reversible, i pij = j pji and hence Conversely, for any i,j, if we have then summing over all i1 ,,in we get Since we have Next Generation Communication Networks Lab.

10. Theorem • For an irreducible DTMC, if we can find nonnegative numbers xi, summing to 1, and nonnegative numbers p*ij which satisfy then it immediately follows that the DTMC is positive recurrent and the numbers xi form the stationary probabilities. Next Generation Communication Networks Lab.

11. Time Reversibility of CTMC • Consider a stationary CTMC with transition rate matrix Q and the stationary probabilities , i.e.,  is the initial distribution. • As in the DTMC, we trace the CTMC reversely in time. That is, for a CTMC Xt we consider X-t. Then we see the reversed process is also a CTMC: Next Generation Communication Networks Lab.

12. For the reversed CTMC the transition rate q*ij, i  j is given by Next Generation Communication Networks Lab.

13. Further, for the reversed CTMC we have, from  Q = 0, That is, the reversed CTMC has the same sojourn time distribution at each state as the original CTMC. Next Generation Communication Networks Lab.

14. When the original CTMC is irreducible and positive recurrent, • the reversed CTMC is also irreducible and positive recurrent • j also form the stationary probabilities for the reversed CTMC because Next Generation Communication Networks Lab.

15. Time reversible CTMC • A CTMC is said to be time reversible if • When a CTMC satisfies qij = q*ij, the CTMC is time reversible because the transition rates of the reversed process are the same as those of the original CTMC. Next Generation Communication Networks Lab.

16. Further, from the relation we have, for a time reversible CTMC Next Generation Communication Networks Lab.

17. For an irreducible positive recurrent CTMC, the CTMC is time reversible if and only if its embedded DTMC is time reversible. Proof: The theorem immediately follows from the facts where rij is the transition probabilities of the embedded DTMC. Next Generation Communication Networks Lab.

18. An ergodic CTMC is time reversible if and only if the product of the transition rates corresponding to any loop is equal to the product of the transition rates for the same loop traversed in the reverse direction. Proof: It immediately follows from the fact Next Generation Communication Networks Lab.

19. Kelly Lemma • For an irreducible regular CTMC having transition rates qij,if we can find nonnegative numbers xi, summing to 1, and nonnegative numbers q*ij, i  j such that then q*ij are transition rates for the reversed CTMC and xi are the stationary probabilities. Proof: The theorem immediately follows from Next Generation Communication Networks Lab.

20. Importance of time reversibility • If X1 (t) and X2 (t) are independent stochastic processes that are each time reversible, then (X1(t), X2(t)) are also time reversible. • Now for a CTMC construct a new CTMC by truncating the state space to a subset E as follows: For any i2 E, let the transition rate from i to j be qij if j2 E, and 0 otherwise. Then, if the CTMC is time reversible, it follows that i qij = j qji for i,j 2 E Next Generation Communication Networks Lab.

21. Therefore we have the following: For a time reversible CTMC with stationary probabilities j , if we consider a new CTMC by truncating the state space to E such that the new CTMC is irreducible, then the new CTMC is time reversible and has the stationary probabilities Next Generation Communication Networks Lab.

22. Example 1: Two M/M/1 queues queue 1 Poisson ( ) queue 2 Poisson ( ) common buffer of size K N1(t) = the number of packets in queue 1 N2(t) = the number of packets in queue 2 Next Generation Communication Networks Lab.

23. state transition diagram (when K = 3) ….. ….. ….. ….. (2,0) (2,1) (2,2) (2,3) ….. (1,0) (1,1) (1,2) (1,3) ….. (0,0) (0,1) (0,2) (0,3) ….. Next Generation Communication Networks Lab.

24. Let • From the time reversibility of (N1(t), N2(t)), we get where c is the normalizing constant. Next Generation Communication Networks Lab.

25. Tandem Queues • Fact 1. An ergodic birth and death process is in steady state time reversible. Proof: It immediately follows the balance equation of a BD process: 0 1 2 i-1 i ….. Next Generation Communication Networks Lab.

26. Fact 2. For the FIFO M/M/m queueing system with the arrival rate  and the service rate  of each server, • the departure process of customers is, in steady state, a Poisson process with rate . • the departure process prior to time t is independent of the future system state, Xs, s¸ t for every epoch t. Next Generation Communication Networks Lab.

27. Proof: Since the M/M/m queue is a birth and death process which is time reversible, the arrival process in the reversed process is also a Poisson process with rate . Since the arrival process in the reversed process is the departure process in the original process, the first statement immediately follows. Since the future arrivals after time t in the original process is independent of the previous system state, the future arrivals after time t in the reversed process (the departure process prior to time t in the original process, resp.) is independent of the previous system states (the future system states, resp.) Next Generation Communication Networks Lab.

28. Consider two FIFO queueing systems in tandem, where the first station is M/M/m queue. Then • For the steady state tandem queue, the numbers X1 (t) and X2(t) of customers presented at the first queue and the second queue, respectively, are independent and Next Generation Communication Networks Lab.

29. Proof: Observe that • the system state X1 (t) at the first queue is independent of the departures prior to t, and • the system state X2(t) at the second queue is determined by the departures prior to t. Therefore, X1 (t) and X2(t) are independent with each other. • Remark: The factorization of the joint distribution as given above is often called the product form. Next Generation Communication Networks Lab.

30. Buffer Analysis • Consider a queueing system where A(t) denotes the amount of work that arrives from a source in the interval [0,t] and service rate C (>0). • Assume that • A(0) = 0. • q(t) is the queue length at time t with q(0) = 0. Next Generation Communication Networks Lab.

31. Then we can show the following: • q(t) =d max0· s· t [A(t) - A(s) - C(t-s)]. present time Maximum value Next Generation Communication Networks Lab.