Chapter 10 Markov Chains

1. Chapter 10Markov Chains Section 10.2 Regular Markov Chains

2. Short-Term Predictions • Recall from the previous section that short-term predictions can be made for n repetitions of an experiment using the initial probability vector and the transition matrix.

3. KickKola Example • Also from the previous section, we found that KickKola soda’s market share increased after each of two following purchases from their original market share of 14%. (See Section 10.1, Example 2 notes) • What will happen to their market share after a large number of repetitions? • The answer to this question requires us to make a long-range prediction and find the level at which the trend will stabilize.

4. Long-Range Predictions • In this section, we try to decide what will happen to the initial probability vector in the long run – that is, as n gets larger and larger. • Assumption: transition matrix probabilities remain constant from repetition to repetition.

5. Regular Transition Matrices • While one of the many applications of Markov chains is in finding long-range predictions, it’s not possible to make long-range predictions with all transition matrices. • Long-range predictions are always possible with regular transition matrices.

6. Regular Transition Matrices • A transition matrix is regular if some power of the matrix contains all positive entries. • A Markov chain is a regular Markov chain if its transition matrix is regular.

7. Determining if Transition Matrices are Regular • If some power of the matrix has all positive, non-zero entries, then the matrix is regular. • If a transition matrix has some zero entries and if these zeros occur in the identical places in both P and P for any k, then the zeros will appear in those places for all higher powers of P. Therefore, P is not regular. k+1 k

8. Example 1 • Decide whether the following transition matrices are regular. a.) b.)

9. Equilibrium Vectors

10. Equilibrium Vector • If a Markov chain with transition matrix P is regular, then there exists a probability vector V such that VP = V • This vector V gives the long-range trend of the Markov chain. Vector V is found by solving a system of linear equations. • If there are two states, then V = [ x y ] If there are three states, then V = [ x y z ]

11. Example 2 • Find the equilibrium vector for the transition matrices below. a.) b.)

12. Properties of a Regular Markov Chain

13. Steps for Making a Long-Range Prediction 1.) Create the transition matrix P. 2.) Determine if the chain is regular. (This occurs when the transition matrix is regular.) 3.) Create the equilibrium matrix V. 4.) Find and simplify the system of equations described by VP = V. 5.) Discard any redundant equations and include the equation x + y = 1 related to V = [x y]. (Or x+y+z=1) 6.) Solve the resulting system. 7.) Check your work by verifying that VP = V.

14. Example 3 - KickKola Revisited A marketing analysis shows that 12% of the consumers who do not currently drink KickKola will purchase KickKola the next time they buy a cola and that 63% of the consumers who currently drink KickKola will purchase it the next time they buy a cola. Make a long-range prediction of KickKola’s ultimate market share, assuming that current trends continue.

15. Example 4 A census report shows that currently 32% of the residents of Metropolis own their home and that 68% rent. Of those that rent, 12% plan to buy a home in the next 12 months, while 3% of the homeowners plan to sell their home and rent instead. Make a long-range prediction of the percent of Metropolis residents who will own their home and the percent who will rent, assuming current trends hold.