1 / 26

Markov Chains: Part I

Markov Chains: Part I. A Typical Problem. The city of Math Island is experiencing a movement of its population to the suburbs. At present, 85% of the total population lives in the city and 15% lives in the suburbs.

ashton-chavez
Download Presentation

Markov Chains: Part I

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Markov Chains: Part I

  2. A Typical Problem The city of Math Island is experiencing a movement of its population to the suburbs. At present, 85% of the total population lives in the city and 15% lives in the suburbs. But each year, 7% of the city people move to the suburbs, while only 1% of the suburb people move back to the city. Assume that the total population remains constant. Let C0, C1, C2, ⋅⋅⋅ represent the percentages of the population in the city, respectively, now, 1 year from now, 2 years from now, and so on. Also, let S0, S1, S2, ⋅⋅⋅ be the percentages of the population in the suburbs, respectively, now, 1 year from now, 2 years from now, and so on.

  3. Question #1 Use the tree diagram shown below to verify that after 1 year 79.2% of the residents of Math Island live in the city and 20.8% live in the suburbs. Now 1 year from now C1 .93 C0 S1 .85 .07 C1 .01 .15 S0 S1 .99

  4. Question #2 Use the extended tree diagram to verify that after 2 years 73.9% of the residents of Math Island live in the city and 26.1% live in the suburbs. Now 1 year from now 2 years from now C2 .93 C1 S2 .93 .07 .01 C2 C0 S1 .85 S2 .07 .99 .93 C2 .01 C1 .15 S2 S0 .07 S1 .01 .99 C2 .99 S2

  5. Question #3 If we used a tree diagram to determine the probability distribution of the residents in 8 years, how many branches would there be on the tree? Hint: The number of branches is a power of 2.

  6. Answering Question #3 Observe the following pattern After n years from now , the number of branches is 2n+1. Hence, after 8 years, the tree diagram would have 28+1 = 29 or 512 branches.

  7. Would you make a tree diagram to find the distribution of the population 8 years from now? Just imagine how long it would take! We hope that there is a more efficient way to predict the future distribution of the population.

  8. A Different Approach In life, we often need to change our ways of dealing with an issue. We now change our view of the Typical Problem. Instead of just using probability tree diagrams to find the future distribution of the population, let us turn to matrix multiplication. Try to see what makes one think of using matrix multiplication!

  9. Question #4 Consider a 1×2 matrix that represents the initial probability distribution of the residents, say City Suburbs City Suburbs and a 2×2 matrix that represents the movement of population, say City Suburbs Find the elements of the matrices D0 and M.

  10. Answering Question #4 Initially, 85% and 15%of the population live in the city and in the suburbs, respectively. Thus: City Suburbs We can visualize the population movement of as follows: City City City Suburbs Suburbs Suburbs The probability of city to city is .93, the probability of city to suburbs is.07, suburbs to city is .01, and suburbs to suburbs is .99 City Suburbs City Suburbs

  11. A word on terminology In Markov Chain theory, the matrix M is called a Transition Matrix. Why is the name appropriate? It is usually denoted by the letter T. The matrix D0 is referred to as the Initial State Matrix. It usually denoted by S0.

  12. Question #5 Verify that the probability distribution of the residents of Math Island after 1 year and after 2 years are given, respectively, by D1 = D0 M and D2 = D1 M.

  13. Answering Question #5 First, we multiply D0 and M, by hand or using a calculator: City Suburbs So, as we found earlier, in question #1, 79.2% of the population will reside in the city and 20.8% in the suburbs.

  14. Answering Question #5 (continued) Now, we multiply D1 and M, by hand or using a calculator: City Suburbs So, as we found earlier, in question #2, 73.9% of the population will reside in the city and 26.1% in the suburbs.

  15. Question #6 a) Find an expression for the probability distribution after 2 years, D2, in terms of D0. b) Deduce from that an expression for the probability distributions after 3, 4, and n years, i.e. D3, D4, and Dn in terms of D0. Hint: Substitute and find a pattern.

  16. Answering Question #6 a) In question #5, we compute D2 from D1. substitute D1= D0 M b) We now find D3. substitute for D2

  17. Answering Question #6 (continued) Following this pattern, we can show that: and more generally We now ready to perform what was a daunting task!

  18. Question #7 Find the population distribution 8 years from now. We don’t need to draw a tree diagram with 512 branches.

  19. Answering Question #7 From question #6, we know that: where and Hence 8 years from now, 49.7% of the population will reside in the city and 50.3% will be in the suburbs.

  20. The Big Picture The Typical Problem that we’ve been working with is an example of a Markov chain, or Markov process. The idea is that a system is evolving from one state to another in such a way that chances are involved in progressing from one state to the next. We consider only the case when the transition matrix (the square matrix that indicates the probability of moving from one state to another) is constant.

  21. Let S0 be the initial state matrix and T the transition matrix for the Markov chain. Then, as was the case in the Typical Problem, the probability distribution at the nth state (when the experiment has been repeated n times) is

  22. Example The buying pattern of Lost City home buyers who buy single-family homes and condominiums has been observed and it was discovered that: 85% of single-family homeowners buy again single-family homes and 65% of condominium owners buy again condominiums. Currently, 80% of the homeowners live in single-family homes and 20% live in condominiums. If this trend continues, what will be the percentage of homeowners in the city that will own single-family homes and condominiums 2 years from now? 5 years from now?

  23. SFH Condos Let S0 be the initial state matrix. Then SFH Condos SFH Condos Let T be the transition matrix. Then we have We need to find S2 and S5.

  24. We use a calculator to compute: So 2 years from now, 72.5% of the residents will own single-family homes and 27.5% will own condos. Similarly, we compute So 5 years from now, 70.3% of the residents will own single-family homes and 29.7% will own condos.

  25. More Questions In Part II, we’ll deal with some thought-provoking questions: • In the Typical Problem, it is evident that the percent of the total population that remains in the city is decreasing. Does the situation ever stabilize? • In the long run, does the probability distribution of the residents depend on the initial distribution D0?

More Related