1 / 12

Markov Chains

Markov Chains. Tom Finke. Outline of presentation The Markov chain model Description and solution of simplest chain Study of steady state solutions Study of dependence on initial conditions The Mathematics of Markov Chains Probability, Matrices, Discrete Process. Overview.

todd
Download Presentation

Markov Chains

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Markov Chains Tom Finke

  2. Outline of presentation The Markov chain model Description and solution of simplest chain Study of steady state solutions Study of dependence on initial conditions The Mathematics of Markov Chains Probability, Matrices, Discrete Process Overview

  3. Set up of the problem • Miniature Golf and Frisbee Golf • Each night some people change sites and some don’t • % remains the same • Night 1, a 50-50 split • Goal: predict % on subsequent nights

  4. Charts of data • Top Pie chart shows the % at each place on night 1 • Second Row of Pie charts show movement of population from one night to next

  5. Probability Tree Diagram • Columns refer to nights • Multiply along tree to get probability of that sequence • In any column, add all probabilities for a given activity to get total

  6. Night 2 Miniature golf: (.50)(.60)+(.50)(.30)=.45 Frisbee golf: (.50)(.40)+(.50)(.70)=.55 Night 3 Miniature golf : (.45)(.60)+(.55)(.30)=.435 Frisbee golf: (.45)(.40)+(.55)(.70)=.565 Some Mathematics

  7. Markov Chain Visualized • Circles represent activities • Arrows represent movement among activities • Numbers on arrows are probabilities

  8. Matrix Mathematics • T is the transition matrix • S(1) is the initial (1st) state vector • Matrix multiplication gives state S(2)

  9. The Change in State • This shows the time evolution of the distribution between the two activities, one night at a time • It suggests there might be a steady state which is eventually reached

  10. S(n+1)=T*S(n); eventually multiplying by T does not change the state. T^m*S(1) for large m Normalized eigenvector of T with associated eigenvalue 1 Steady State

  11. Start the process with a different state S(1) Same steady state Next Step: change the transition matrix Initial Conditions

  12. A mathematical process that is stochastic, discrete, and has the property that the probability that a particular outcome occurs depends only on the previous outcome is called a Markov chain. It can be visualized by a chain, or directed graph. It can be analyzed by matrix theory. Conclusions

More Related