1 / 5

STA107 Lecture 25 Markov Chain Probabilities

X 2. X 4. X 3. X 1. X 5. STA107 Lecture 25 Markov Chain Probabilities. tomorrow. today. yesterday & past. Markov Property: The state of the system at time t +1 depends only on the state of the system at time t. Time Homogeneous Property:

gisela
Download Presentation

STA107 Lecture 25 Markov Chain Probabilities

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. X2 X4 X3 X1 X5 STA107 Lecture 25Markov Chain Probabilities tomorrow today yesterday & past Markov Property: The state of the system at time t+1 depends only on the state of the system at time t

  2. Time Homogeneous Property: Transition probabilities are independent of time (t) Example • Weather: • raining today 40% rain tomorrow • 60% no rain tomorrow • not raining today 20% rain tomorrow • 80% no rain tomorrow

  3. 0.6 0.4 0.8 rain no rain 0.2 Transition matrix • Stochastic matrix: • Rows sum up to 1 • Rows are conditional mass functions

  4. p p p p 0 1 99 2 100 Start (10$) 1-p 1-p 1-p 1-p Markov ProcessGambler’s Example • – Gambler starts with $10 • - At each play we have one of the following: • • Gambler wins $1 with probabilityp • • Gambler looses $1 with probability 1-p • – Game ends when gambler goes broke, or gains a fortune of $100 • (Both 0 and 100 are absorbing states)

  5. p p p p 0 1 99 2 100 Start (10$) 1-p 1-p 1-p 1-p Markov Process • Markov process - described by graph • Markov chain - a random walk on this graph (distribution over paths) • One step transition matrix gives

More Related