1 / 12

Markov Models: Overview

Markov Models: Overview. Gerald F. Kominski, Ph.D. Professor, Department of Health Services. Markov Models: Why Are They Necessary?. Conventional decision analysis models assume: Chance events Limited time horizon Events that do not recur What happens if we have a problem with:

leif
Download Presentation

Markov Models: Overview

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Markov Models: Overview Gerald F. Kominski, Ph.D. Professor, Department of Health Services

  2. Markov Models: Why Are They Necessary? • Conventional decision analysis models assume: • Chance events • Limited time horizon • Events that do not recur • What happens if we have a problem with: • An extended time horizon, say, over a lifetime • Events can reoccur throughout a lifetime

  3. Decision Tree for Atrial Fibrillation

  4. State-Transition Diagram for Atrial Fibrillation p11=0.7 Well p12=0.2 p13=0.1 p22=0.9 Post- Stroke Dead p23=0.1 p33=1.0 The probabilities for all paths out of a state must sum to 1.0. Death is known as an absorbing state, because individuals who enter that state cannot transition out of it.

  5. Transition Probabilities State of Next Cycle State of Current Cycle Transition probabilities that remain constant over time are characteristic of stationary Markov models, aka Markov chains

  6. Markov Model Definitions • Any process evolving over time with uncertainty is a stochastic process, and models based on such processes are stochastic or probabilistic models • If the process is both stochastic and the behavior of the model in one time period (i.e., cycle) does not depend on the previous time period, the process is Markovian • The process has “lack of memory” • Even processes where the previous state does matter can be made Markovian through definition of temporary states know as tunnel states

  7. Tunnel States Post- Stroke 2 Post- Stroke 3 Post- Stoke 1 Post- Stroke Well Dead

  8. Defining a Markov Model • Define the initial states • Determine the cycle length • Consider possible transitions among states • Determine transition probabilities • Determine utilities, and costs (if cost-effectiveness analysis), for each state

  9. Evaluating Markov Models:Cohort Simulation The data in the last column is used to produce a survival curve, aka a Markov trace.

  10. Estimating Markov Models:Monte Carlo Simulation • Instead of processing an entire cohort and applying probabilities to the cohort, simulate a large number (e.g., 10,000) cases proceeding through the transition matrix • Monte Carlo simulation • TreeAge will do this for you quickly, without programming • The advantage of this approach is that it provides estimates of variation around the mean • Monte Carlo simulation is most valuable because it permits efficient modeling of complex prior history • Such variables are known as tracker variables

  11. Example of a 5-State Markov Source: Kominski GF, Varon SF, Morisky DE, Malotte CK, Ebin VJ, Coly A, Chiao C. Costs and cost-effectiveness of adolescent compliance with treatment for latent tuberculosis infection: results from a randomized trial. Journal of Adolescent Health 2007;40(1):61-68.

  12. Key Assumptions of the Markov Model

More Related