120 likes | 289 Views
Markov Models: Overview. Gerald F. Kominski, Ph.D. Professor, Department of Health Services. Markov Models: Why Are They Necessary?. Conventional decision analysis models assume: Chance events Limited time horizon Events that do not recur What happens if we have a problem with:
E N D
Markov Models: Overview Gerald F. Kominski, Ph.D. Professor, Department of Health Services
Markov Models: Why Are They Necessary? • Conventional decision analysis models assume: • Chance events • Limited time horizon • Events that do not recur • What happens if we have a problem with: • An extended time horizon, say, over a lifetime • Events can reoccur throughout a lifetime
State-Transition Diagram for Atrial Fibrillation p11=0.7 Well p12=0.2 p13=0.1 p22=0.9 Post- Stroke Dead p23=0.1 p33=1.0 The probabilities for all paths out of a state must sum to 1.0. Death is known as an absorbing state, because individuals who enter that state cannot transition out of it.
Transition Probabilities State of Next Cycle State of Current Cycle Transition probabilities that remain constant over time are characteristic of stationary Markov models, aka Markov chains
Markov Model Definitions • Any process evolving over time with uncertainty is a stochastic process, and models based on such processes are stochastic or probabilistic models • If the process is both stochastic and the behavior of the model in one time period (i.e., cycle) does not depend on the previous time period, the process is Markovian • The process has “lack of memory” • Even processes where the previous state does matter can be made Markovian through definition of temporary states know as tunnel states
Tunnel States Post- Stroke 2 Post- Stroke 3 Post- Stoke 1 Post- Stroke Well Dead
Defining a Markov Model • Define the initial states • Determine the cycle length • Consider possible transitions among states • Determine transition probabilities • Determine utilities, and costs (if cost-effectiveness analysis), for each state
Evaluating Markov Models:Cohort Simulation The data in the last column is used to produce a survival curve, aka a Markov trace.
Estimating Markov Models:Monte Carlo Simulation • Instead of processing an entire cohort and applying probabilities to the cohort, simulate a large number (e.g., 10,000) cases proceeding through the transition matrix • Monte Carlo simulation • TreeAge will do this for you quickly, without programming • The advantage of this approach is that it provides estimates of variation around the mean • Monte Carlo simulation is most valuable because it permits efficient modeling of complex prior history • Such variables are known as tracker variables
Example of a 5-State Markov Source: Kominski GF, Varon SF, Morisky DE, Malotte CK, Ebin VJ, Coly A, Chiao C. Costs and cost-effectiveness of adolescent compliance with treatment for latent tuberculosis infection: results from a randomized trial. Journal of Adolescent Health 2007;40(1):61-68.