Advanced Statistical Computing: Markov Chain Monte Carlo Techniques

1. Advanced Statistical ComputingFall 2016 Steve Qin

2. Markov Chain Monte Carlo • The goal is to generates sequence of random samples from an arbitrary probability density function (usually high dimensional), • The sequence of random samples form a Markov chain, • The purpose is simulation (Monte Carlo).

3. Motivation • Generate iid r.v. from high-dimensional arbitrary distributions is extremely difficult. • Drop the “independent” requirement. • How about also drop the “identically distributed” requirement?

4. Markov chain • Assume a finite state, discrete Markov chain with N different states. • Random process Xn, n = 0,1,2,… • Markov property, • Time-homogeneous • Order • Future state depends on the past m states.

5. Key parameters • Transition matrix • Initial probability distribution π(0) • Stationary distribution (invariant/equilibrium)

6. Reducibility • A state j is accessible from state i (written i j) if • A Markov chain is irreducible if it is possible to get to any state from any state.

7. Recurrence • A state i is transient if given that we start in state i, there is a non-zero probability that we will never return to i. State i is recurrent if it is not transient.

8. Ergodicity • A state i is ergodic if it is aperiodic and positive recurrent. If all states in an irreducible Markov chain are ergodic, the chain is ergodic.

9. Reversible Markov chains • Consider an ergodic Markov chain that converges to an invariant distribution π. A Markov chain is reversible if for all x, y ϵS, which is known as the detailed balance equation. • An ergodic chain in equilibrium and satisfying the detailed balance condition has πas its unique stationary distribution.

10. Markov Chain Monte Carlo • The goal is to generates sequence of random samples from an arbitrary probability density function (usually high dimensional), • The sequence of random samples form a Markov chain, in Markov chain, P π in MCMC, π P.

11. Examples

12. Bayesian Inference Genotype Haplotype Frequency Prior

13. History • Metropolis, N., Rosenbluth, A. W., Rosenbluth, M. N., Teller, A. H. and Teller, E. (1953). Equation of state calculations by fast computing machines. Journal of Chemical Physics, 21, 1087–1092.

14. Metropolis algorithm • Direct sampling from the target distribution is difficult, • Generating candidate draws from a proposal distribution, • These draws then “corrected” so that asymptotically, they can be viewed as random samples from the desired target distribution.

15. Pseudo code • Initialize X0, • Repeat • Sample Y ~ q(x,.), • Sample U ~ Uniform (0,1), • If U ≤ α(X,Y), set Xi = y, • Otherwise Xi = x.

16. An informal derivation • Find α(X,Y): • Joint density of current Markov chain state and the proposal is g(x,y) = q(x,y)π(x) • Suppose q satisfies detail balance q(x,y) π(x) = q(y,x)π(y) • If q(x,y) π(x) > q(y,x)π(y), introduce α(x,y) < 1 and α(y,x) =1 hence • If q(x,y) π(x) > q(y,x)π(y), … • The probability of acceptance is

17. Metropolis-Hastings Algorithm

18. Remarks • Relies only on calculation of target pdf up to a normalizing constant.

19. Remarks • How to choose a good proposal function is crucial. • Sometimes tuning is needed. • Rule of thumb: 30% acceptance rate • Convergence is slow for high dimensional cases.

20. Illustration of Metropolis-Hastings • Suppose we try to sample from a bi-variate normal distributions. • Start from (0, 0) • Proposed move at each step is a two dimensional random walk

21. Illustration of Metropolis-Hastings • At each step, calculate since

22. Convergence • Trace plot Good

23. Convergence • Trace plot Bad

24. Convergence • Trace plot

25. Convergence • Autocorrelation plot Good

26. Convergence s = 0.5 • Autocorrelation plot Bad

27. Convergence s = 3.0 • Autocorrelation plot Okay

28. References • Metropolis et al. 1953, • Hastings 1973, • Tutorial paper: Chib and Greenberg (1995). Understanding the Metropolis--Hastings Algorithm. The American Statistician49, 327-335.

29. Gibbs Sampler • Purpose: Draw random samples form a joint distribution (high dimensional) • Method: Iterative conditional sampling

30. Illustration of Gibbs Sampler

32. References • Geman and Geman 1984, • Gelfand and Smith 1990, • Tutorial paper: Casella and George (1992). Explaining the Gibbs sampler. The American Statistician, 46, 167-174.

33. Remarks • Gibbs Sampler is a special case of Metropolis-Hastings • Compare to EM algorithm, Gibbs sampler and Metropolis-Hastings are stochastic procedures • Verify convergence of the sequence • Require Burn in • Use multiple chains

34. References • Geman and Geman 1984, • Gelfand and Smith 1990, • Tutorial paper: Casella and George (1992). Explaining the Gibbs sampler. The American Statistician, 46, 167-174.