1 / 13

Markov Chain Sampling Methods for Dirichlet Process Mixture Models R.M. Neal

Markov Chain Sampling Methods for Dirichlet Process Mixture Models R.M. Neal. Summarized by Joon Shik Kim 12.03.15.(Thu) Computational Models of Intelligence. Abstract .

creola
Download Presentation

Markov Chain Sampling Methods for Dirichlet Process Mixture Models R.M. Neal

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Markov Chain Sampling Methods for Dirichlet Process Mixture ModelsR.M. Neal Summarized by Joon Shik Kim 12.03.15.(Thu) Computational Models of Intelligence

  2. Abstract • This article reviews Markov chain methods for sampling from the posterior distribution of a Dirichlet process mixture model and presents two new classes of methods. • One new approach is to make Metropolis-Hastings updates of the indicators specifying which mixture component is associated with each observation, perhaps supplemented with a partial form of Gibbs sampling.

  3. Chinese Restaurant Process (1/2)

  4. Chinese Restaurant Process (2/2)

  5. Introduction (1/2) • Modeling a distribution as a mixture of simpler distribution is useful both as a nonparametric density estimation method and as a way of identifying latent classes that can explain the dependencies observed between variables. • Use of Dirichlet process mixture models has become computationally feasible with the development of Markov chain methods for sampling from the posterior distribution of the parameters of the component distribution and/or of the associations of mixture components with observations.

  6. Introduction (2/2) • In this article, I present two new approaches to Markov chain sampling. • A very simple method for handling non-conjugate priors is to use Metropolis-Hastings updates with the conditional prior as the proposal distribution. • A variation of this method may sometimes sample more efficiently, particularly when combined with a partial form of Gibbs sampling.

  7. Dirichlet Process Mixture Models (1/5) • The basic model applies to data y1,…,yn which we regard as part of an indefinite exchangeable sequence, or equivalent, as being independently drawn from some unknown distribution.

  8. Dirichlet Process Mixture Models (2/5) • We model the distribution from which the yi are drawn as a mixture of distributions of the form F(θ), with the mixing distribution over θ given G. We let the prior for this mixing distribution be a Dirichlet process, with concentration parameter αand base distribution G0.

  9. Dirichlet Process Mixture Models (3/5)

  10. Dirichlet Process Mixture Models (4/5)

  11. Dirichlet Process Mixture Models (5/5) • If we let K go to infinity, the conditional probabilities reach the following limits:

  12. Gibbs Sampling when Conjugate Priors are used (3/4)

  13. Nested CRP Day 1 Day 2 Day 3

More Related