1 / 8

Enhancing Mean Field Approximation with Mixture Distributions: A Generalized Methodology

The paper presents a method to improve mean field approximation by using mixture distributions. It discusses inference in graphical models, utilizing conditional dependency with Markov properties. The improved approach uses multimodal mixture distributions, assuming simple factorized components for independent fluctuations. The methodology aims to enhance probabilistic calculations by fitting parameters and bounding likelihood results. An empirical comparison shows superior performance over simple factorized mean field approximation. Future work includes combining mixture approximation with exact methods for further research and development.

icrawford
Download Presentation

Enhancing Mean Field Approximation with Mixture Distributions: A Generalized Methodology

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Improving the Mean Field Approximation via the Use of Mixture Distributions Tommi S. Jaakola Michael I. Jordan

  2. 1. Introduction • Inference in Graphical Model • make use of conditional dependency with Markov properties : sparse graph without long cycles • mean field approximation : dense graph with many clusters • Improved Mean Field Approximation • multimodal rather than unimodal by mixture distribution • components of mixture are assumed to be simple factorized distributions : independent fluctuations

  3. 2. Mean Field Approximation • Target calculation :

  4. 3. Mixture Approximation

  5. 4.Computing the Mutual Information

  6. 5. Finding the Mixture Paramters

  7. 6. Discussion • In the paper • presented a general methodology for using mixture distributions in approximate probabilistic calculations • showed that the bound on the likelihood resulting from mixture-based approximations • presented fitting parameters • Bishop, et al (1998) have presented empirical results using mixture approximation : better than simple factorized mean field approximation • Further research • combine the mixture approximation with exact methods

More Related