1 / 12

Expectation Maximization Algorithm

José M. Bioucas-Dias Instituto Superior Técnico 2005. Expectation Maximization Algorithm. Expectation Maximization (EM) Tool. Problem: Compute the MAP estimate. is often very hard to compute. EM approach: Approximate by a tractable iterative procedure.

darci
Download Presentation

Expectation Maximization Algorithm

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. José M. Bioucas-Dias Instituto Superior Técnico 2005 Expectation Maximization Algorithm

  2. Expectation Maximization (EM) Tool Problem: Compute the MAP estimate is often very hard to compute EM approach: Approximate by a tractable iterative procedure [Dempster et. al , 77], [Little & Rubin, 87], [McLachlan and T. Krishnan, 97]

  3. Complete and Missing Data Let where h is a non-invertible function such that Particular case: are termed missing data and complete data, respectively

  4. A Minorization Bound for • Define • Facts: • is the Kullback-Leibler distance between the densities • Note: Given two densities p and q E

  5. EM Concept

  6. EM Algorithm • Define the sequence • Then is non-decreasing • Proof Maximization Kullback

  7. EM Algorithm (Cont.) • Computing Q • It is not necessary to compute the second mean value (it does not depend on )

  8. EM Acronym E – Expectation M – Maximization

  9. Generalized EM (GEM) Algorithm • The EM rationale remains valid if we replace • with • i.e., it is not neccessary to maximaze w.r.t. It suffices to assure that increases.

  10. Generalized EM (GEM) Algorithm (cont.)

  11. Convergency of • Assume that is continuous on both arguments • then all limit points of • are stationary points of and the GEM sequence converges monotonically to for some stationary point [Wu, 83].

  12. References • A. Dempster, N. Laird, and D. Rubin. “Maximum likelihood estimation from incomplete data via the EM algorithm.” Journal of the Royal Statistical Society B, vol. 39, pp. 1-38, 1977. • R. Little and D. Rubin. Statistical Analysis with Missing Data. John Wiley & Sons, New York, 1987 • G. McLachlan and T. Krishnan. The EM Algorithm and Extensions. John Wiley & Sons, New York, 1997. • M. Tanner. Tools for Statistical Inference. Springer-Verlag, New York, 1993. • C. Wu, “On the convergence properties of the EM algorithm,” The Annal of Statistics, vol. 11, no. 1, pp. 95-103, 1983

More Related