10 likes | 114 Views
A Cluster- Cumulant Expansion at the Fixed Points of Belief Propagation Max Welling Andrew E. Gelfand Alexander Ihler. Cluster Cumulant Expansion (CCE). Summary. Pairwise Markov Network. Calculating the partition function is central to many problems
E N D
A Cluster-Cumulant Expansion at the Fixed Points of Belief Propagation Max Welling Andrew E. GelfandAlexander Ihler Cluster Cumulant Expansion (CCE) Summary Pairwise Markov Network • Calculating the partition function is central to many problems • Belief Propagation (BP) is a celebrated algorithm for approximating marginal probabilities and the partition function • “Correction” methods, such as the Loop Series, compute modifications to BP’s partition function estimate • The Cluster Cumulant Expansion is a “Correction” method that: • More efficiently aggregates terms than the Loop Series • Empirically improves upon the Loop Series’ partial estimates • Utilizes subsets of factors to improve upon BP, but unlike Generalized BP does not require message passing. • CCE expresses Zb as: • Ω is a collection of subsets of factors (i.e. clusters), ordered wrt set inclusion • καis an over-counting number • Zαis the partition function for the partial distribution pα(x) • Key Observation • Zα‘s are comprised of Cluster Cumulant • contributions from lower order clusters! • CCE is an efficient expansion: • Cα = 0 if factor subgraphassociated with αis singly connected • Cα = 0 if factor subgraphis disconnected • Cα ≈ 0 if αcontains one new factor with nearly uniform beliefs Cluster Cumulantposet Belief Propagation • Re-Parameterization Perspective of BP • BP searches for a re-parameterization of p(x) in terms of beliefs • Variational Optimization Perspective of BP • BP can be viewed as minimizing the Bethe Free Energy Approximate by truncating the hierarchy: 2nd level approximation: Original Parameterization BP Re-Parameterization Variable Beliefs Factor Beliefs Experiments • Studied the CCE on synthetic and benchmark problem instances • Compared with the Truncated Loop Series (TLS) of Gomez et al. • Accuracy Improvement of CCE depends on poset of clusters • “blind” enumeration is intractable, so considered following enumeration strategies: • Truncated at particular level in hierarchy (e.g. all triplets) • Use sequence of generalized loops produced by the TLS • Exploit structure (e.g. faces of planar graph) bn2o instances “Correction” Methods • Utilize the following relationship at a BP fixed point: • Loop Series expresses Zb as: • One term for each generalized loop • Each term is a product of factors in the loop • Finite, but enormous number of terms in the series • Gomez et al. proposed a “smart” truncation scheme • Cluster Cumulant Expansion expresses Zb in terms of clusters Complete Markov Networks promedas instances