Summary

1. A Cluster-Cumulant Expansion at the Fixed Points of Belief Propagation Max Welling Andrew E. GelfandAlexander Ihler Cluster Cumulant Expansion (CCE) Summary Pairwise Markov Network • Calculating the partition function is central to many problems • Belief Propagation (BP) is a celebrated algorithm for approximating marginal probabilities and the partition function • “Correction” methods, such as the Loop Series, compute modifications to BP’s partition function estimate • The Cluster Cumulant Expansion is a “Correction” method that: • More efficiently aggregates terms than the Loop Series • Empirically improves upon the Loop Series’ partial estimates • Utilizes subsets of factors to improve upon BP, but unlike Generalized BP does not require message passing. • CCE expresses Zb as: • Ω is a collection of subsets of factors (i.e. clusters), ordered wrt set inclusion • καis an over-counting number • Zαis the partition function for the partial distribution pα(x) • Key Observation • Zα‘s are comprised of Cluster Cumulant • contributions from lower order clusters! • CCE is an efficient expansion: • Cα = 0 if factor subgraphassociated with αis singly connected • Cα = 0 if factor subgraphis disconnected • Cα ≈ 0 if αcontains one new factor with nearly uniform beliefs Cluster Cumulantposet Belief Propagation • Re-Parameterization Perspective of BP • BP searches for a re-parameterization of p(x) in terms of beliefs • Variational Optimization Perspective of BP • BP can be viewed as minimizing the Bethe Free Energy Approximate by truncating the hierarchy: 2nd level approximation: Original Parameterization BP Re-Parameterization Variable Beliefs Factor Beliefs Experiments • Studied the CCE on synthetic and benchmark problem instances • Compared with the Truncated Loop Series (TLS) of Gomez et al. • Accuracy Improvement of CCE depends on poset of clusters • “blind” enumeration is intractable, so considered following enumeration strategies: • Truncated at particular level in hierarchy (e.g. all triplets) • Use sequence of generalized loops produced by the TLS • Exploit structure (e.g. faces of planar graph) bn2o instances “Correction” Methods • Utilize the following relationship at a BP fixed point: • Loop Series expresses Zb as: • One term for each generalized loop • Each term is a product of factors in the loop • Finite, but enormous number of terms in the series • Gomez et al. proposed a “smart” truncation scheme • Cluster Cumulant Expansion expresses Zb in terms of clusters Complete Markov Networks promedas instances