Spatial decay of correlations and efficient methods for computing partition functions.

1. Spatial decay of correlations and efficient methods for computing partition functions. David Gamarnik Joint work with Antar Bandyopadhyay (U of Chalmers), Dmitriy Rogozhnikov-Katz (MIT) June, 2006

2. Talk Outline • Partition functions. Where do we see them ? • Computing partition functions. Monte Carlo method. • Correlation decay. • Our results: computation tree, correlation decay and • Deterministic algorithm for approximate computation of partition functions for matchings and colorings. • Structural results and large deviations. • Conclusions

3. Partition functions - feature in • statistical mechanics Gibbs measure and Ising models • computer science and combinatoricscounting problems • queueing theoryproduct form loss networks • electrical engineering • coding theory • statisticsbayesian networks

4. Queueing Example:loss system with shared resources • Calls arrive as and request communication link • Call is accepted only if no other link attached to is occupied • Unaccepted call is lost • Call duration is

5. At any moment the set of occupied links is amatching • The steady-state distribution is product form: - partition function.

6. Example II:multicasting in a communication network • Calls arrive as and occupy a node • Call is accepted only if no neighbor is occupied • Unaccepted call is lost • Call duration is

7. At any moment the set of occupied nodes is anindependent set • The steady-state distribution is product form: - partition function.

8. Example III:multicasting with many frequencies • Calls arrive as and occupy a node and use frequency • Call is accepted only if no neighbor is occupied and uses the same fr. • Unaccepted call is lost • Call duration is

9. At any moment the set of occupied nodes is apartial coloring • The steady-state distribution is product form: - partition function.

10. From queueing to statistical physics • Communication (matching) problem with - Gibbs distribution on Ising type models. Important object in stat mechanics. - inverse temperature - Monomer-dimer model.

11. From statistical physics to computer science • Matching problem with total number of matchings in the graph (counting)

12. Can we compute partition function?... … easily when the underlying graph is a tree. Example (independent sets) This leads to

13. Theorem.Spitzer [75], Zachary [83,85], Kelly [85]. In -ary tree Is independent from the boundary condition (correlation decay) if and only if Implication: if the graph is locally-tree like, then computing marginals is possible in the regime Ramanan, Sengupta,Zeidins, Mitra [2002] Related work on unicasting and multicasting on trees

14. Computing partition function in general • Valiant [1979] -- #P complexity class. Exact counting is hard for most of the counting problems (matchings, independent sets, colorings, etc. ) • Focus – approximate counting. • Our contribution: - use of correlation decay for • - Deterministic (non-simulation based)algorithms for computing approximately partition functions for • Matchings in low degree graphs • Colorings in low degree graphs • - Structural properties of partition functions in special classes of graphs

15. Existing approaches for computing partition function • Main approximation method: Markov Chain Monte Carlo (MCMC) • The MCMC is based on • - computing the marginal distribution via simulation. • - reducing partition function to marginals (cavity method). • Jerrum, Valiant & Vazirani [86] • Technical challenge: establishing rapid mixing

16. Computing partition functions using MCMC

17. (Temporal) Decay of correlations in Markov chains A Markov chain with transition matrix satisfies decay of correlation (mixes) if and only if it is aperiodic (Spatial) Decay of correlations Same thing, but time is replaced by a “spatial” distance

18. Correlation Decay A sequence of spatially (graph) related random variables exhibits a decay of correlation (long-range independence),if when is large Principle motivation - statistical phyisics. Uniqueness of Gibbs measures on infinite lattices, Dobrushin [60s].

19. What is known aboutcorrelation decay ? • Link between Correlation Decay and rapid mixing of MC: • CD implies rapid mixing in subexp. growing graphs. • Converse not true Kenyon, Mossel & Peres [01].

20. Our results: Theorem I. There exists a deterministic algorithm for computing approximately the partition function corresponding to matchings in graphs with constant degree (deterministic FPTAS), for arbitrary Related work: Weitz [2005]. Self-avoiding walk based algorithm for counting independent sets when

21. Algorithm and proof: Step I. Reduce computing partition function to computing marginals (cavity method) Thus computing marginals implies computing the partition function

22. Step II. Cavity recursion

23. Step II. Cavity recursion

24. Step II. Cavity recursion

25. Algorithm: repeat the recursion times. - Computation tree Initialize at the bottom arbitrarily. Compute recursively.

26. Proof: look at the recursion function: Proposition. The computation tree satisfies the decay of correlation property Introduce change of variables:

27. Mean Value Theorem: - contraction

30. Theorem II. There exists a deterministic algorithm for computing approximately the number of list colorings in triangle-free graphs when the size of each list is constant and for all nodes

31. Cavity recursion

32. Cavity recursion x x x

33. Cavity recursion x x x We establish correlation decay for this recursion

34. Problem: We need accuracy in order to have accuracy Why can’t we use conventional decay of correlation directly for counting by computing marginals locally for small (constant) ?

35. But:

36. Structural results The decay of correlation property implies the following large deviations results: Theorem III. The partition function of independent sets in every r-regular locally tree-like graphs satisfies when

37. Queueing/large deviations interpretation 1. In a multicasting model (independent sets) the probability that nobody is transmitting a signal is 2. The probability that the set of active nodes is is given as

38. Structural results • Theorem IV. The partition function of the number of q-colorings in every r-regular graph with large girth satisfies These results are not “provable” using MCMC technique

39. Note: removing a node when computing marginals destroys regularity A fix comes from a rewiring trick Mezard-Parisi [05]. Lemma. The rewiring operation can be performed on pairs of nodes without creating small cycles.

40. Final thoughts and goals • Queueing and stationarity. • Consider a queueing version of the “matching” problem. Assume FIFO. • Does the loss of stationarity occur before or after onset of long-range dependence?

41. Final thoughts and goals • Create an implementable version of our algorithm (aka Belief Propagation). Our algorithm is only nominally efficient. • Combining algorithm with importance sampling to handle large degree instances. • Other counting problems: permanent, volume of a polyhedron. • What other structures have the underlying computation tree satisfy the correlation decay property? Markov random fields?