Combinatorial optimization and the mean field model

1. Combinatorial optimization and the mean field model Johan Wästlund Chalmers University of Technology Sweden

2. Random instances of optimization problems

3. Random instances of optimization problems

4. Random instances of optimization problems • Typical distance between nearby points is of order n-1/2

5. Random instances of optimization problems • A tour consists of n links, therefore we expect the total length of the minimum tour to scale like n1/2 • Beardwood-Halton-Hammersley (1959):

6. Mean field model of distance • Distances Xij chosen as i.i.d. variables • Given n and the distribution of distances, study the random variable Ln • If the distribution models distances in d dimensions, we expect Ln to scale like n1-1/d • In particular, pseudo-dimension 1 means Ln is asymptotically independent of n

7. Mean field model of distance • The edges of a complete graph on n vertices are given i. i. d. nonnegative costs • Exponential(1) distribution.

8. Mean field model of distance • We are interested in the cost of the minimum matching, minimum traveling salesman tour etc, for large n.

9. Mean field model of distance Convergence in probability to a constant?

10. Matching • Set of edges that gives a pairing of all points

11. Statistical Physics / C-S • Feasible solution • Cost of solution • Cost of minimal solution • Artificial parameter T • Gibbs measure • n→∞ • Spin configuration • Hamiltonian • Ground state energy • Temperature • Gibbs measure • Thermodynamic limit

12. Statistical physics • Replica-cavity method of statistical mechanics has given spectacular predictions for random optimization problems • M. Mézard, G. Parisi 1980’s • Limit of p2/12 for minimum matching on the complete graph (Aldous 2000) • Limit 2.0415… for the TSP (Wästlund 2006)

13. A. Frieze (2004): “Up to now there has been almost no progress analysing this random model of the travelling salesman problem.” • N. J. Cerf et al (1997): “Researchers outside physics remain largely unaware of the analytical progress made on the random link TSP.”

14. Non-rigorous derivation of the p2/12 limit • Matching problem on Kn for large n. • In principle, this requires even n, but we shall consider a relaxation • Let the edges be exponential of mean n, so that the sequence of ordered edge costs from a given vertex is approximately a Poisson process of rate 1.

15. Non-rigorous derivation of the p2/12 limit • The total cost of the minimum matching is of order n. • Introduce a punishment c>0 for not using a particular vertex. • This makes the problem well-defined also for odd n. • For fixed c, let n tend to infinity. • As c tends to infinity, we expect to recover the behavior of the original problem.

16. Non-rigorous derivation of the p2/12 limit • For large n, suppose that the problem behaves in the same way for n-1 vertices. • Choose an arbitrary vertex to be the root • What does the graph look like locally around the root? • When only edges of cost <2c are considered, the graph becomes locally tree-like

17. Non-rigorous derivation of the p2/12 limit • Non-rigorous replica-cavity method • Aldous derived equivalent equations with the Poisson-Weighted Infinite Tree (PWIT)

18. Non-rigorous derivation of the p2/12 limit • Let X be the difference in cost between the original problem and that with the root removed. • If the root is not matched, then X = c. Otherwise X = xi – Xi, where Xi is distributed like X, and xi is the cost of the i:th edge from the root. • The Xi’s are assumed to be independent.

19. Non-rigorous derivation of the p2/12 limit It remains to do some calculations. We have where Xi is distributed like X

20. X  Non-rigorous derivation of the p2/12 limit • Let -u

21. Non-rigorous derivation of the p2/12 limit • Then if u>-c,

22. Non-rigorous derivation of the p2/12 limit Hence is constant

23. Non-rigorous derivation of the p2/12 limit f(-u) • The constant depends on c and holds when –c<u<c f(u)

24. Non-rigorous derivation of the p2/12 limit • From definition, exp(-f(c)) = P(X=c) = proportion of vertices that are not matched, and exp(-f(-c)) = exp(0) = 1 • e-f(u) + e-f(-u) = 2 – proportion of vertices that are matched = 1 when c = infinity.

25. Non-rigorous derivation of the p2/12 limit

26. Non-rigorous derivation of the p2/12 limit • What about the cost of the minimum matching?

27. Non-rigorous derivation of the p2/12 limit

28. Non-rigorous derivation of the p2/12 limit

29. Non-rigorous derivation of the p2/12 limit • Hence J = area under the curve when f(u) is plotted against f(-u)! • Expected cost = n/2 times this area • In the original setting = ½ times the area = p2/12.

30. The equation has the explicit solution • This gives the cost

32. The exponential bipartite assignment problem n

33. The exponential bipartite assignment problem • Exact formula conjectured by Parisi (1998) • Suggests proof by induction • Researchers in discrete math, combinatorics and graph theory became interested • Generalizations…

34. Generalizations • by Coppersmith & Sorkin to incomplete matchings • Remarkable paper by M. Buck, C. Chan & D. Robbins (2000) • Introduces weighted vertices • Extremely close to proving Parisi’s conjecture!

35. Incomplete matchings n m

36. Weighted assignment problems • Weights 1,…,m, 1,…, n on vertices • Edge cost exponential of rate ij • Conjectured formula for the expected cost of minimum assignment • Formula for the probability that a vertex participates in solution (trivial for less general setting!)

37. a3 a1 a2 The Buck-Chan-Robbins urn process • Balls are drawn with probabilities proportional to weight

38. Proofs of the conjectures • Two independent proofs of the Parisi and Coppersmith-Sorkin conjectures in 2003 (Nair, Prabhakar, Sharma and Linusson, Wästlund)

39. Rigorous method • Relax by introducing an extra vertex • Let the weight of the extra vertex go to zero • Example: Assignment problem with 1=…=m=1, 1=…=n=1, and m+1 =  • p = P(extra vertex participates) • p/n = P(edge (m+1,n) participates)

40. Rigorous method • p/n = P(edge (m+1,n) participates) • When →0, this is • Hence • By Buck-Chan-Robbins urn theorem,

41. Rigorous method • Hence • Inductively this establishes the Coppersmith-Sorkin formula

42. Rigorous results • Much simpler proofs of Parisi, Coppersmith-Sorkin, Buck-Chan-Robbins formulas • Exact results for higher moments • Exact results and limits for optimization problems on the complete graph

43. The 2-dimensional urn process • 2-dimensional time until k balls have been drawn

44. Limit shape as n→∞ • Matching: • TSP/2-factor:

45. Mean field TSP • If the edge costs are i.i.d and satisfy P(l<t)/t→1 as t→0 (pseudodimension 1), then as n →∞,

46. For the TSP, the replica-cavity approach gives

47. It follows that is constant, and = 1 by boundary conditions • Replica-cavity prediction agrees with the rigorous result (Parisi 2006)

48. Further exact formulas

49. LP-relaxation of matching in the complete graph Kn

50. Future work • Explain why the cavity method gives the same equation as the limit shape in the urn process • Reprove results of one method with the other • Find the variance with the replica method • Find rigorously the distribution of edge costs participating in the solution (there is an exact conjecture)