1 / 44

Message Passing on Planted Models: What do we know? Why do we care?

Message Passing on Planted Models: What do we know? Why do we care? . Elchanan Mossel Joint works with: 1. Uri Feige and Danny Vilenchik 2. Amin Coja-Oghlan and Danny Vilenchik. Planted 3SAT model.

carter
Download Presentation

Message Passing on Planted Models: What do we know? Why do we care?

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Message Passing on Planted Models:What do we know?Why do we care? Elchanan Mossel Joint works with: 1. Uri Feige and Danny Vilenchik 2. Amin Coja-Oghlan and Danny Vilenchik

  2. Planted 3SAT model Generates a random satisfiable formula, with probability proportional to the number of satisfying assignments that it has. • Generate m clauses at random. • Fix an assignment a at random. • Remove those clauses not satisfied by a. (Roughly 7m/8 clauses remain.)

  3. Planted 3-coloring model Take a random 3-partite (2d)-regular graph. • Take a partition into 3 sets of size n/3. • Each node connected to d random nodes in each of the other two parts. • Color each part with a different color.

  4. Why study these models? • Clearly satisfiable by construction. • Clearly “Replica Symmetric” • Become easier at higher densities. • So why study?

  5. Why study these models? • At high constant density, the model is “close” to the standard SAT model conditioned on SAT (Coja-Oghlan, Krivelevich,Vilenchik). • Feige: An efficient alg. for distinguishing (almost) SAT dense formulas from “typical” formulas ) “Hardness of Approximation” results.

  6. Why study these models? • Crypto: • Planted models a good “one way function” if they are hard to solve.

  7. Why study these models? • “Real Life Models”: • In real life models, constraints are not random. • Constraints are correlated via “nature”. • More constraints make problems easier. • Planted models may be a better model than random “SAT”.

  8. Random-SAT model Random 3CNF formula with n variables and m clauses. Conjecture: When m < 4.2n almost all formulas are satisfiable. When m > 4.3n almost all formulas not satisfiable. Theorem: Finding satisfying assignments is more difficult near the threshold. (Proof: add random clauses.)

  9. Planted-SAT model Random 3CNF formula with n variables and m clauses. Fix an assignment a Remove clauses not consistent with the assignment. “Observation”: Many clauses make the problem easier. “Pf”: E.G. > n log n clauses: For each variable i, set ai = 1 if appears more positive than negative.

  10. Our contribution Feige-M-Vilenchik: A proof that warning propagation finds satisfying assignments for most 3CNF formulas with planted assignments and large enough constant density. Coja-Oghlan-M-Vilenchik: Same for BP with density O(log n) Coja-Oghlan-M-Vilenchik: A proof that BP find satisfying assignment for most planted 3-coloring problems with planted assignments and large enough constant density. First rigorous analysis of message passing algorithms for satisfiability problems + Convergence for all variables (not just for a “typical one”)

  11. Overview of message passing algorithms The factor graph: • A bipartite graph, variables on one side and clauses on the other. • Edges connect variables to the clauses that contain them. Nodes send messages to their neighbors, and iteratively update the messages that they send based on messages received.

  12. Factor Graph X1 X2 X3 X4 X5 X6 X7 C1 C2 C3 C4

  13. Cavity principle Message from clause C to variable x: • Based on the messages received by C from other variables, an estimate of how badly C needs x to satisfy it. Message from variable x to clause C: • Based on the messages received by x from other clauses, an estimate of how likely x is to satisfy C.

  14. Different message passing algorithms Warning propagation (WP): 0/1 messages. Belief propagation (BP): fractional messages. Survey propagation (SP): fractional messages, but with a more sophisticated update rule. In all variants, if the algorithm converges, a decimation stage follows that fixes the value of some variables (those that do not receive conflicting messages) and simplifies the formula.

  15. Previous analysis of message passing algorithms (typically, BP) Converges correctly when the factor graph is a tree, or contains at most one cycle [Weiss 1997]. W.h.p., corrects most (but not necessarily all) errors in random Low Density Parity Check (LDPC) codes. W.h.p., corrects all errors for specially designed LDPC codes [Luby, Mitzenmacher, Shokrollahi, Spielman].

  16. Known algorithmic results for planted model When m >> n log n, majority vote finds planted assignment. When m > cn (for sufficiently large constant c), a “complicated” algorithm finds a satisfying assignment [Flaxman], and so does a certain local search algorithm [Feige and Vilenchik]. Analysis based on principles developed in [Alon and Kahale, SICOMP 1997].

  17. Our results: WP for planted SAT For most formulas with large enough constant clause density, w.h.p. (over initialization of random messages) WP converges after O(log n) iterations. Moreover, with probability close to 1, the formula that results after decimation is pure. WP makes no further progress on it,but this formula can be satisfied by a trivial greedy algorithm.

  18. Structural properties Sufficiently dense random formulas with a planted solution are known to w.h.p. have the following structure: • There is a large subset of the clauses which are jointly satisfied only by the induced planted solution. This is called a core. • Decimation for all core variables makes the resulting formula simple (the factor graph decomposes into small components).

  19. Algorithmic consequences Algorithms for the planted model first try to get the core variables right (for some core). Thereafter, the rest of the formula can be handled efficiently because of its simple structure. The majority vote algorithm followed by unassigning variables that do not support many clauses produces a core.

  20. WP and core For WP, the core is not an intrinsic property of the input formula. It also depends on the random initialization of messages in WP. We show: • W.h.p., after one iteration of WP a core forms – a large set of variables for which all subsequent messages agree with the planted assignment. • After O(log n) additional iterations, all noncore variables converge (either to their planted value or to don’t care). • Decimation of all variables that do care results in a pure formula.

  21. Results for BP for Planted SAT Similar results but analysis harder and degrees logarithmic due to unbounded messages (analysis should extend to the case of bounded degrees)

  22. Results for BP for Planted Coloring Note: Inherit symmetry of the coloring problem. Advantage: Initial messages could be chosen “close” to (1/3,1/3,1/3). Disadvantage: How does the algorithm breaks the symmetry?

  23. Results for BP for Planted Coloring Techniques: Use linearization + Spectral techniques to show symmetry is broken. Potential extensions: To random SAT problems with many solutions.

  24. Positive conclusions Rigorous analysis proving that a message passing algorithm finds satisfying assignments in nontrivial SAT problems.

  25. Open questions Still far from explaining the empirical success of survey propagation on random 3CNF formulas near the density threshold.

  26. Random Graphical Games on (Random) GraphsJoint work with Costas Daskalakis and Alex Dimakis

  27. Motivation • What is the effect of selfishness in networks such as: • Recommendation networks? • Network design? • Load balancing? • A crucial question: • Do networks have Nash equilibrium? • What is the effect of network topology? • Study random games. • Joint work with Costis Daskalakis and Alex Dimakis

  28. Chicken game Paul plays John plays

  29. Pure Nash Equilibrium • A player is in best response if there is no incentive to change strategy, if others do not. • A strategy profile is a Pure Nash equilibrium, if allplayers are in best response. (DC or CD) • But doesn’t always exist…

  30. Matching pennies same different Paul plays John plays There exists no Pure Nash equilibrium

  31. Best response tables Chicken game Matching pennies game

  32. Player 1 Player 1 Random Binary Games Player 2 • What is the chance that a 2 player, 2 strategy random game has a PNE? 1/21+1/23/4

  33. N strategies, 2 players? (1pt) • What is the chance that a 2 player, N strategy random game has a PNE? N

  34. Previous work For two players, many strategies, we know exactly the distribution of the number of PNE. (Dresher 1970, Powers 1990, Stanford 1996) For many players? For complete graph: Rinott et al, (2000): There exist PNE with constant probability. Further obtain asymptotic distribution on their number. (Poisson)

  35. Main Question What if the effect of connectivity? Model: Study G(n,p).

  36. Constant degrees • Claim 1: For an (undirected) random graphical game with constant degrees, whp at least a constant fraction of players are always unhappy.

  37. 0,0,1 A’=0,B’=0,C’=0 D=0, D=1 D’=0 … D’=1 For A=0,B=0,C=1 For A=0,B=0,C=0 D=0, D=1 D=0, D=1 A witness for failure: indifferent matching pennies A never happy at the same time D B C …

  38. Indifferent matching pennies shows up a lot

  39. Logarithmic Connectivity Suffcies • Claim 2: for directed graphical games, on GD(N,p), there exists a PNE with at least a constant probability. • (for all p > 2logN/N) • In fact, number of matching Pennies is asymptotically Poisson(1) with high probability.

  40. Expected # of pure Nash • For each strategy profile, a assign an indicator Xa =1 if a is a PNE. Define the RV S to be the sum of Xa: = 1

  41. Poisson limit behavior In order to establish Poisson limit behavior: Suffices to find for each strategy profile a a set B(a) of strategies such that Xa is independent of all strategies out not in B(a) &: S1 = ab 2 B(a) E[Xa Xb] = o(1) S2 = ab 2 B(a) E[Xa] E[Xb] = o(1) This follows from Stein’s method (Arratia et. Al).

  42. This is an important figure Poisson on random graphs • Given the graph G, and a=0, • B(0) = {x : 9 i: 8 j with (i,j) 2 G: x(j) = 0} • Claim: X0 is independent from (Xj : j 2 B0). • Pf: No information about responses outside B(0) • In order to prove Poisson behavior, suffices to show that E[S1], E[S2] -> 0. S

  43. Poisson on random graphs Lemma 1: If a is of weight s then: EG[E[Xa] E[X0] 1(a in B(0))] = 2-2n PG[a 2 B(0)] · 2-2n min(1,n (1-p)s-1) Lemma 2: If a is of weight s then: EG[E[Xa X0] 1(a in B(0))] · 2-2n (1 + (1-ps))n-s – (1-(1-ps))n-s) After some calculation this gives the desired result.

  44. Future research • Find exact threshold. • Prove Poisson behavior. • Other models of random graphs? • Efficiently finding equilibrium?

More Related