Survey Propagation

1. Survey Propagation

2. Outline • Survey Propagation: an algorithm for satisfiability1 • Warning Propagation • Belief Propagation • Survey Propagation • Survey Propagation Revisited2 • SP’s relation to BP • Covers 1 Braunstein, A., Mézard, M., and Zecchina, R. 2005. Survey propagation: An algorithm for satisfiability. Random Struct. Algorithms 27, 2 (Sep. 2005), 201-226. 2 Kroc, L.; Sabharwal, A.; and Selman, B. 2007. Survey propagation revisited. In UAl 2007.

3. Survey Propagation and Message Passing • New threshold defining two regions of solutions, easy SAT and hard SAT • Easy SAT region has many, highly-clustered solutions, and any two solutions are connected by a few number of steps • Hard SAT region has many clusters of solutions, with fewer overall solutions than easy SAT • SP is an efficient way of computing solutions in the Hard SAT region

4. New Model - Factor Graph

5. Message Passing • Treat the Factor Graph as a network • Depending on the current state of the network, messages will be passed between the nodes • These messages will later be used to determine the final value of a variable

6. Cavity Fields • Used to determine the value of a given message, and varies depending on the actual algorithm in use • Calculated by removing a function node from consideration

7. Warning Propagation • Messages are from the set {0,1} • Passed from Function nodes to Variable Nodes • Product of cavity fields • Initially randomly generate messages • Calculate new warnings for edges until we finish the time, or we hit a fixed point for all edges • New warnings are calculated from the cavity fields (∑b Є V+(j)\a ub→j) – (∑b Є V-(j)\a ub→j)

9. WP Example • Local Field • Contradiction Numbers The whole algorithm is an iterative process involving multiple WP executions, simplifying the graph using local fields

10. Belief Propagation • Finds the total number of solutions, as well as the fraction of assignments with variable Xi is true • Two types of messages: • From Function node to Variable node • Given a value for Xi, the probability that it satisfies the function clause • From Variable node to Function node • Probability that variable i takes value Xi in the absence of function clause a

11. Satisfiable and Unsatisfiable Sets

14. BP Example Blue is the cavity field of the variable node, red is the message from function to variable. Feed these values into another function to compute probabilities of being true

15. WP and BP • Work 100% of the time when the factor graph is a tree • Can be used as heuristics when it is not a tree, but no guarantee on solution • Goal: Create a more efficient scheme for the general case

16. Basics of Survey Propagation • Generally the same as BP, but deals better with clusters • Details of the message change • A message sent from function to variable is the probability that a warning is sent to the variable node • A warning is sent if all other variable nodes connected to the function do not satisfy the function. • Note: Behaves exactly like WP on tree factor graphs

17. Clustering of Configurations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . SAT Assignments . . . The problem with BP’s approximations is that they do not hold globally, but do hold if you restrict the computation to a specific cluster. SP fixes this.

20. SP Results with Don’t Care State

21. Testing Data

22. Results • New way to solve SAT using cavity fields and message passing • Works well on generic input, as well as the tree structure • Has not been rigorously examined, but is based off of similar work

23. Why does SP Work? • As of 2008, SP is the only method of solving SAT problems with 1,000,000 variables (and larger) • Does so in near linear time (even in the hardest region) • SP behaves like backtrack searches, except it almost never has to backtrack

24. SP’s relation to BP • BP works well when the number of SAT clusters is low • SP seems to work well ALL the time • Work in 2005 shows that SP is equivalent to BP search over a new object, covers • This is a generalization of the clusters described previously

25. Covers (x V ¬y V ¬z)Λ(¬x V y V ¬z) Λ(¬x V ¬y V z) σЄ{0,1,*}ⁿ is a cover of F if: • Every clause has at least one satisfying literal, or two *-ed literals • No unsupported variables are assigned 0 or 1 • X = • Y = • Z =

26. Computing Covers • New Factor Graphs:

27. Solving SAT using Covers Let P(F) be a cover of F: The factor graph of P(F) can bee seen as a Bayesian Network, so marginals can be computed using Bayes’ Theorem These marginals, when fed to the BP algorithm, computes a solution for the original SAT problem (and results in the SP algorithm). The SP algorithm has been shown to compute marginals over these covers.

28. SP, BP and Marginals Magnetization of a Variable: The difference between the marginal being positive vs. negative

29. Results • The reason why SP works so well seems to be related to these covers • These covers seem to expose hidden properties of a SAT instance, such as backbone variables

30. Interesting Aside • SAT and Decision SAT are self-reducible • Finding covers is NOT self-reducible (x V ¬y V ¬z)Λ(¬x V y V ¬z) Λ(¬x V ¬y V z) x = 1? Oracle: yes (y V ¬z) Λ(¬y V z) y = 0? Oracle: yes (¬z) (1,0,0) a cover of the original? No!