The Evergreen Project: How To Learn From Mistakes Caused by Blurry Vision in MAX-CSP Solving

1. The Evergreen Project:How To Learn From Mistakes Caused by Blurry Vision in MAX-CSP Solving Karl J. Lieberherr Northeastern University Boston joint work with Ahmed Abdelmeged, Christine Hang and Daniel Rinehart

2. Boolean MAX-CSP solvingfor CSU 670 • Do nothing: The algorithm for the classic version finds a reasonable • assignment. Both the randomized and derandomized algorithm work for • non-symmetric formulas. But you can do much better. • Chronological Backtracking • Non-Chronological Backtracking: around page 26.

5. Introduction • Boolean MAX-CSP(G) for rank d, G = set of relations of rank d • Input • Input = Bag of Constraint • Constraint = Relation + Set of Variable • Relation = int. // Relation number < 2 ^ (2 ^ d) in G • Variable = int • Output • (0,1) assignment to variables which maximizes the number of satisfied constraints. • Example Input: G = {22} of rank 3 • 22:1 2 3 0 • 22:1 2 4 0 • 22:1 3 4 0 1in3 has number 22 M = {1 !2 !3 !4} satisfies all

6. Decision: MAX-CSP(G,f) MAX-CSP({22},f): Given a MAX-CSP({22}) instance (a bag of constraints using relation 22 = 1in3) expressed in n variables which may assume only the values 0 or 1, find an assignment to the n variables which satisfies at least the fraction f of the constraints. Example: Constraints use 1in3 = 22. 22:1 2 3 0 22:1 2 4 0 22:1 3 4 0 22: 2 3 4 0

7. MAX-CSP • Search approach: Look for forced variables before making a decision (as in Sudoku) • Look-forward: make informed decisions • Abstract representation based on look-ahead polynomials • Look-backward: avoid past mistakes • Transition system based on superresolution

8. Organization of Solver look back look forward

9. Look-ahead polynomial • The look-ahead polynomial computes the expected fraction of satisfied constraints among all random assignments that are produced with bias p.

10. Consider an instance: 40 variables,1000 constraints (1in3) 1, … ,40 22: 6 7 9 0 22: 12 27 38 0 Abstract representation: reduce the instance to look-ahead poly. 3p(1-p)2

11. 3p(1-p)2 for MAX-CSP({22})

12. SAT Rank 2 example9 constraints 14 : 1 2 014 :       3 4 014 :           5 6 0  7 : 1     3            0 7 : 1         5        0 7 :       3   5        0 7 :   2     4          0 7 :   2         6      0 7 :         4   6      0 14: 1 2 = or(1 2) 7: 1 3 = or(!1 !3) What is the look-ahead polynomial?

13. excellent peripheral vision Blurry vision • What do we learn from the abstract representation? • set 1/3 of the variables to true (maximize). • the best assignment will satisfy at least 7/9 constraints. • very useful but the vision is blurred in the “middle”. appmean = lookahead is an approximation of the true mean

14. Forget about computation ... • Focus on purely mathematical question first • Algorithmic solution will follow • Mathematical question: Given a MAX-CSP(G,f) instance. For which fractions f is there always an assignment satisfying fraction f of the constraints? In which constraint systems is it impossible to satisfy many constraints?

15. Simple example MAX-CSP({22},f): For f <= u: problem has always a solution For f = u + e: problem has not always a solution, e>0. 1 not always (solid) u = critical transition point always (fluid) 0

16. 3p(1-p)2 for MAX-CSP({22})

17. The Magic Number • u = 4/9

18. Look-ahead Polynomial • F is a MAX-CSP(G) instance. • N is an arbitrary assignment. • The look-ahead polynomial laF,N(p) computes the expected fraction of satisfied constraints of F when each variable in N is flipped with probability p.

19. The general case MAX-CSP(G) G = {R1, … }, tR(F) = fraction of constraints in F that use R. x = p

21. General Dichotomy Theorem MAX-CSP(G,f): For each finite set G of relations there exists an algebraic number tG For f <= tG: MAX-CSP(G,f) has polynomial solution For f = tG+ e: MAX-CSP(G,f) is NP-complete, e>0. polynomial solution: Use maximally biased coin. Derandomize. hard (solid) tG = critical transition point easy (fluid) 0 due to Lieberherr/Specker

22. Observations • The look-ahead polynomial look-forward approach has not been used in state-of-the-art MAX-SAT and Boolean MAX-CSP solvers. • Often a fair coin is used. The optimally biased coin is often significantly better.

24. Where we are • Introduction • Look-forward • Look-back • Packed truth tables

25. Observation • Optimally biased coin technique based on look-ahead polynomials is “best-possible”. • If we could improve it by a trillionth in polynomial time, then P=NP. • We improve it now by learning new constraints that will influence the polynomial.

26. Algorithm plan • start with assignment N = all zero. • while (proof incomplete) { • try to improve N by creating new assignment from scratch using optimally biased coin to flip the assignments; • success: Update N; • failure: learn a new constraint that will prevent same mistake and will “improve” the polynomial. }

27. N0 ={!v1,!v2,!v3,!v4}

28. N0‘ ={v1,!v2,!v3,!v4}

29. Transition Rules • Unit-Propagation (UP): M || F || SR || N → Mk || F || SR || N • if k is undefined in M, and • unsat (M¬k,SR) > 0 or unsat(M¬k,F) ≥ unsat(N,F).

30. Transition Rules • Decide (D): M || F || SR || N → Mkd || F || SR || N • if k is undefined in M, and • v(k) occurs in some constraint of F.

31. Transition Rules • Update: M || F || SR || N → M || F || SR || M • if M is complete, and • unsat(M,F) < unsat(N,F).

32. Transition Rules • Restart: M || F || SR || N → { } || F || SR || N

33. Transition Rules • Finale: M || F || SR || N → M || F || SR || N • if Φ SR or unsat(N,F) = 0.

34. Transition Rules • Semi-Superresolution (SSR): NewSR = V (¬k), where k Md M || F || SR || N → M || F || SR, NewSR || N • if unsat(M,SR) > 0 or unsat(M,F) ≥ unsat(N,F).

35. Transition Rules

36. Transition Rules (cont.)

37. Transition Manager

38. Where we are • Introduction • Look-forward • Look-back • Packed truth tables

39. Requirements • The look-ahead polynomial can be computed efficiently. Requires efficient truth table analysis. • Reduction of an instance must be efficient. • Efficiently compute the forced variables. • Each relation has a unique representation.

40. Packed Truth tables

41. end for now

42. Rank 2 example • 14 : 1 2 014 :       3 4 014 :           5 6 0  7 : 1     3            0 7 : 1         5        0 7 :       3   5        0 7 :   2     4          0 7 :   2         6      0 7 :         4   6      0

43. appmean is an approximation of the true mean

45. Transition Manager

47. MAX-CSP:Superresolution and P-Optimality Karl J. Lieberherr Northeastern University Boston joint work with Ahmed Abdelmeged, Christine Hang and Daniel Rinehart

49. Binomial Distribution