Counting CSP Solutions Using Generalized XOR Constraints

1. Counting CSP Solutions Using Generalized XOR Constraints Carla P. Gomes, Willem-Jan van Hoeve,Ashish Sabharwal, Bart Selman Cornell University AAAI Conference, 2007 Vancouver, BC

2. Problem Description • Constraint Satisfaction Problem (CSP) P: Input: a set V of variables a set of corresponding domains of variable values [finite] a set of constraints on V Output: a solution, i.e. an assignment of values to variables in V such that all constraints are satisfied • Solution Counting: how many solutions does P have? • #P-complete problem • Many applications to probabilistic reasoning,adversarial reasoning (bounded length), etc. [Roth ’96, Littman et al. ’01, Sang et al. ’04, Darwiche ’05, Domingos ’06] AAAI 2007

3. a 0 1 c b 0 0 1 1  d c d 0 1 0 0 1 1   d d e e 0 0 … … 1 1     Counting Techniques for CSPs Best known generic method: exhaustive systematic search Continue branch-and-bound search after first solution found • Simple implementation • If terminates: obtain exact solution count • If out of time: obtain lower bound • Not as advanced as systematic counters for SAT [Relsat, Cachet] • No problem division into components • No formula caching / component caching • No conflict-based learning • Scalability issue AAAI 2007

4. Specialized Counting Methods E.g. for Binary CSPs[Angelsmark-Johnson ’03, Kask-Dechter-Gogate ’04] • Count “blocks” of solutions at a time • Similar to SAT model counter called Relsat • Limited by scalability issues of high granularity exact counters (counts often range in 1015 to 1060 and higher) For integer domains[Morgado-Matos-Manquinho-MarquesSilva ’06] • Solution count preserving translation into pseudo-Boolean form • Optional further translation into Boolean form (SAT)[Bailleux-Boufkhad-Roussel ’06] • Often better than direct integer domain counting • Still limited scalability AAAI 2007

5. This Work A generic technique for efficiently counting CSP solutions • Provides lower and upper bounds on the solution count • Provides a correctness confidence(e.g. a 99% guarantee) • Often very fast Builds upon the MBound framework for counting for SAT XOR-streamlining[Gomes-Sabharwal-Selman AAAI-06] Extends the idea to general CSPs by exploiting • Richer structure : generalized XORs • Modular solution techniques : specialized filtering / propagation AAAI 2007

6. Background

7. What are XOR/Parity Constraints? Special constraints on Boolean variables • a  b  c  d = 1 : satisfied if an odd number of a,b,c,d are set to 1 e.g. (a,b,c,d) = (1,1,1,0) satisfies it (1,1,1,1) does not • b  d  e = 0 : satisfied if an even number of b,d,e are set to 1 • These translate into a small set of CNF clauses • Used earlier in randomized reductions in Theoretical CS[Valiant-Vazirani ’86] AAAI 2007

8. XORs for Counting SAT MBound approach:choose constraint X uniformly at random from all XOR constraints of size k (i.e. with k variables) Two crucial properties: • For any k, for every truth assignment A,Pr [ A satisfies X ] = 0.5 • When k=n/2, for every two truth assignments A and B,“A satisfies X” and “B satisfies X” are independent events(pairwise independence) Good average behavior, some guarantees,lower bounds Provides low variation,stronger guarantees,upper bounds AAAI 2007

9. XORs for Counting SAT Key idea: instead of modifying the solver for counting,modify the problem and feed to the usual solver Theorem: Algorithm is correct with probability 1  2t [Gomes-Sabharwal-Selman AAAI-06][Gomes-Hoffmann-Sabharwal-Selman SAT-07] solution count  2s or solution count  2s+ SATinstance Off-the-shelfSAT Solver s random XORconstraints Iterate t times AAAI 2007

10. M = 50 solutions 22 survive 13 survive unique solution 3 survive 7 survive The Desired Effect If each XOR cut the solution space roughly in half, wouldget down to a unique solution in roughly log2 M steps AAAI 2007

11. Counting CSP Solutions

12. Counting CSP Solutions We propose 3 XOR constraintsbased approaches: Individualfiltering  based on watched vars Binary XORson shadow model Globalfiltering  a  b  c = 1 Lower and upper bounds CSPinstance based on Gaussian elim.  Individualfiltering Generalized XORson CSP vars based on dynamic prog.for knapsack constraints a + b + c = r (mod D) Lower bounds only  Global filter. Inefficient, incomplete AAAI 2007

13. CSP Counting: Binary XORs • Relatively simple idea: • Create a binary shadow model for the CSP • For each (x, v Domain(x)), create a new variable yx,v  {0,1} • Add constraint: yx,v = 1 iff x == v • Add random binary XORs over variables {yx,v} • Apply Boolean XOR framework to obtain solution counts • Same correctness guarantees as for SAT • Often works quite well! CSP +binary shadowmodel CSP CSP Solver solution count(bounds) random binary XORson shadow vars iterate AAAI 2007

14. 1 0 1 Filtering Binary XORs: Individual • Filtering possible iff XOR constraint has exactly 1 unbound variable • For efficiency, use watched variables technique from SAT • Maintain watch on two yet unassigned variables • Process constraint only when a watched variable becomes bound • Find new variable to watch • If no such variable possible, do unit propagation Variable assignment: Consider c = 0 a = 1 a  c  d  f = 1 d = 1 Unit propagation: f = 1(filter out 0 from Domain(f)) AAAI 2007

15. a 1 1 1 1 0 b 1 0 1 1 0 1 c = 0 1 0 0 1 1 d 1 e Filtering Binary XORs: Global • Equivalent to a system of linear equations over F2 • Use Gaussian-elimination style filtering algorithm • Diagonalize the matrix • If a row has all 0’s and r.h.s. = 1, system unsatisfiable • If a row has only one 1, assign value to variable based on r.h.s. • Can prove: achieves complete filtering • each remaining free variable has support for both 0 and 1 Consider the system of binary XOR constraints: a  b  c  d = 1 b  c  e = 0 a  d  e = 1 AAAI 2007

16. CSP Counting: Generalized XORs • No new binary shadow variables • Instead, generalized XORs mod D directly on CSP vars: a + b + d + f + g = r (mod D), r  {0, 1, …, D-1} Max. domainsize CSP Solver CSP solution count(bounds) random generalizedXORs mod D iterate AAAI 2007

17. CSP Counting: Generalized XORs Solution counting algorithm: • Independently repeat t times: Add s random generalized XORs mod D to the CSP Solve streamlined CSP • If all t iterations have solution, output: num-solutions  Ds Else Fail Theorem: For every CSP, Pr [output is correct]  1  Dt Proof sketch: • Compute expected number of surviving solutions • Apply Markov’s inequality and independence of iterations Note: hybrid version: add XORs, count remaining solutions, take min/avg AAAI 2007

18. Filtering Generalized XORs • Individual filtering for each generalized XOR • User dynamic programming approach used for complete filtering of the knapsack constraint [Trick ’03] • E.g. a + b + c = 1 (mod 3), a {0,1}, b  {1}, c  {0,1,2} 2 2 2 Don’t include edges that cannotbe reached from the red node Remove edges that do notlead to the green node Filter variable valuesE.g. c cannot be 1 2 1 1 1 1 0 1 1 0 0 0 0 0 a b c AAAI 2007

19. Counting CSP Solutions: Recap. 3 XOR constraints based approaches: Individualfiltering  based on watched vars Binary XORson shadow model Globalfiltering  a  b  c = 1 Lower and upper bounds CSPinstance based on Gaussian elim.  Individualfiltering Generalized XORson CSP vars based on dynamic prog.for knapsack constraints a + b + c = r (mod D) Lower bounds only  Global filter. Inefficient, incomplete AAAI 2007

20. Experimental Highlights, #1 N-Queens problem order 20order 30 true count 3.9 x 1010 --- pure CSP search  3.5 x 106  4.1 x 1061 hour binary XORs, individual  1.1 x 106  5.4 x 108 binary XORs, global  2.1 x 106  5.4 x 10813-196 sec generalized XORs  6.6 x 108  9.2 x 1015 Pure CSP does not scale: limited by the number of search nodes that can be traversed within 1 hour (99% confidence) AAAI 2007

21. Experimental Highlights, #2 Graph coloring problems, e.g. “games120”: Pure CSP  4.3 x 1081 hr Pseudo-Boolean counter  1.1 x 1060.5 hr Relsat (SAT counter) on solution  1.4 x 1060.5 hrcount preserving translation[Morgado et al. ’06, Bailleux et al. ’06] Generalized XORs  4.5 x 10421 min AAAI 2007

22. Experimental Highlights, #3 Spatially balanced Latin squares • Impractical for SAT counters due to average distance computations • Using streamlined version for experiments order 15order 17 pure CSP  112, 1 hr --- 1 hr generalized XORs  1748, 8 min  1058, 14 min Note: order 17 instance cannot be solved at all by pure CSP in 1 hr! AAAI 2007

23. Summary • 3 approaches to extend the XOR-based counting framework from SAT to CSPs • Generalized XORs • Advanced filtering / propagation techniques for XORs • Experimental results very promising • Quite fast in practice • Very high counts • Provable correctness guarantees AAAI 2007