Tradeoffs in Backdoors : Inconsistency Detection, Dynamic Simplification, and Preprocessing

1. Tradeoffs in Backdoors:Inconsistency Detection, DynamicSimplification, and Preprocessing Bistra Dilkina, Carla Gomes, Ashish Sabharwal Cornell University ISAIM 2008 Fort Lauderdale, FL Note: Part of this work was presented at CP-07

2. Context • Constraint Satisfaction Problems (CSPs) • In particular, Boolean Satisfiability or SAT : • Given a Boolean formula F in conjunctive normal forme.g. F = (a or b) and (a or c or d) and (b or c)determine whether F is satisfiable • NP-complete • widely used in practice, e.g. in hardware & software verification, design automation, AI planning, … ISAIM-08

3. SAT: Gap between theory & practice From annual SAT competitions: • Large industrial benchmarks (10K+ vars) are solved within seconds by state-of-the-art complete SAT solvers • Good scaling behavior seems to defy “NP-completeness” ! How can we explain this gap between theory and practice? Real-world problems have tractable sub-structure “Backdoors” explain how solvers can get “smart” and solve very large instances ISAIM-08

4. Backdoors to tractability Informally [Gomes et al ’03]: A backdoor is a critical setof variables for a givenproblem such that, once assigned values, theremaining instance simplifies to a “tractable” class How is tractability captured formallyin the notion of backdoors? • sub-solver capturing poly-time mechanisms(not necessarily syntactically defined) • e.g. Unit Propagation, Arc Consistency, LP. Note: concept of backdoor applicable to CSPs, MIP, … ISAIM-08

5. Tractability: sub-solver Sub-solver S: An algorithm that, given formula F, satisfies four properties: • Efficiency:S runs in polynomial time(often linear or quadratic time) • Trivial solvability:S can determine whether F is trivially True (has no clauses) or trivially False (has an empty clause, or empty domain) • Trichotomy: reject, declare sat, declare unsat • Self-reducibility: also solves F[v/x] ISAIM-08

6. Strong Backdoors A subset B of variables is a strong backdoor fora formula F w.r.t. a sub-solver S if: for every truth assignment  to BS solves the “simplified” formula F[/B] • Note: have provided powerful insights, leading to techniques like randomization, restarts, and algorithm portfolios for SAT • Two key features A. Dynamic simplification based on truth values and constraint semantics (e.g. unit propagation) B. Trivial solvability / inconsistency detection ISAIM-08

7. Our Results • Relaxing these two key features leads to exponentially larger backdoors • Finding strong backdoors can become harder* than NPjust by adding inconsistency detection to tractable class(“static” notions trivially within NP, thus “weaker”) • State-of-the-art solvers do find very small strong backdoors (with inconsistency detection) • Preprocessing: small & mixed effect on backdoor size • Satisfiable instances: backdoors smaller w.r.t. PL than UP ISAIM-08

8. Our Results • Relaxing these two key features leads to exponentially larger backdoors • Finding strong backdoors can become harder* than NPjust by adding inconsistency detection to tractable class(“static” notions trivially within NP, thus “weaker”) • State-of-the-art solvers do find very small strong backdoors (with inconsistency detection) • Preprocessing: small & mixed effect on backdoor size • Satisfiable instances: backdoors smaller w.r.t. PL than UP ISAIM-08

9. From algorithmic backdoors to syntactically defined tractable classes

10. Horn/2CNF backdoors • Horn formula – every clause has at most one positive literal • 2CNF formula – every clause is binary • “Strong” Backdoors w.r.t. Horn/2CNF a subset B of variables such that for every value assignment to B, the simplified sub-formula is Horn/2CNF (not including trivial inconsistency) • [Nishimura, Ragde, Szeider ’04] : “Strong” backdoors for Horn/2CNF equivalent to deletion backdoors ISAIM-08

11. Horn/2CNF backdoors • Horn formula – every clause has at most one positive literal • 2CNF formula – every clause is binary • “Strong”Backdoors w.r.t. Horn/2CNF a subset B of variables such that for every truth assignment to B, the simplified sub-formula is Horn/2CNF (not including trivial inconsistency) • [Nishimura, Ragde, Szeider ’04] : “Strong” backdoors for Horn/2CNF equivalent to deletion backdoors • Aside: [Dechter ’92] Cycle cutset is a deletion backdoor w.r.t. the tractable class of acyclic CSPs Deletion once deleted from F, • [Chandru, Hooker ’92] ISAIM-08

12. Deletion backdoors for Horn/2CNF • Deciding whether a formula has Horn/2CNF deletion backdoor of size k is fixed-parameter tractable (FPT) • O(f(k)nc) – possibly exponential in k,but polynomial in n independent of k Why do we care about deletion backdoors? In general • Often easier to reason about and characterize (“syntactic”) • Deletion backdoors for a class are also “strong” backdoors • BUT, are deletion backdoors always as small as “strong” backdoors?(for Horn/2CNF: yes; what about richer classes?) [Nishimura, Ragde, Szeider ’04] ISAIM-08

13. Not The Case in General! Renamable Horn (RHorn) formulas • The formula is Horn up to renaming of the variables • Rename(v) : flip the sign of all literal occurrences of v Theorem. “Strong” backdoors w.r.t. RHorn can be exponentially smaller than deletion backdoors w.r.t. RHorn Proof idea: • given a strong backdoor set B, • for each assignment to B, the simplified formula F[/B] is RHorn • However, different  require different and mutually incompatiblerenamings to convert to Horn ISAIM-08

14. Lesson? • Deletion backdoors ignore the interplay between semantics of the constraints and value assignments • Thus, they capture only static tractable sub-structure • Dynamic structure analysis also much more powerful in other contexts, e.g. • dynamic caching for constraint reasoning beyond SAT[Bacchus CP-07 invited talk] Ignoring truth value assignmentsmay not always be a good idea ISAIM-08

15. How about inconsistency detection? What if the simplified formula is not Horn/2CNF butcontains an empty clause? • Clearly unsatisfiable due to x0 • But smallest Horn backdoor has size n !! • Missing trivial solvability – empty clause detection ISAIM-08

16. Empty clause detection C{} : class of formulas containing an empty clause • Inconsistency detection is a key ingredient of backtrack solvers (distinguishes DPLL from naïve search) • size of C{} backdoors correlated with search effort by zChaff [Lynce et al ’04] Proposition: Strong backdoors w.r.t. C{}can bearbitrarily smaller than strong backdoors w.r.t.pure Horn/2CNF [from the example on previous slide] ISAIM-08

17. Our Results • Relaxing these two key features leads to exponentially larger backdoors • Finding strong backdoors can become harder* than NPjust by adding inconsistency detection to tractable class(“static” notions trivially within NP, thus “weaker”) • State-of-the-art solvers do find very small strong backdoors (with inconsistency detection) • Preprocessing: small & mixed effect on backdoor size • Satisfiable instances: backdoors smaller w.r.t. PL than UP ISAIM-08

18. Backdoors w.r.t. Horn/2CNF  C{} • Horn/2CNF  C{}  sub-solver • Deletion backdoors  Strong backdoors • Deciding the existence of strong backdoor of size k w.r.t. pure Horn/2CNF is NP-complete [Nishimura et al ’04] • Intuitively speaking, not surprising since harder-to-find objects often capture richer structure more succinctly Theorem. Deciding the existence of a strong backdoor of size k w.r.t. Horn/2CNF  C{} is both NP-hard and coNP-hard. ISAIM-08

19. Our Results • Relaxing these two key features leads to exponentially larger backdoors • Finding strong backdoors can become harder* than NPjust by adding inconsistency detection to tractable class(“static” notions trivially within NP, thus “weaker”) • State-of-the-art solvers do find very small strong backdoors (with inconsistency detection) • Preprocessing: small & mixed effect on backdoor size • Satisfiable instances: backdoors smaller w.r.t. PL than UP ISAIM-08

20. What matters in practice • Backdoors characterize hidden structure in interesting combinatorial problems • It is a key notion to understand the behavior of state-of-the-art solvers and their sophisticated propagation mechanisms • Backdoors provide powerful insights motivating new strategies for algorithm design (e.g., randomization, restarts, algorithm portfolios) Do problems exhibit small backdoors? Do SAT solvers exploit them? ISAIM-08

21. An Experimental Study of Backdoors Finer grained insights into backdoors

22. Experimental study Syntactic classes: Horn, Rhorn Algorithmic sub-solvers:UP (unit propagation), PL (pure literal elimination), UP+PL,UP+PL+probing (“branch-free Satz”, failed lits) Preprocessors:2-Simplify, HyPre, SatELite, 3-Resolution Various problem domains: Graph coloring, logistics planning,car configuration, game theory (finding pure Nash eq.) Kilby et al. ’05: Satz-backdoor sizecorrelated with problem hardness ISAIM-08

23. Finding minimum deletion backdoors • Decision version within NP(unlike strong backdoors) • For Horn (same as “strong” backdoors) and RHorn • Use Integer Programming: E.g for Horn, one {0,1} variable yi for each variable i in F Minimize size: Every clause should be Horn: ISAIM-08

24. Finding small strong backdoors • Not within NP rely on indirect methods for finding a strong backdoor • Gives upper bound on the minimum strong backdoor size • Randomized version of Satz[Li and Anbulagan ’97] • a complete backtrack search solver that employs UP, PL, probing • can be easily configured to use selective propagation (e.g. only PL) • Using Satz to find (several) strong backdoors • Run Satz-Rand multiple times without restarts • Record the set B of vars not fixed by propagation mechanism S • Can verify: B is a strong backdoor w.r.t. S[for unsat instances] ISAIM-08

25. UP+PL+probing |B|/n 12/n Results: graph coloring • 3 binary variables per graph node • Planted clique = backdoor of size 4x3=12 w.r.t. C{} optimal upper bounds ISAIM-08

26. Results: graph coloring • 3 binary variables per graph node • Planted clique = backdoor of size 4x3=12 w.r.t. C{} • Propagation sub-solvers find even smaller backdoors • Horn and RHorn backdoors ignore constraint semantics • backdoor size linear in number of variables optimal upper bounds ISAIM-08

27. Results: logistics planning • SATZ backdoors of size 0 !! • UP+PL+probing solves these problems without branching • UP and UP+PL also excellent at finding the core inconsistencies • Horn and RHorn backdoors quite large  48% ISAIM-08

28. Results: car configuration • RHorn deletion backdoors are also very small • The original formulas are almost Horn • Again SATZ is excellent at finding small backdoor sets ISAIM-08

29. Results: game theory • Games on random graphs G(n,p) with avg. degree 3 • SATZ backdoors again almost all of size 0 • UP backdoors of size 6 to 8 critical players • Horn and RHorn backdoors quite large  66% ISAIM-08

30. Our Results • Relaxing these two key features leads to exponentially larger backdoors • Finding strong backdoors can become harder* than NPjust by adding inconsistency detection to tractable class(“static” notions trivially within NP, thus “weaker”) • State-of-the-art solvers do find very small strong backdoors (with inconsistency detection) • Preprocessing: small & mixed effect on backdoor size • Satisfiable instances: backdoors smaller w.r.t. PL than UP ISAIM-08

31. Effect of Preprocessors How does preprocessing affect backdoor size? [Refer to paper for details] • Considered 2-Simplify, HyPre, SatELite, 3-Resolution • Compared to backdoor size without preprocessing • Key observations • Backdoor size in the same order of magnitude(e.g. 156-219 vars for UP+PL on an instance) • Mixed effect overall: often decreases but can increase • SatELite usually gave the smallest backdoors • Preprocessing often simplifies formula but sometimesobfuscates hidden structure ISAIM-08

32. Backdoors for Satisfiable Instances How do strong backdoors behave for satisfiable instances? • Satisfiable formulas mostly studied w.r.t. weak backdoors, although strong backdoors are also applicable [Refer to paper for details] • Considered satisfiable car configuration instances • Also tested with various preprocessing techniques • Key observations: • PL yields smaller strong backdoors than UP • Perhaps because pure literal elimination is a “streamliner” • Preprocessing does not have any significant effect ISAIM-08

33. Summary • The notion of backdoor sets takes into account two key features of state-of-the-art constraint solvers: • interplay between variable assignments and constraint semantics • inconsistency detection • These features are critical for small backdoors • In the worst-case, backdoor detection is not in NP(unless PH collapses to NP) • Despite this, in practice, state-of-the art solvers do find surprisingly small backdoors ISAIM-08

34. Future Directions • Study of backdoors in CSPs, MIP, … • Semantics of backdoors • Backdoors in the context of learning/caching,and restarts • Finer theoretical characterization of the complexity of backdoor detection ISAIM-08