1 / 94

Constraint Satisfaction & Constraint Programming Advanced Issues

Constraint Satisfaction & Constraint Programming Advanced Issues My Thanks to Toby Walsh and Roman Bartak (for “stealing some slides”). Overview. Higher Consistencies for binary constraints Path Consistency, k -consistency, SAC, PIC, NIC, RPC Non-binary Constraints

wilson
Download Presentation

Constraint Satisfaction & Constraint Programming Advanced Issues

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Constraint Satisfaction & Constraint Programming Advanced Issues My Thanks to Toby Walsh and Roman Bartak (for “stealing some slides”) ΑΝΑΠΑΡΑΣΤΑΣΗ ΓΝΩΣΗΣ - Lecture 1

  2. Overview • Higher Consistencies for binary constraints • Path Consistency, k-consistency, SAC, PIC, NIC, RPC • Non-binary Constraints • Search Algorithms & Consistencies for non-binary constraints • GAC, Bounds Consistency, PWC • specialized filtering algorithms • Encodings of non-binary constraints into binary • decomposable constraints • encodings of non-binary into binary constraints • Alternative search methods • Limited Discrepancy Search • Optimization ΑΝΑΠΑΡΑΣΤΑΣΗ ΓΝΩΣΗΣ - Lecture 1

  3. Path Consistency (PC) • A CSP is path consistent iff every pair of assignments (x,a) and (y,b) where a and b are compatible, can be extended to a consistent assignment of every third variable • a PC algorithm adds binary no-goods • The assignments (x,a) and (y,b) cannot be made simultaneously • Plus/Minus + detects more inconsistencies than AC - extensional representation of constraints - changes in graph connectivity • Directional PC, Restricted PC ΑΝΑΠΑΡΑΣΤΑΣΗ ΓΝΩΣΗΣ - Lecture 1

  4. Path Consistency • Path consistency will detect that there is no solution in this problem • by adding new binary constraints • Naturally PC is more expensive than AC • O(n3d3) time complexity compared to O(n2d2) • PC algorithms have high space complexity as well • O(n3d2) X 1 2 1 2 Y Z 1 2 ΑΝΑΠΑΡΑΣΤΑΣΗ ΓΝΩΣΗΣ - Lecture 1

  5. k-consistency • k-consistency = (k-1,1) consistency • consistent assignment of (k-1) variables can be extended to k-th variable • strong k-consistencyj-consistency for each jk • NC  strong 1-consistency • AC  strong 2-consistency • PC  strong 3-consistency • What is 6-consistency? • The cost of k-consistency is exponential in k • impractical for large k ΑΝΑΠΑΡΑΣΤΑΣΗ ΓΝΩΣΗΣ - Lecture 1

  6. Consistency Completeness • strongly N-consistent constraint graph with N nodes => solution • strongly K-consistent constraint graph with N nodes (K<N) => ??? path consistent but no solution • Special graph structures • tree structured graph => (D)AC is enough • cycle cutset, MACE  A D {1,2,3} {1,2,3}     C B  {1,2,3} {1,2,3} ΑΝΑΠΑΡΑΣΤΑΣΗ ΓΝΩΣΗΣ - Lecture 1

  7. Domain Filtering Consistencies • An important disadvantage of path consistency and k-consistency in general is that they alter the structure of the constraint graph and the constraints’ relations • this implies an exponential (in k) space complexity • Domain filtering consistencies are consistencies that only remove values from the domains of variables • i.e. they only add unary constraints • arc consistency is the most widely used such consistency • many more have been proposed • inverse and singleton consistencies ΑΝΑΠΑΡΑΣΤΑΣΗ ΓΝΩΣΗΣ - Lecture 1

  8. Singleton Arc Consistency (SAC) • A value a of a variable x is singleton arc consistent (SAC) if after reducing the domain of x to {a} and applying AC, there is no domain wipe-out • A CSP is SAC iff all values in the domains of all variables are SAC • A SAC algorithm deletes all values that are not SAC • Singleton consistency is a generic notion that can be combined with many consistencies • singleton path consistency • singleton k-consistency • Singleton consistencies do not alter the constraints • they only remove inconsistent values ΑΝΑΠΑΡΑΣΤΑΣΗ ΓΝΩΣΗΣ - Lecture 1

  9. Singleton Arc Consistency (SAC) • Singleton arc consistency will detect that there is no solution in this problem • without adding new binary constraints • Naturally SAC is more expensive than AC • O(end3) time complexity compared to O(ed2) • The space complexity is: • O(end2) X 1 2 1 2 Y Z 1 2 ΑΝΑΠΑΡΑΣΤΑΣΗ ΓΝΩΣΗΣ - Lecture 1

  10. Path Inverse Consistency (PIC) • k-consistency is otherwise known as (k-1,1) consistency • consistent assignment of (k-1) variables can be extended to k-th variable • path consistency is (2,1) consistency • What about the inversek-consistency = (1,k-1) consistency? • consistent assignment of 1 variable can be extended to any (k-1) variables • The cost of k-inverse-consistency is exponential in k • impractical for large k • Path inverse consistency is (1,2) consistency • consistent assignment of 1 variable can be extended to any 2 variables ΑΝΑΠΑΡΑΣΤΑΣΗ ΓΝΩΣΗΣ - Lecture 1

  11. Path Inverse Consistency (PIC) • Path inverse consistency will detect that there is no solution in this problem • without adding new binary constraints • Naturally PIC is more expensive than AC • O(ed2+cd3) time complexity, where c is the number of 3-cliques • The space complexity is: • O(ed+cd) X 1 2 1 2 Y 1 2 ΑΝΑΠΑΡΑΣΤΑΣΗ ΓΝΩΣΗΣ - Lecture 1

  12. Neighborhood Inverse Consistency (NIC) • A value a of a variable x is neighborhood inverse consistent (NIC) if it can be extended to a consistent instantiation of all the neighbors of x (i.e. all variables connected to x) • NIC deletes values that cannot be part of a solution to the sub-problem defined by the neighbors of x • The behavior of NIC is dependent on the structure of the constraint graph • what happens if the graph is complete (i.e. each variable is constrained with all the others)? • NIC is cost effective when used on problems with sparse graphs ΑΝΑΠΑΡΑΣΤΑΣΗ ΓΝΩΣΗΣ - Lecture 1

  13. Neighborhood Inverse Consistency (NIC) • NIC will detect that there is no solution in this problem • NIC can be very expensive • Its time complexity depends on the maximum number of neighbors that a variable has • this can be as much as n-1 • The only proposed algorithm has complexity of O(eg2dg+1) • g is the maximum degree X 1 2 1 2 Y 1 2 ΑΝΑΠΑΡΑΣΤΑΣΗ ΓΝΩΣΗΣ - Lecture 1

  14. Restricted Path Consistency (RPC) • Restricted Path Consistency (RPC) is an intermediate level of consistency between AC and PC • it removes all arc inconsistent values and it checks the path consistency of all pairs of values (x,a) , (y,b) such that (y,b) is the only support for (x,a) in the domain of y. • if such as pair is path inconsistent, its deletion would lead to the arc inconsistency of value a of x • RPC only removes a (it does not add the binary constraint) • RPC performs few more checks than AC, while deleting more values without changing the structure of the constraint graph ΑΝΑΠΑΡΑΣΤΑΣΗ ΓΝΩΣΗΣ - Lecture 1

  15. Restricted Path Consistency (RPC) • RPC will detect that there is no solution in this problem • PRC is relatively cheap • O(ed2 + cd2 ) where c is the number of 3-cliques • Preliminary experiments have shown that it is the most promising alternative to AC X 1 2 1 2 Y 1 2 ΑΝΑΠΑΡΑΣΤΑΣΗ ΓΝΩΣΗΣ - Lecture 1

  16. Relations between Local Consistencies NIC SRPC SAC PIC RPC AC strong PC stronger consistency incomparable consistencies ΑΝΑΠΑΡΑΣΤΑΣΗ ΓΝΩΣΗΣ - Lecture 1

  17. Non-binary Constraints: Outline • Definition of non-binary constraints • Modeling with non-binary constraints • Solving non-binary CSPs • Search and constraint propagation with non-binary constraints • Encodings of non-binary CSPs into binary • Practical benefits: case study • Golomb rulers • Crossword puzzles • Sudoku • A non-exhaustive list of specialized non-binary constraints… ΑΝΑΠΑΡΑΣΤΑΣΗ ΓΝΩΣΗΣ - Lecture 1

  18. Definition • Binary constraint • Relation on 2 variables identifying those pairs of values disallowed (no-goods) • E.g. not-equals constraint: X1 X2. • Non-binary constraint • Relation on 3 or more variables identifying tuples of values disallowed • E.g: • alldifferent(X1,X2,X3) • X1+X2+X3>X4 ΑΝΑΠΑΡΑΣΤΑΣΗ ΓΝΩΣΗΣ - Lecture 1

  19. Some non-binary examples • Timetabling • Variables: Lecture1, Lecture2, … • Values: time1, time2, … • Constraint that lectures do not conflict: alldifferent(Lecture1,Lecture2,…). • Scheduling • Variables: Job1, Job2, … • Values: machine1, machine2, … • Constraint on number of jobs on each machine: atmost(2,[Job1,Job2,…],machine1), atmost(1,[Job1,Job2,…],machine2). ΑΝΑΠΑΡΑΣΤΑΣΗ ΓΝΩΣΗΣ - Lecture 1

  20. Why use non-binary constraints? • We know that any non-binary constraint can be represented using binary constraints • E.g. alldifferent(X1,X2,X3) is “equivalent” to X1 X2, X1 X3, X2 X3 • In theory therefore they’re not needed • But in practice, they are! • most real problems are naturally represented using non-binary constraints ΑΝΑΠΑΡΑΣΤΑΣΗ ΓΝΩΣΗΣ - Lecture 1

  21. Modeling with non-binary constraints • Benefits include: • Natural representation of real constraints • Compact, declarative specifications • Efficient constraint propagation • However, non-binary constraints post some challenges: • Most algorithms and techniques have been developed for binary constraints • can we adapt them to the non–binary case ? ΑΝΑΠΑΡΑΣΤΑΣΗ ΓΝΩΣΗΣ - Lecture 1

  22. Modeling with non-binary constraints • Consider the all-different constraint • It can be represented using binary constraints • This representation is not very compact • alldifferent([X1,…Xn]) expands into n(n-1)/2 binary not-equals constraints, Xi Xj • one non-binary constraint or O(n2) binary constraints? x1 x2 x3 x4 x5 x6 x1 x2 x6 x3 x4 x5 ΑΝΑΠΑΡΑΣΤΑΣΗ ΓΝΩΣΗΣ - Lecture 1

  23. Solving non-binary CSPs • There are two approaches we can follow: • Extend algorithms and heuristics for binary CSPs to deal with non-binary constraints • Search algorithms, local consistencies, variable/value ordering heuristics, local search techniques, etc. • Devise new algorithms if necessary • Translate any given non-binary CSP into a binary one and solve it using standard techniques for binary constraints • How can we do the translation? • Is this efficient? ΑΝΑΠΑΡΑΣΤΑΣΗ ΓΝΩΣΗΣ - Lecture 1

  24. Local Consistencies for Non-binary Constraints • Generalized arc-consistency (GAC) for non-binary constraints • A non-binary constraint is GAC iff for every value a for a variable x there is a consistent and valid tuple including (x,a) and values for all other variables in the constraint • A tuple is consistent if it is allowed by the constraint • A tuple is valid if none of the values in the tuple has been removed from the corresponding domain • Supports are not single values, but tuples of values • We can prune values that are not supported • GAC = AC on binary constraints value 5 of X4 has the support <2,2,2> value 6 of X4 has no support {0,1,2} {0,1,2} {0,..,6} {0,1,2} x1 x3 x4 x2 X1+X2+X3>X4 ΑΝΑΠΑΡΑΣΤΑΣΗ ΓΝΩΣΗΣ - Lecture 1

  25. Non-binary model alldifferent(X1,X2,X3) is not GAC Binary model X1 X2, X1 X3, X2 X3 are all AC But GAC is, in general, much more expensive O(ekdk) optimal time complexity where k is the maximum arity of the constraints GAC is stronger than AC {2,3} X1   {2,3} {2,3}  X2 X3 ΑΝΑΠΑΡΑΣΤΑΣΗ ΓΝΩΣΗΣ - Lecture 1

  26. GAC-3: An Algorithm for GAC procedureREVISE(x,c) for each value a in domain of xdo if there is no tuple in the relation ofc that includes (x,a) and is consistent and valid then delete a from the domain of x procedureGAC-3(G) Let Q be the set of (undirected) constraints of G while Q not empty do select and remove any constraintc from Q; for each variable x participating in c REVISE(x,c) if REVISE(x,c) changed the domain of xthen add to Q the set of all constraints that includex (except c); ΑΝΑΠΑΡΑΣΤΑΣΗ ΓΝΩΣΗΣ - Lecture 1

  27. Algorithms for GAC • The time complexity of GAC-3 is O(ek2dk+1) • this can be reduced to O(ek2dk) using a similar data structure as in AC-2001 (algorithm GAC-2001/3.1) • Other binary AC algorithms have been extended to the non-binary case • GAC-4 • GAC-Schema (a generalization of AC-7) • achieves multi-directionality and has optimal worst-case complexity O(ekdk) • but uses complicated data structures and is difficult to implement procedureREVISE-2001/3.1(x,c) for each value a in domain of xdo if there is noconsistent and validtuple τ in the domain of y such thatτ> Lastx,a,c then delete a from the domain of x else Lastx,a,y = first such tuple ΑΝΑΠΑΡΑΣΤΑΣΗ ΓΝΩΣΗΣ - Lecture 1

  28. Achieving GAC • By exploiting “semantics” of certain constraints, we can often enforce GAC much more efficiently than with a generic algorithm • Consider alldifferent([X1,…Xn]) with each Xi having domain of size d • Generic GAC algorithm runs in O(ndn) • A specialized GAC algorithm for the alldifferent constraint runs in O(dnn) based on network flow • Designing such algorithms is a hot topic in constraint programming research • more on this later… ΑΝΑΠΑΡΑΣΤΑΣΗ ΓΝΩΣΗΣ - Lecture 1

  29. Alldifferent GAC pruning • How to make an all-different constraint GAC? Given domains, create domain/variable bipartite graph x1 1 x2 2 3 x3 4 x4 5 x5 ΑΝΑΠΑΡΑΣΤΑΣΗ ΓΝΩΣΗΣ - Lecture 1

  30. Alldifferent GAC pruning • Pruning? Which edges are in no matching? Find them and prune the corresponding values from the domains x1 1 Domain is sharply reduced x2 2 3 x3 4 x4 5 x5 ΑΝΑΠΑΡΑΣΤΑΣΗ ΓΝΩΣΗΣ - Lecture 1

  31. Bounds Consistency • Local consistencies for non-binary constraints are expensive in the general case • GAC exponential in the arity of the constraints • Bounds consistency (BC) is a restricted (and cheap) form of (G)AC that applies (G)AC only on the values at the bounds of the variables’ domains • e.g. BC will only check if values 0 and 9 are AC for a variable with domain {0,…9} • BC can be very cost effective for certain types of constraints • can you think of such a constraint? ΑΝΑΠΑΡΑΣΤΑΣΗ ΓΝΩΣΗΣ - Lecture 1

  32. Other Local Consistencies for Non-binary CSPs • Local consistencies for non-binary constraints are not as studied as in the binary case • because they are expensive in general • Some strong consistencies have been proposed • relational consistencies • pairwise consistency • will say more about it later • hyper k-consistencies • In all these cases, the primary entities where the consistencies operate are the constraint relations, not the variables ΑΝΑΠΑΡΑΣΤΑΣΗ ΓΝΩΣΗΣ - Lecture 1

  33. Search Algorithms for Non-binary CSPs • Some search algorithms are easily generalized to the non-binary case • MAC => MGAC while for others it is less obvious • FC => ? • Lets start with the simplest algorithm. How can we generalize chronological backtracking to handle non-binary problems? • perform a constraint check only when the current variable is the last variable in a constraint • e.g. constraint x1+x2+x3>x4 will be checked when the algorithm reaches x4 (assuming a static variable ordering) ΑΝΑΠΑΡΑΣΤΑΣΗ ΓΝΩΣΗΣ - Lecture 1

  34. FC for non-binary constraints • Forward checking has been generalized to non-binary CSPs in many ways: • nFC0 • check a constraint when there is exactly one variable unassigned • e.g. constraint x1+x2+x3>x4 will be checked when the algorithm reaches x3 • nFC1 • apply one pass of AC to each constraint and projection involving current variable and one future variable • nFC2 • apply one pass of AC to the set of constraints involving the current variable and at least one future variable • nFC3 • apply AC to the set of constraints involving the current variable and at least one future variable ΑΝΑΠΑΡΑΣΤΑΣΗ ΓΝΩΣΗΣ - Lecture 1

  35. FC for non-binary constraints • Forward checking has been generalized to non-binary CSPs in many ways: • nFC4 • apply one pass of AC to the set of constraints involving at least one past variable (or the current variable) and at least one future variable • nFC5 • apply AC to the set of constraints involving at least one past variable (or the current variable) and at least one future variable ΑΝΑΠΑΡΑΣΤΑΣΗ ΓΝΩΣΗΣ - Lecture 1

  36. nFC - Example c1 c2 c3 Assume the assignments (x,a) and (u,a) are made. What pruning do algorithms nFC0-nFC5 achieve? ΑΝΑΠΑΡΑΣΤΑΣΗ ΓΝΩΣΗΣ - Lecture 1

  37. Constraint Programming Solvers • CP Solvers offer: • A rich constraint language • Arithmetic, higher-order, logical constraints • Global constraints for natural substructures • Easy specification of a search procedure • Definition of search tree to explore through modeling decisions • Specification of search strategy • Choice of branching heuristics • The user • Models the problem as a CSP by specifying variables, domains, and constraints • Selects the search strategy to be used • Feeds the problem to the solver ΑΝΑΠΑΡΑΣΤΑΣΗ ΓΝΩΣΗΣ - Lecture 1

  38. Illustrative artificial example • Color a map of (part of) Europe: Belgium, Denmark, France, Germany, Netherlands, Luxembourg • No two adjacent countries same color • Are four colors enough? enum Country {Belgium,Denmark,France,Germany,Netherlands,Luxembourg}; enum Colors {blue,red,yellow,gray}; var Colors color[Country]; solve { color[France] <> color[Belgium]; color[France] <> color[Luxembourg]; color[France] <> color[Germany]; color[Luxembourg] <> color[Germany]; color[Luxembourg] <> color[Belgium]; color[Belgium] <> color[Netherlands]; color[Belgium] <> color[Germany]; color[Germany] <> color[Netherlands]; color[Germany] <> color[Denmark]; }; • Variables non-numeric • Constraints are non-linear ΑΝΑΠΑΡΑΣΤΑΣΗ ΓΝΩΣΗΣ - Lecture 1

  39. Domain store For each variable: what is the set of possible values? If empty for any variable, then infeasible If singleton for any variable, then solution Constraint store Capture interesting and well studied substructures called global constraints Need to Determine if constraint is feasible WRT the domain store Prune “impossible” values from the domains This is done using specialized or generic GAC algorithms Bounds consistency algorithms Constraint Programming CP solvers consist of: ΑΝΑΠΑΡΑΣΤΑΣΗ ΓΝΩΣΗΣ - Lecture 1

  40. Turning non-binary constraints into binary • Two methods • Encodings: Replace with binary constraints by introducing new variables • Decompositions: (For restricted classes of non-binary constraints) Replace with binary constraints on same variables • Theoretical results are informative • Comparing non-binary constraint propagation with binary • Suggests where non-binary constraints are valuable ΑΝΑΠΑΡΑΣΤΑΣΗ ΓΝΩΣΗΣ - Lecture 1

  41. Let’s start with the easy case! Decomposable constraints: Non-binary constraints that can be represented by binary constraints with introducing new variables It’s a special case that sometimes occurs about which we can be (theoretically) quite precise • Certain non-binary constraints decompose into binary constraints on same variables • Sometimes called “network decomposable” ΑΝΑΠΑΡΑΣΤΑΣΗ ΓΝΩΣΗΣ - Lecture 1

  42. Binary decompositions • Two examples: • all-different(x1,x2,x3) is x1≠x2, x1≠x3, x2≠x3 • monotone(x1,x2,x3) is x1 < x2, x2 < x3 • One non-example: • even(x1+x2+x3) • Can you see why not? • Can you think of any other decomposable constraints? ΑΝΑΠΑΡΑΣΤΑΣΗ ΓΝΩΣΗΣ - Lecture 1

  43. Binary decompositions • Theoretical comparison direct • compare pruning of variables in binary decomposition with that in non-binary • Empirical experiments reinforce theory • decomposing non-binary constraints can add orders of magnitude to solution cost ΑΝΑΠΑΡΑΣΤΑΣΗ ΓΝΩΣΗΣ - Lecture 1

  44. Binary decompositions • Upper and lower bound on FC nFC1 on non-binary > FC on decomposition > nFC0 on non-binary • Gaps can be exponential Consider n-ary all-different with n-1 values nFC1 takes (n-1) branches FC on decomposition takes (n-1)! branches • GAC lower bound GAC on non-binary > AC on decomposition Gap again can be exponential But if we decompose too much, GAC=AC! • GAC upper bound In general, GAC ~ NIC, GAC ~ PIC .. BUT if decomposition to clique, NIC > GAC ΑΝΑΠΑΡΑΣΤΑΣΗ ΓΝΩΣΗΣ - Lecture 1

  45. Binary decompositions • Tighter results provable for stricter classes • Tree decomposable constraints • constraint graph is tree • Triangle preserving constraints • non-binary constraints on all triangles ΑΝΑΠΑΡΑΣΤΑΣΗ ΓΝΩΣΗΣ - Lecture 1

  46. Binary decompositions • Tree decomposable constraints • e.g. monotone(x1,x2,x3) • GAC=AC • not surprising as AC is enough to solve the problem! • Decomposition here doesn’t lose us anything • but even one cycle is enough for GAC>AC ΑΝΑΠΑΡΑΣΤΑΣΗ ΓΝΩΣΗΣ - Lecture 1

  47. Binary decompositions • Triangle preserving decomposition e.g. all-different(x1,x2,x3), quasigroups, ... GAC > PIC, gap can again be exponential GAC ~ SAC, strongPC • PIC is very strong consistency to be achieving at each node • GAC can do even better than this! • decomposition carries a very large price ΑΝΑΠΑΡΑΣΤΑΣΗ ΓΝΩΣΗΣ - Lecture 1

  48. Binary decompositions • Experimental results • quasigroup completion • quasigroup existence • Quasigroup is a Latin square • completion is completing partially filled square • existence is finding one with additional properties ΑΝΑΠΑΡΑΣΤΑΣΗ ΓΝΩΣΗΣ - Lecture 1

  49. Binary decompositions • Modelling the quasigroup problem • n2 vars, each with domain of size n • Non-binary model • 2n all-different constraints (one for each row and column) • Binary decomposition • 2n cliques of not-equals constraints ΑΝΑΠΑΡΑΣΤΑΣΗ ΓΝΩΣΗΣ - Lecture 1

  50. Binary decompositions • Quasigroup completion • Gomes & Selman report “heavy-tailed” distributions • Maintaining AC on binary decomposition • problems often take long time to solve • Maintaining GAC on all-different • almost all problems trivial • Quasigroup existence • of interest to design theory • Open results first proved by computer • in some cases, only ever proved by computer • Maintaining GAC very competitive • compared to specialized model finders like FINDER, SEM ΑΝΑΠΑΡΑΣΤΑΣΗ ΓΝΩΣΗΣ - Lecture 1

More Related