Constraint Satisfaction Problems

1. Constraint Satisfaction Problems Chapter 6

2. Outline • Constraint Satisfaction Problems (CSP) • Backtracking search for CSPs • Local search for CSPs

3. Constraint satisfaction problems (CSPs) • Standard search problem: • state is a "black box“ – any data structure that supports successor function, heuristic function, and goal test • CSP: • state is defined by variablesXi with values from domainDi • goal test is a set of constraints specifying allowable combinations of values for subsets of variable • Simple example of a formal representation language • Allows useful general-purpose algorithms with more power than standard search algorithms

4. Example: Map-Coloring • VariablesWA, NT, Q, NSW, V, SA, T • DomainsDi = {red,green,blue} • Constraints: adjacent regions must have different colors • e.g., WA ≠ NT, or (WA,NT) in {(red,green),(red,blue),(green,red), (green,blue),(blue,red),(blue,green)}

5. Example: Map-Coloring • Solutions are complete and consistent assignments, e.g., WA = red, NT = green,Q = red,NSW = green,V = red,SA = blue,T = green

6. Constraint graph • Binary CSP: each constraint relates two variables • Constraint graph: nodes are variables, arcs are constraints

7. Components of CSP • CSP = Constraint Satisfaction Problem • Given (V, D, C) (1) V: a finite set of variables (2) D: a domain of possible values (often finite) (3) C: a set of constraints that limit the values the variables can take on • A solution is an assignment of a value to each variable such that all the constraints are satisfied. • Tasks might be to decide if a solution exists, to find a solution, to find all solutions, or to find the “best solution” according to some metric f.

8. SEND + MORE = MONEY Assign distinct digits to the letters S, E, N, D, M, O, R, Y such that S E N D + M O R E = M O N E Y holds.

9. SEND + MORE = MONEY Assign distinct digits to the letters S, E, N, D, M, O, R, Y such that S E N D + M O R E = M O N E Y holds. Solution 9 5 6 7 + 1 0 8 5 = 1 0 6 5 2

10. Modeling Formalize the problem as a CSP: • Variables: v1,…,vn • Domains: Z, integers • Constraints: c1,…,cm  n • problem: Find a =(v1,…,vn) nsuch that a ci , for all 1  i  m

11. A Model for MONEY • variables: { S,E,N,D,M,O,R,Y} • constraints: c1={(S,E,N,D,M,O,R,Y) 8 | 0  S,…,Y  9 } c2={(S,E,N,D,M,O,R,Y) 8 | 1000*S + 100*E + 10*N + D + 1000*M + 100*O + 10*R + E = 10000*M + 1000*O + 100*N + 10*E + Y}

12. A Model for MONEY (cont.) • more constraints c3= {(S,E,N,D,M,O,R,Y) 8 | S  0 } c4= {(S,E,N,D,M,O,R,Y) 8 | M  0 } c5= {(S,E,N,D,M,O,R,Y) 8 | S…Y all different}

13. Solution for MONEY c1={(S,E,N,D,M,O,R,Y) 8 | 0S,…,Y9 } c2={(S,E,N,D,M,O,R,Y) 8 | 1000*S + 100*E + 10*N + D + 1000*M + 100*O + 10*R + E = 10000*M + 1000*O + 100*N + 10*E + Y} c3= {(S,E,N,D,M,O,R,Y) 8 | S  0 } c4= {(S,E,N,D,M,O,R,Y) 8 | M  0 } c5= {(S,E,N,D,M,O,R,Y) 8 | S…Y all different} Solution: (9,5,6,7,1,0,8,2)8

14. Varieties of CSPs • Discrete variables • finite domains: • n variables, domain size d  O(dn) complete assignments • e.g., Boolean CSPs, incl.~Boolean satisfiability (NP-complete) • infinite domains: • integers, strings, etc. • e.g., job scheduling, variables are start/end days for each job • need a constraint language, e.g., StartJob1 + 5 ≤ StartJob3 • Continuous variables • e.g., start/end times for Hubble Space Telescope observations • linear constraints solvable in polynomial time by linear programming

15. Varieties of constraints • Unary constraints involve a single variable, • e.g., SA ≠ green • Binary constraints involve pairs of variables, • e.g., SA ≠ WA • Higher-order constraints involve 3 or more variables, • e.g., cryptarithmetic column constraints

16. Example: Cryptarithmetic • Variables: F T U W R O X1 X2 X3 • Domains: {0,1,2,3,4,5,6,7,8,9} • Constraints: Alldiff (F,T,U,W,R,O) • O + O = R + 10 ·X1 • X1 + W + W = U + 10 · X2 • X2 + T + T = O + 10 · X3 • X3 = F, T ≠ 0, F≠ 0

17. Real-world CSPs • Assignment problems • e.g., who teaches what class • Timetabling problems • e.g., which class is offered when and where? • Transportation scheduling • Factory scheduling • Notice that many real-world problems involve real-valued variables

18. Standard search formulation (incremental) Let's start with the straightforward approach, then fix it States are defined by the values assigned so far • Initial state: the empty assignment { } • Successor function: assign a value to an unassigned variable that does not conflict with current assignment  fail if no legal assignments • Goal test: the current assignment is complete • This is the same for all CSPs • Every solution appears at depth n with n variables use depth-first search • Path is irrelevant, so can also use complete-state formulation • b = (n - l )d at depth l, hence n! ·dn leaves

19. Backtracking search • Variable assignments are commutative}, i.e., [ WA = red then NT = green ] same as [ NT = green then WA = red ] • Only need to consider assignments to a single variable at each node  b = d and there are $d^n$ leaves • Depth-first search for CSPs with single-variable assignments is called backtracking search • Backtracking search is the basic uninformed algorithm for CSPs • Can solve n-queens for n≈ 25

20. Backtracking search

21. Backtracking example

22. Backtracking example

23. Backtracking example

24. Backtracking example

25. Problems with backtracking • Thrashing: keep repeating the same failed variable assignments • Consistency checking can help • Intelligent backtracking schemes can also help • Inefficiency: can explore areas of the search space that aren’t likely to succeed • Variable ordering can help

26. Improving backtracking efficiency • General-purpose methods can give huge gains in speed: • Which variable should be assigned next? • In what order should its values be tried? • Can we detect inevitable failure early?

27. Most constrained variable • Most constrained variable: choose the variable with the fewest legal values • a.k.a. minimum remaining values (MRV) heuristic

28. Most constraining variable • Tie-breaker among most constrained variables • Most constraining variable: • choose the variable with the most constraints on remaining variables

29. Least constraining value • Given a variable, choose the least constraining value: • the one that rules out the fewest values in the remaining variables • Combining these heuristics makes 1000 queens feasible

30. Forward checking • Idea: • Keep track of remaining legal values for unassigned variables • Terminate search when any variable has no legal values

31. Forward checking • Idea: • Keep track of remaining legal values for unassigned variables • Terminate search when any variable has no legal values

32. Forward checking • Idea: • Keep track of remaining legal values for unassigned variables • Terminate search when any variable has no legal values

33. Forward checking • Idea: • Keep track of remaining legal values for unassigned variables • Terminate search when any variable has no legal values

34. Constraint propagation • Forward checking propagates information from assigned to unassigned variables, but doesn't provide early detection for all failures: • NT and SA cannot both be blue! • Constraint propagation repeatedly enforces constraints locally

35. Arc consistency • Simplest form of propagation makes each arc consistent • X Y is consistent iff for every value x of X there is some allowed y

36. Arc consistency • Simplest form of propagation makes each arc consistent • X Y is consistent iff for every value x of X there is some allowed y

37. Arc consistency • Simplest form of propagation makes each arc consistent • X Y is consistent iff for every value x of X there is some allowed y • If X loses a value, neighbors of X need to be rechecked

38. Arc consistency • Simplest form of propagation makes each arc consistent • X Y is consistent iff for every value x of X there is some allowed y • If X loses a value, neighbors of X need to be rechecked • Arc consistency detects failure earlier than forward checking • Can be run as a preprocessor or after each assignment

39. Arc consistency algorithm AC-3 • Time complexity: O(n2d3)

40. X1 1 2 3 X2 X3 4 5 X4 X5 Variables: X1 X2 X3 X4 X5 Domains: D1 = {hoses, laser, sheet, snail,steer} D2 = {hike, aron, keet, earn, same} D3 = {run, sun, let, yes, eat, ten} D4 = {hike, aron, keet, earn, same} D5 = {no, be, us, it} Constraints (explicit/extensional): C12 = {(hoses,same), (laser,same), (sheet,earn), (steer,earn)} C13 = ...

41. 1 2 3 4 5 Constraint propagation: XWORD example X1 X2 X4 hoses hike hike laser aron aron sheet keet keet snail earn earn steer same same

42. K-consistency • Arc consistency does not detect all inconsistencies: • Partial assignment {WA=red, NSW=red} is inconsistent. • Stronger forms of propagation can be defined using the notion of k-consistency. • A CSP is k-consistent if for any set of k-1 variables and for any consistent assignment to those variables, a consistent value can always be assigned to any kth variable. • E.g. 1-consistency or node-consistency • E.g. 2-consistency or arc-consistency • E.g. 3-consistency or path-consistency

43. Strong K-consistency • A CSP (as a hyrpergraph) is strongly k-consistent if • It is k-consistent and • Is also (k-1) consistent, (k-2) consistent, … all the way down to 1-consistent. • This is ideal if we have an easy way to check strong k-consistency, since a solution can be found in time O(nd). • That is, for any CSP that is strongly K-consistent, if we find an appropriate variable ordering (one with “small enough” branching factor), we can solve the CSP without backtracking • YET no free lunch: any algorithm for establishing n-consistency must take time exponential in n, in the worst case.

44. Example: K-consistency • C: w  {1,2,3,4} x  { 2,3,4} y  { 2,3,4} z  { 2,3,4} • P: all_different(w,x,y,z) • Arc Consistency: After w = 2 (or 3, 4), and x = 3 (or 4), AC3 will detect a conflict. • 3-Consistency: After w = 2 (or 3, 4), it will detect a conflict. • 4-Consistency: 2,3,4 will be removed from domain(w).

45. The Sudoku Puzzle • Number place • Main properties • NP-complete [Yato 03] • Well-formed Sudoku: has 1 solution [Simonis 05] • Minimal Sudoku • In a 9x9 Sudoku: smallest known number of givens is 17 [Royle] • Symmetrical puzzles • Many axes of symmetry • Position of the givens, not their values • Often makes the puzzle non-minimal • Level of difficulties • Varied ranking systems exist • Mimimality and difficulty not related #15 on Royle’s web site

46. Sudoku as a CSP (9x9 puzzles) • Variables are the cells • Domains are sets {1,…,9} • Two models • Binary constraints: 810 all-different binary constraint between variables • Non-binary constraints: 27 all-different 9-ary constraints

47. Solving Sudoku as a CSP • Search • Builds solutions by enumerating consistent combinations • Constraint propagation • Removes values that do not appear in a solution • Example, arc-consistency:

48. Search • Backtrack search • Systematically enumerate solutions by instantiating one variable after another • Backtracks when a constraint is broken • Is sound and complete (guaranteed to find solution if one exists) • Propagation • Cleans up domain of ‘future’ variables, during search, given current instantiations • Forward Checking (FC): weak form of arc-consistency • Maintaining Arc-Consistency (MAC): arc-consistency

49. Forward Checking (FC) • Forward Checking on the binary constraints

50. Maintaining Arc-Consistency (MAC) • Arc Consistency on the binary constraints