Advanced Constraint Networks for Problem-Solving

1. Constraint Networks ( slides courtesy of Natalia Flerova, based on slides courtesy of Rina Dechter)

2. Reading • Russell and Norvig • Chapter 6 (3rd ed.); Chapter 5 (2nd ed.) • For next week: • Chapter 7 (for Tuesday) • Chapter 8 (for Thursday)

3. Good source of advanced information • Rina Dechter, • Constraint Processing, • Morgan Kaufmann

4. Outline • Constraint Satisfaction Problems (CSPs): • Definition, and simple modeling examples • Representing constraints • Consistency algorithms (arc-, path- and i-consistency) • General search strategy • Look-ahead methods

5. E A B red green red black green red green black black green black red A D B F G C Constraint Satisfaction Example: map coloring Variables - countries (A,B,C,etc.) Values - colors (e.g., red, green, black) Constraints:

6. Constraint Network; Definition • A constraint network is: R=(X,D,C) • Xvariables • Ddomains • Cconstraints • Rexpresses allowed tuples over scopes • A solution is an assignment to all variables that satisfies all constraints (join of all relations). • Tasks: consistency?, one or all solutions, counting, optimization 6

7. 4 2 Example The 4-queen problem • Standard CSP formulation of the problem: • Variables: each row is a variable. Place 4 Queens on a chess board of 4x4 such that no two queens reside in the same row, column or diagonal. 1 2 3 4 Q Q Q Q Q Q Q Q Q Q Q Q • Domains: ( ) • Constraints: There are = 6 constraints involved: • Constraint Graph :

8. Example: configuration and design Spring 2009 10

9. Configuration and design • Want to build: recreation area, apartments, houses, cemetery, dump • Recreation area near lake • Steep slopes avoided except for recreation area • Poor soil avoided for developments • Highway far from apartments, houses and recreation • Dump not visible from apartments, houses and lake • Lots 3 and 4 have poor soil • Lots 3, 4, 7, 8 are on steep slopes • Lots 2, 3, 4 are near lake • Lots 1, 2 are near highway Spring 2009 11

10. Outline • CSP: Definition, and simple modeling examples • Representing constraints • Consistency algorithms (arc-, path- and i-consistency) • General search strategy • Look-ahead methods

11. Constraint’s representations • Relation: allowed tuples • Algebraic expression: • Propositional formula: Spring 2009 13

12. Operations with relations • Intersection • Union • Difference • Selection • Projection • Join • Composition

13. Figure 1.8: Example of set operations intersection, union, and difference applied to relations. Spring 2009 15

14. selection, projection, and join operations on relations. Spring 2009 16

15. Constraint Graphs: Primal, Dual and Hypergraphs • A (primal) constraint graph: a node per variable • arcs connect constrained variables. • A dual constraint graph: a node per constraint’s scope, an arc connect nodes sharing variables =hypergraph Spring 2009 17

16. Outline • CSP: Definition, and simple modeling examples • Representing constraints • Consistency algorithms (arc-, path- and i-consistency) • General search strategy • Look-ahead methods

17. Consistency methods • Constraint propagation – inferring new constraints • Can get such an explicit network that the search will find the solution without dead-ends. • Approximation of inference: • Arc, path and i-consistency • Methods that transform the original network into a tighter and tighter representations

18. Arc-consistency - infer constraints based on pairs of variables X Y  1, 2, 3 1, 2, 3 1  X, Y, Z, T  3 X  Y Y = Z T  Z X T  = 1, 2, 3 1, 2, 3  T Z Insures that every legal value in the domain of a single variable has a legal match In the domain of any other selected variable

19. 1 3 2 3 Arc-consistency X Y  1  X, Y, Z, T  3 X  Y Y = Z T  Z X  T  =  T Z

20. Arc-consistency

21. Revise for arc-consistency

22. AC-1 • Complexity (Mackworth and Freuder, 1986): • e = number of arcs, n variables, k values • (ek^2, each loop, nk number of loops), best-case = ek, • Arc-consistency is:

23. AC-3 • Complexity: • Best case O(ek), since each arc may be processed in O(2k) • e = number of arcs, n variables, k values

24. Sudoku –Constraint Satisfaction • Variables: empty slots • Domains = {1,2,3,4,5,6,7,8,9} • Constraints: • 27 all-different • Constraint • Propagation • Inference 2 34 6 2 Each row, column and major block must be alldifferent “Well posed” if it has unique solution: 27 constraints

25. Path-consistency

26. Path-consistency

27. Revise-3 • Complexity: O(k^3) • Best-case: O(t) • Worst-case O(tk) • e = number of arcs, n variables, k values, t = tightness: number of tuples in the constraint relation

28. PC-1 • Complexity: • e = number of arcs, n variables, k values, t = tightness: number of tuples in the constraint relation

29. PC-2 • Complexity: • Optimal PC-4: • (each pair deleted may add: 2n-1 triplets, number of pairs: O(n^2 k^2)  size of Q is O(n^3 k^2), processing is O(k^3)) • e = number of arcs, n variables, k values, t = tightness: number of tuples in the constraint relation

30. Example: before and after path-consistency • PC-1 requires 2 processings of each arc while PC-2 may not • Can we do path-consistency distributedly?

31. I-consistency

32. Outline • CSP: Definition, and simple modeling examples • Representing constraints • Consistency algorithms (arc-, path- and i-consistency) • General search strategy • Look-ahead methods

33. Backtracking algorithm • No matter how much we reason about the problem – we are still left with choices → need to search the space of possible choices. • Backtracking search – assign values to each variable in turn, making sure each assigned value is consistent with values assigned thus far. • Encounter variable with no domain value consistent with previous assignment – dead-end → backtrack – change the value of previous variables.

34. Backtracking

39. The search space • A tree of all partial solutions • A partial solution: (a1,…,aj) satisfying all relevant constraints • The size of the underlying search space depends on: • Variable ordering • Level of consistency possessed by the problem

40. Search space and the effect of ordering Z evenly divides x,y,l 20 legal states 1 dead-end leaf 48 legal states 18 dead-end leaves

41. Dependency on consistency level • After arc-consistency z=5 and l=5 are removed • After path-consistency • R’_zx • R’_zy • R’_zl • R’_xy • R’_xl • R’_yl

42. Backtrack-free network

43. Complexity of extending a partial solution: Complexity of consistent O(e log t), t bounds tuples, e constraints Complexity of selectvalue O(e k log t) Backtracking

44. Outline • CSP: Definition, and simple modeling examples • Representing constraints • Consistency algorithms (arc-, path- and i-consistency) • General search strategy • Look-ahead methods

45. Look-ahead: value orderings • Intuition: • Choose value least likely to yield a dead-end • Approach: apply propagation at each node in the search tree • Forward-checking • (check each unassigned variable separately) • Maintaining arc-consistency (MAC) • (apply full arc-consistency) • Full look-ahead • One pass of arc-consistency (AC-1) • Partial look-ahead • directional-arc-consistency

46. Forward-checking • The most limited form of constraint propagation during search • Propagates the effect of a tentative value selection to each future variable separately • If domain of one of the future variables becomes empty – value is not selected → try next value.

47. Forward-checking example

48. Forward-checking • Complexity of selectValue-forward-checking at each node:

49. Arc-consistency look-ahead(Gashnig, 1977) • Applies full arc-consistency on all un-instantiated variables following each value assignment to the current variable. • Complexity: • If optimal arc-consistency is used: • SelectValue-Arc-Consistency: The repeat...until loop – essentially AC-1 with some variables instantiated. • Maintaining-Arc-Consistency (MAC): Performs full arc-consistency after each domain value is rejected.

50. Arc-consistency look-ahead:(maintaining arc-consistency MAC)