Constraint Satisfaction Problems

1. Constraint Satisfaction Problems ECE457 Applied Artificial Intelligence Spring 2008 Lecture #4

2. Outline • Defining constraint satisfaction problems (CSP) • CSP search algorithms • Russell & Norvig, chapter 5 ECE457 Applied Artificial Intelligence R. Khoury (2008) Page 2

3. States • What do we know about states? • A state might be a goal (goal test) • A state has a value (cost or payoff) • An agent moves from state to state using actions • The state space can be discreet or continuous • Ties in with the problem definition • Initial state, goal test, set of actions and their costs defined in problem ECE457 Applied Artificial Intelligence R. Khoury (2008) Page 3

4. Definitions • A constraint-satisfaction problem has • A set of variables • V = {X1, X2, …, Xn} • Each variable has a domain of values • Xi Di = {di1, di2, …, din} • A set of constraints on the values each variable can take • C = {C1, C2, …, Cm} • A state is a set of assignment of values • S1 = {X1 = d12, X4 = d45} ECE457 Applied Artificial Intelligence R. Khoury (2008) Page 4

5. Definitions • Consistent (legal) state • Does not violate any constraints • Complete state • All variables have a value • Goal state • Consistent and complete • Might not exist • Proof of inconsistency ECE457 Applied Artificial Intelligence R. Khoury (2008) Page 5

6. Example: 8-queen • Variables: 64 squares, number of queens • V = {S1,1, S1,2, …, S8,8, Number_of_queens} • Values: Queen or no queen • Si,j DS = {queen, empty} • Number_of_queens  DN = [0, 64] • Constraints: Attacks, queen count • {Number_of_queens = 8, Si,j = queen Si,j+n = empty, Si,j = queen Si+n,j = empty, Si,j = queen Si+n,j+n = empty} • States: All board configurations • 2.8x1014 complete states • 1.8x1014 complete states with 8 queens • 92 complete and consistent states • 12 unique complete and consistent states ECE457 Applied Artificial Intelligence R. Khoury (2008) Page 6

7. Example: 8-queen ECE457 Applied Artificial Intelligence R. Khoury (2008) Page 7

8. Example: The Einstein Puzzle • There are 5 houses in 5 different colours. In each house lives a person with a different nationality. These 5 owners drink a certain beverage, smoke a certain brand of cigar and keep a certain pet. No owners have the same pet, smoke the same brand of cigar or drink the same drink. Who keeps the fish? • The English lives in a red house. • The Swede keeps dogs as pets. • The Dane drinks tea. • The green house is on the left of the white house. • The green house owner drinks coffee. • The person who smokes Pall Mall rears birds. • The owner of the yellow house smokes Dunhill. • The man living in the house right in the centre drinks milk. • The Norwegian lives in the first house. • The man who smokes Blend lives next to the one who keeps cats. • The man who keeps horses lives next to the man who smokes Dunhill. • The owner who smokes Blue Master drinks beer. • The German smokes Prince. • The Norwegian lives next to the blue house. • The man who smokes Blend has a neighbour who drinks water. ECE457 Applied Artificial Intelligence R. Khoury (2008) Page 8

9. 1 2 3 4 5 Example: The Einstein Puzzle • 25 variables • V = {N1, …, N5, C1, …, C5, D1, …, D5, S1, …, S5, P1, …, P5} • Domains • Ni {English, Swede, Dane, Norwegian, German} • Ci {green, yellow, blue, red, white} • Di {tea, coffee, milk, beer, water} • Si {Pall Mall, Dunhill, Blend, Blue Master, Prince} • Pi {dog, cat, horse, fish, birds} ECE457 Applied Artificial Intelligence R. Khoury (2008) Page 9

10. Example: The Einstein Puzzle • The Norwegian lives in the first house. • N1 = Norwegian • The English lives in a red house. • (Ni = English)  (Ci = Red) • The green house is on the left of the white house. • (Ci = green)  (Ci+1 = white) • (C5≠ green) • (C1≠ white) ECE457 Applied Artificial Intelligence R. Khoury (2008) Page 10

11. Example: Map Colouring • Colour map of the provinces of Australia • 3 colours (red, green, blue) • No adjacent provinces of the same colour • Define CSP ECE457 Applied Artificial Intelligence R. Khoury (2008) Page 11

12. Example: Map Colouring CSP • Variables • {WA, NT, SA, Q, NSW, V, T} • Domain • {R, G, B} • Constrains • {WA  NT, WA SA, NT SA, NT Q, SA Q, SA NSW, SA V, Q NSW, NSW V} ECE457 Applied Artificial Intelligence R. Khoury (2008) Page 12

13. Solving CSP • Iterative improvement methods • Start with random complete state, improve until consistent • Tree searching • Start with empty state, make consistent variable assignments until complete ECE457 Applied Artificial Intelligence R. Khoury (2008) Page 13

14. Iterative Improvement • Min-Conflict Algorithm: • Assign to each variable a random value • While state not consistent • Pick a variable (randomly or with a heuristic) • Find its values that minimize the number of violated constraints • If there is only one such value • Assign that value to the variable • If there are several values • Assign a random value from that set to the variable ECE457 Applied Artificial Intelligence R. Khoury (2008) Page 14

15. Min-Conflict Algorithm ECE457 Applied Artificial Intelligence R. Khoury (2008) Page 15

16. Min-Conflict Algorithm • Advantage • Very efficient • Disadvantages • Only searches states that are reachable from the initial state • Might not search all state space • Does not allow worse moves • Might get stuck in a local optimum • Not complete! • Might not find a solution even if one exists ECE457 Applied Artificial Intelligence R. Khoury (2008) Page 16

17. Tree Search • Formulate CSP as tree • Root node: no variables are assigned a value • Action: assign a value if it does not violate any constraints • Solution node at depth n for n-variable problem ECE457 Applied Artificial Intelligence R. Khoury (2008) Page 17

18. Backtracking Search • Start with empty state • While not complete • Pick a variable (randomly or with heuristic) • If it has a value that does not violate any constraints • Assign that value • Else • Go back to previous variable • Assign it another value ECE457 Applied Artificial Intelligence R. Khoury (2008) Page 18

19. Backtracking Search • Depth-first search algorithm • Goes down one variable at a time • In a dead end, back up to last variable whose value can be changed without violating any constraints, and change it • If you backed up to the root and tried all values, then there are no solutions • Algorithm complete • Will find a solution if one exists • Will expand the entire (finite) search space if necessary • Depth-limited search with limit = n ECE457 Applied Artificial Intelligence R. Khoury (2008) Page 19

20. Example: Backtracking Search WA = G WA = R WA = B NT = G NT = B Q = B Q = R NSW = G NSW = G SWA = B NSW = R SA = ? SA = B SA = ? V = R T = G T = B T = R ECE457 Applied Artificial Intelligence R. Khoury (2008) Page 20

21. Conflict-Directed Backjumping • Suppose we colour Australia in this order: • WA – R • NSW – R • T – B • NT – B • Q – G • SA - ? • Dead-end at SA • No possible solution with WA = NSW • Backtracking will try to change T on the way, even though it has nothing to do with the problem, before going to NSW ECE457 Applied Artificial Intelligence R. Khoury (2008) Page 21

22. Conflict-Directed Backjumping • Backtracking goes back one level in the search tree at a time • Chronological backtrack • Not rational in cases where the previous step is not involved to the conflict • Conflict-directed backjumping (CBJ) • Should go back to a variable involved in the conflict • Skip several levels if needed to get there • Non-chronological backtrack ECE457 Applied Artificial Intelligence R. Khoury (2008) Page 22

23. Conflict-Directed Backjumping • Maintain a conflict set for each variable • List of previously-assigned variables that are related by constraints • When we hit a dead-end, backjump to the deepest variable in the conflict set ECE457 Applied Artificial Intelligence R. Khoury (2008) Page 23

24. Conflict-Directed Backjumping • Learn from the conflict by updating the conflict set of the variable we jumped to • Conflict at Xj, backjump to Xi • conf(Xi)={X1,X2,X3} conf(Xj)={X3,X4,X5,Xi} • conf(Xi) = conf(Xi)  conf(Xj) – {Xi} • conf(Xi) = {X1,X2,X3,X4,X5} • Xiabsorbed the conflict set of Xj ECE457 Applied Artificial Intelligence R. Khoury (2008) Page 24

25. Conflict-Directed Backjumping • SA backjump to Q • conf(Q) = {WA,NSW,NT} • Meaning: “There is no consistent solution from Q onwards, given the preceding assignments of WA, NSW and NT together” conf(NSW) = {WA} conf(NT) = {WA,NSW} conf(Q) = {WA,NSW,NT} • Q backjump to NT • conf(NT) = {WA,NSW} • Meaning: “There is no consistent solution from NT onwards, given the preceding assignments of WA and NSW together” • NT backjump to NSW • conf(NSW) = {WA} • Skips T, which is irrelevant in this conflict • Discovers the relationship between NSW and WA, which is not present in our constraints ECE457 Applied Artificial Intelligence R. Khoury (2008) Page 25

26. Heuristics • Backtracking and CDJ searches are variations of depth-limited search • Uninformed search technique • Can we make it an informed search? • Add some heuristics • Which variable to assign next? • In which order should the values be tried? • How to detect dead-ends early? ECE457 Applied Artificial Intelligence R. Khoury (2008) Page 26

27. Variable & Value Heuristics • Most constrained variable • Choose the variable with the fewest legal values remaining in its domain • aka Minimum remaining values • Most constraining variable • Choose the variable that’s part of the most constraints • Useful to pick first variable to assign • aka Degree heuristic • Least constraining variable • Pick the variable that’s part of the fewest constrains • Keeps maximum flexibility for future assignments ECE457 Applied Artificial Intelligence R. Khoury (2008) Page 27

28. Variable & Value Heuristics Values: 2Constraints: 2 Most constrained variable Values: 2Constraints: 2 Values: 2Constraints: 1 Values: 2Constraints: 2 Most constrained & least constraining variable Most constrained variable Most constraining variable Most constraining variable Most constrained variable Values: 2Constraints: 1 Least constraining variable ECE457 Applied Artificial Intelligence R. Khoury (2008) Page 28

29. Forward Checking • How to detect dead-ends early? • Keep track of the domain of unassigned variables • Use constraints to prune domain of unassigned variables • Backtrack when a variable has an empty domain • Do not waste time exploring further ECE457 Applied Artificial Intelligence R. Khoury (2008) Page 29

30. Example: Forward Checking ECE457 Applied Artificial Intelligence R. Khoury (2008) Page 30

31. Problem with Forward Checking ECE457 Applied Artificial Intelligence R. Khoury (2008) Page 31

32. Constraint Propagation • Propagate the consequences of a variable’s constraints onto other variables • Represent CSP as constraint graph • Nodes are variables • Arcs are constraints • Check for consistency NT Q WA NSW SA V T ECE457 Applied Artificial Intelligence R. Khoury (2008) Page 32

33. Checking for Consistency • Node consistency • Unary constraint (e.g. NT  G) • A node is consistent if and only if all values in its domain satisfy all unary constraints • Arc consistency • Binary constraint (e.g. NT  Q) • An arc Xi Xj or (Xi, Xj) is consistent if and only if, for each value a in the domain of Xi, there is a value b in the domain of Xj that is permitted by the binary constraints between Xiand Xj. • Path consistency • Can be reduced to arc consistency ECE457 Applied Artificial Intelligence R. Khoury (2008) Page 33

34. Checking for Consistency • Node consistency • Simply scan domain of values of each variable and remove those that are not valid • Arc consistency • Examine edges and delete values from domains to make arcs consistent ECE457 Applied Artificial Intelligence R. Khoury (2008) Page 34

35. Checking for Consistency • Remove inconsistent values from a variable’s domain • Backtrack if empty domain • Maintaining node and arc consistency reduces the size of the tree • More computationally expensive than Forward Checking, but worth it ECE457 Applied Artificial Intelligence R. Khoury (2008) Page 35

36. AC-3 Algorithm • Keep queue of arcs (Xi, Xj) to be checked for consistency • If checking an arc removes a value from the domain of Xi, then all arcs (Xk, Xi) are reinserted in the queue ECE457 Applied Artificial Intelligence R. Khoury (2008) Page 36

37. AC-3 Algorithm • Add all arcs to queue • While queue not empty • Get next arc (Xi, Xj) from queue • For each value di of Xi • If there is no consistent value dj in Xj • Delete di • If a value di was deleted • For each neighbour Xk of Xi • Add arc (Xk, Xi) to queue ECE457 Applied Artificial Intelligence R. Khoury (2008) Page 37

38. AC-3 Algorithm Partial Arc Queue (SA,WA) (NT,WA) (SA,V) (NSW,V) (NT,SA) (NSW,SA) (Q,SA) (Q,NT) (Q,NSW) (WA,SA) (V,SA) (SA,NT) (WA,NT) (V,NSW) (SA,NSW) ECE457 Applied Artificial Intelligence R. Khoury (2008) Page 38

39. AC-3 Algorithm • Advantages • Prunes tree, reduces amount of searching • For n-node CSP that’s n-consistent, solution is guaranteed with no backtracking • Disadvantage • Computationally expensive • If pruning takes longer than searching, it’s not worth it ECE457 Applied Artificial Intelligence R. Khoury (2008) Page 39