Chapter 5. Constraint Satisfaction Problems

1. Chapter 5.Constraint Satisfaction Problems CS570, Artificial Intelligence Group 11: Jin H. Park, Mi H. Koo, Chon H. Lee

2. Contents • What is CSP? • Solving CSP by search • Heuristics for CSP • Summary

3. DefinitionWhat is CSP? • A special kind of problem: • States are defined by values of a fixed set of variables variables : {X1, X2, …, Xn} Domain according to a variable Xi (Di) • Goal test is defined by constraints on variable values constraints : {C1, C2, … Cn} • Problem structure is represented by constraint graph • A problem where one is given (i) a (finite) set of variables, (ii) a function which maps every variable to a domain, and (iii) a set of constraints. Each constraint restricts the combination of values that a set of variables may take simultaneously.[http://cswww.essex.ac.uk/CSP/tutorial.html]

4. Has different color VAR: WA,NT,SA,Q,NSW,V,T TerminologiesWhat is CSP? • Consistent (or legal) assignment • An assignment that does not violate any constraints • Complete assignment • An assignment in which every variable is mentioned • Solution • A complete and consistent assignment • A complete assignment that satisfies all the constraints • Constraint graph • Node : variable • Arc : constraint

5. Incremental formulation as standard search problemWhat is CSP? • States • Set of value assignments to some or all of the variables • Initial state • The empty assignment {} • Successor function • Value assignment to any unassigned variable without conflicts with previously assigned variables • Goal test • Finding out the current assignment is complete • Path Cost • A constant cost

6. Benefits from treating a problem as a CSPWhat is CSP? • The successor function and goal test can be written in a generic way that applies to all CSPs. • We can develop effective, generic heuristics that require no additional, domain-specific expertise. • The structure of the constraint graph can be used to simplify the solution process.

7. Examples of CSP(1/2) – 8-queen problemWhat is CSP? • Variables • {Q1,Q2,Q3,Q4,Q5,Q6,Q7,Q8} • Domain • Q1, …, Q8 ∊ {(1,1), (1,2), …, (8,8)} • Constraint set • No queen attacks any other

8. Examples of CSP(2/2) – map-coloring problemWhat is CSP? • Variables • {WA, NT, Q, NSW, V, SA,T} • Domain • WA, NT, …, SA ∊ {red, green, blue} • Constraints • Neighboring regions must have distinct colors. ( WA ≠ NT, WA ≠ SA, …)

9. The classification of CSP - by variable’s domain What is CSP? • Discrete variables • Finite domains • 8-queen problem, map-coloring problem • Boolean CSP • Includes 3SAT(NP-complete) => Don’t expect that in the worst case, we can solve finite-domain CSP in less than exponential time. • Infinite domains • Job scheduling • Continuous variables • Linear programming problem

10. The classification of CSP - by constraints’ characteristic What is CSP? • Absolute Constraint • Unary Constraint • Ex. SA ≠ green • Binary Constraint • Ex. SA ≠ WA • Higher-order Constraint • Ex. Crypt-arithmetic column constraints • Preference Constraint • ex) Prof. X might prefer teaching in the morning. • Many real-world CSP • Solved with optimization search methods

11. Backtracking SearchSolving CSP by search(1) • BFS vs. DFS • BFS  terrible! • A tree with n!dn leaves : (nd)*((n-1)d)*((n-2)d)*…*(1d) = n!dn • Reduction by commutativity of CSP • A solution is not in the permutations but in combinations. • A tree with dn leaves • DFS • Used popularly • Every solution must be a complete assignment and therefore appears at depth n if there are n variables • The search tree extends only to depth n. • A variant of DFS : Backtracking search • Chooses values for one variable at a time • Backtracts when failed even before reaching a leaf. • Better than BFS due to backtracking, but still inefficient!!

12. Backtracking SearchSolving CSP by search(2) function BACKTRACKING-SEARCH (csp) returns a solution, or failure return RECURSIVE-BACKTRACKING({}, csp) function RECURSIVE-BACKTRACKING(assignment, csp) returns a solution, or failure ifassignment is complete then returnassignment var SELECT-UNASSIGNED-VARIABLE(VARIABLES[csp], assignment, csp) for eachvalue in ORDER-DOMAIN-VALUES(var, assignment, csp) do ifvalue is consistent with assignment according to CONSTRAINTS[csp] then add {var=value} to assignment result RECURSIVE-BACKTRACKING(assignment, csp) ifresult != failurethen returnresult remove {var = value} from assignment return failure ☜ BACKTRACKING OCCURS HERE!!

13. Variable & value ordering to increase the likelihood to success Early failure-detection to decrease the likelihood to fail Restructuring to reduce the problem’s complexity Backtracking SearchImproving backtracking efficiency • Which variable should be assigned next? • MRV heuristic • In what order should its values be tried? • LCV heuristic • Can we detect inevitable failure early? • Forward checking • Constraint propagation (Arc Consistency) • When a path fails, can the search avoid repeating this failure? • Backjumping • Can we take advantage of problem structure? • Tree-structured CSP

14. Backtracking SearchImproving backtracking efficiency function BACKTRACKING-SEARCH (csp) returns a solution, or failure return RECURSIVE-BACKTRACKING({}, csp) function RECURSIVE-BACKTRACKING(assignment, csp) returns a solution, or failure ifassignment is complete then returnassignment var SELECT-UNASSIGNED-VARIABLE(VARIABLES[csp], assignment, csp) for eachvalue in ORDER-DOMAIN-VALUES(var, assignment, csp) do ifvalue is consistent with assignment according to CONSTRAINTS[csp] then add {var=value} to assignment result RECURSIVE-BACKTRACKING(assignment, csp) ifresult != failurethen returnresult remove {var = value} from assignment return failure

15. MRV & LCVHeuristics for variable & value ordering • Variable selection • MRV(minimum remaining values) heuristic • Choose the variable with the fewest legal values. • Degree heuristic • In first variable selection, MRV heuristic doesn’t help at all. • Use degree heuristic (selects the variable with the largest degree.)  Tie-breaker !! • Value ordering • Least-Constraining-value heuristic  not to rule out neighbors

16. MRV & LCV – an example for degree heuristic Heuristics for variable & value ordering NT NT Q SA SA SA NT ={R, G} Q = {R, G} WA = {R, G} SA = B NSW = {R, G} V = {R, G} T = {R, G, B} 3 3 2 5 3 2 0 NT ={R, G, B},3 Q = {R, G, B},3 WA = {R, G, B},2 SA = {R, G, B},5 NSW = {R, G, B},3 V = {R, G, B},2 T = {R, G, B},0 NT = G Q = {R} WA = {R} SA = G NSW = {R, G} V = {R, G} T = {R, G, B}

17. MRV & LCV – an example for MRV Heuristics for variable & value ordering NT NT Q SA SA SA NT ={G, B} Q = {R, G, B} WA = R SA = {G, B} NSW = {R, G, B} V = {R, G, B} T = {R, G, B} NT ={R, G, B} Q = {R, G, B} WA = {R, G, B} SA = {R, G, B} NSW = {R, G, B} V = {R, G, B} T = {R, G, B} NT =G Q = {R, B} WA = R SA = {B} NSW = {R, G, B} V = {R, G, B} T = {R, G, B}

18. MRV & LCV – an example for LCVHeuristics for variable & value ordering Red in Q is least constraint value NT Q WA NT WA WA Q NT WA NT ={G, B} Q = {R, G, B} WA = R SA = {G, B} NSW = {R, G, B} V = {R, G, B} T = {R, G, B} NT ={R, G, B} Q = {R, G, B} WA = {R, G, B} SA = {R, G, B} NSW = {R, G, B} V = {R, G, B} T = {R, G, B} NT =G Q = {R, B} WA = R SA = {B} NSW = {R, G, B} V = {R, G, B} T = {R, G, B} NT =G Q = R WA = R SA = {B} NSW = {G, B} V = {R, G, B} T = {R, G, B} NT =G Q = B WA = R SA = ? NSW = {R,G} V = {R, G, B} T = {R, G, B}

19. Forwarding checking(1/4)Heuristics for early failure-detection • How can we find out which variable is minimum remaining? • What does eliminate the values from domain? NT = red, Q = green WA = {red, green, blue} SA = {red, green, blue} NSW = {red, green, blue} V = {red, green, blue} T = {red, green, blue} ⅹ ⅹ ⅹ ⅹ

20. Forwarding checking(2/4)Heuristics for early failure-detection • Forward checking • A variable X is assigned. • Looks at each unassigned variable Y connected to X by a constraint • Deletes any values of Y’s domain that is inconsistent with X • If there is any Y with empty domain, undo this assigning.  Backtrack!!  If not, MRV !! • Forward checking is an obvious partner of MRV heuristic !!  MRV heuristic is activated after forward checking.

21. Forwarding checking(3/4) – an exampleHeuristics for early failure-detection NT Q WA SA NSW V T

22. Forwarding checking(4/4) - Heuristic flowHeuristics for early failure-detection Forward Checking Yes Tie? No Degree heuristic Select the MRV Forward Checking Select the LCV Goal?

23. ⅹ o Constraint propagation: arc consistency Heuristics for early failure-detection • The (directed) arc (XY) is consistent ≡ For all x ∊ X, there is some y ∊ Y. • Detects an inconsistency that is not detected by pure F/C. • Makes CSP possibly with reduced domain. • Provides a fast method of constraint propagation • For preprocessing or propagation step • O(n2d3) • n2 : the maximum number of arcs • d : the maximum number of insertion for one node • d2 : the number of forward checking

24. O(n2d3) Constraint propagation: arc consistency – algorithmHeuristics for early failure-detection function AC-3(csp) returns the CSP, possibly with reduced domains inputs: csp, a binary CSP with variables {X1, X2, …, Xn} local variables:queue, a queue of arcs, initially all the arcs in csp whilequeue is not empty do (Xi,Xj)  REMOVE-FIRST(queue) if REMOVE-INCONSISTENT-VALUES(Xi,Xj) then foreachXkin NEIGHBORS[Xi] do add(Xk,Xi) to queue function REMOVE-INCONSISTENT-VALUES() returns true iff we remove a value removed  false for eachx in DOMAIN[Xi] do if no value y in DOMAIN[Xj] allows (x,y) to satisfy the constraint between Xiand Xj thendelete x from DOMAIN[Xi]; removed  true returnremoved

25. Special constraints – Alldiff Heuristics for early failure-detection • Problem-dependent constraints • Alldiff constraint • If there are m variables involved in the constraint and n possible distinct values, m > n  inconsistent for Alldiff constraint • An example • 3 variables: NT,SA,Q • NT, SA, Q ∊ {green, blue} • m=3 > n=2 • inconsistent WA NSW

26. Special constraints – AtmostHeuristics for early failure-detection • Resource constraint • Atmost constraint(Use limited resource) • An example • Totally, no more than 10 personnel should be assigned to jobs. • Atmost(10, PA1, PA2, PA3, PA4) • Job1~4 ∊ {3,4,5,6} ( min value = 3 ) • 3+3+3+ 3 > 10 (minimum resource 12 but available resource 10) • inconsistent • Job1~4 ∊ {2, 3,4,5,6} • 2+2+2+2 <10 (consistent) • 2+2+2+5(or 6) > 10(inconsistent) • Delete 5,6 from domain

27. BackjumpingHeuristics for early failure-detection 1 • The weakness of Backtracking • If failed in SA, backtrack to T. • But, T is not relevant to SA. • Backjumping • Backtracks to the most recent variable in the conflict set. • Conflict set : the set of variables that caused the failure • If failed in SA, backjump to V. Q NSW 2 5 V 3 Next variable T 4

28. Local Search(1/2)Min-conflicts heuristic • The initial state assigns a value to every variable, and successor function usually works by changing the value of one variable at a time. • Select the value that results in the minimum of conflicts with other variables • Min-conflicts is effective for many CSPs, particularly when given a reasonable initial state. • Ex. n-queen problem, Hubble space telescope scheduling • Specially important in scheduling problems • Airline schedule (when the schedule is changed)

29. Local Search(2/2)Min-conflicts heuristic • Example of min-conflict heuristic

30. GeneralHeuristics for complexity reduction • The only way we can possibly hope to deal with the real world is to decompose it into many sub-problems. • Independent sub-problems • Solve them respectively  if assignment Si is a solution of CSPi, then UiSi is a solution of UiCSPi • Why important? • Each CSPi has c variables from the total of n variables  n/c subproblems • dc work for each sub-problem, so total work is O(dcn/c)  linear in n !! • Without decomposition, O(dn)  exponential in n !! • Ex. n(node)=80, d(domain)=2, c(variables in subproblem)=20 • 280 vs. 4*220( 4 billion years vs. 0.4 sec) • Ex) The mainland and Tasmania in previous map-coloring. • But, rare!!!

31. Tree-structured CSPHeuristics for complexity reduction • If constraint graph is a tree, we solve the CSP in time linear in the number of variables 1. Choose a variable as root 2. Order variables from root to leaves such that every node’s parent precedes it in the ordering Makes constraint graph arc-consistent 3. For j from n down to 2, apply arc consistency to the arc(Xi, Xj) • Xi : parent of Xj Just finds a consistent value.(no backtrack because of arc-consistency) 4. For j from 1 to n, assign Xj consistently with Parent(Xj) • Time complexity : O(nd2)

32. Tree-structured CSP – an linear ordering example Heuristics for complexity reduction

33. Transforming general CSP to tree-structured CSP(1/4)Heuristics for complexity reduction • Cutset conditioning • Remove some variables S (Assign values to some variables) • S is called Cycle cutset when constraint graph is a tree after the removal of S • Time Complexity : O(dc(n-c)d2) • c : cutset size • dc: the # of possible cutset assignment : • Time complexity in tree-structred CSP : (n-c)d2

34. Transforming general CSP to tree-structured CSP(2/4)Heuristics for complexity reduction

35. Transforming general CSP to tree-structured CSP(3/4)Heuristics for complexity reduction • Tree decomposition of constraint graph • Each sub-problem is solved independently, and the resulting solutions are combined. • Subprogram should not be too large. • Requirements for tree decomposition • Every variable in the original problem appears in at least on of the sub-problems. • If two variables are connected by a constraint in the original problem, they must appear together in at least one of the sub-problems. • If a variable appears in two sub-problems in the tree, it must appear in every sub-problem along the path connecting those sub-problems. • Divides graph to many components (sub-tree) • If components are regarded as nodes, it will be a tree. • Applies tree-structured algorithm • In arc consistency, check the common variables in the two component

36. Transforming general CSP to tree-structured CSP(4/4)Heuristics for complexity reduction • Time Complexity : O(ndw+1) • # of possible value assignment of last ordering component : dw • # of components : n/w • Time complexity for arc consistency • Max # of common values in two components : w • w * d

37. Summary • CSPs are a special kind of problem: • States defined by values of a fixed set of variables • Goal test defined by constraints on variable values • are represented by constraint graph • Backtracking: a DFS with one variable assigned per node • Variable ordering & value selection heuristics help significantly • Forward checking prevents assignments that guarantee later failure • Constraint propagation does additional work to constrain values and detect inconsistencies • The CSP representation allows analysis of problem structure • Tree-structured CSPs can be solved in linear time • Cutset conditioning can reduce a general CSP to a tree –structured on • If cutset is small, cutset conditioning can be very efficient • Tree decomposition techniques transform the CSP into a tree of subprograms • Tree width should be small for tree decomposition technique to be efficient