Foundations of Constraint Processing CSCE421/821, Spring 2008:

1. Constraint Satisfaction 101 Foundations of Constraint Processing CSCE421/821, Spring 2008: www.cse.unl.edu/~choueiry/S08-421-821/ Berthe Y. Choueiry (Shu-we-ri) AVH 123B choueiry@cse.unl.edu Tel: +1(402)472-5444 Overview 1

2. Outline • Motivating example, application areas • CSP: Definition, representation • Some simple modeling examples • More on definition and formal characterization • Basic solving techniques • (Implementing backtrack search) • Advanced solving techniques • Issues & research directions Overview 1

3. Motivating example • Context: You are a senior in college • Problem: You need to register in 4 courses for the Spring semester • Possibilities: Many courses offered in Math, CSE, EE, CBA, etc. • Constraints: restrict the choices you can make • Unary: Courses have prerequisites you have/don't have Courses/instructors you like/dislike • Binary: Courses are scheduled at the same time • n-ary: In CE: 4 courses from 5 tracks such as at least 3 tracks are covered • You have choices, but are restricted by constraints • Make the right decisions!! Overview 1

4. Motivating example (cont’d) • Given • A set of variables: 4 courses at UNL • For each variable, a set of choices (values) • A set of constraints that restrict the combinations of values the variables can take at the same time • Questions • Does a solution exist? (classical decision problem) • How two or more solutions differ? How to change specific choices without perturbing the solution? • If there is no solution, what are the sources of conflicts? Which constraints should be retracted? • etc. Overview 1

5. Practical applications Adapted from E.C. Freuder • Radio resource management (RRM) • Databases (computing joins, view updates) • Temporal and spatial reasoning • Planning, scheduling, resource allocation • Design and configuration • Graphics, visualization, interfaces • Hardware verification and software engineering • HC Interaction and decision support • Molecular biology • Robotics, machine vision and computational linguistics • Transportation • Qualitative and diagnostic reasoning Overview 1

6. Constraint Processing • is about ... • Solving a decision problem… • … While allowing the user to state arbitrary constraints in an expressive way and • Providing concise and high-level feedback about alternatives and conflicts • Related areas: • AI, OR, Algorithmic, DB, TCS, Prog. Languages, etc. Overview 1

7. relax reinforce Power of Constraints Processing • Flexibility & expressiveness of representations • Interactivity • users can constraints Overview 1

8. Outline • Motivating example, application areas • CSP: Definition, representation • Some simple modeling examples • More on definition and formal characterization • Basic solving techniques Overview 1

9. Defining a problem • General template of any computational problem • Given: • Example: a set of objects, their relations, etc. • Query/Question: • Example: Find x such that the condition y is satisfied • How about the Constraint Satisfaction Problem? Overview 1

10. Definition of a CSP • GivenP = (V, D, C ) • V is a set of variables, • D is a set of variable domains (domain values) • C is a set of constraints, • Query: can we find a value for each variable such that all constraints are satisfied? Overview 1

11. Other queries… • Find a solution • Find all solutions • Find the number of solutions • Find a set of constraints that can be removed so that a solution exists • Etc. Overview 1

12. Representation I • Given P = (V, D, C ), where • Find a consistent assignment for variables • Constraint Graph • Variable   node (vertex) • Domain  node label • Constraint  arc (edge) between nodes Overview 1

13. v1 < v2 V1 {1, 2, 3, 4} { 3, 6, 7 } V2 v1+v3 < 9 v2 > v4 v2 < v3 { 3, 4, 9 } { 3, 5, 7 } V3 V4 Representation II • Given P = (V, D, C ), where • Example I: • Define C ? Overview 1

14. Outline • Motivating example, application areas • CSP: Definition, representation • Some simple modeling examples • More on definition and formal characterization • Basic solving techniques Overview 1

15. Example II: Temporal reasoning • Give one solution: ……. • Satisfaction, yes/no: decision problem Overview 1

16. Example III: Map coloring Using 3 colors (R, G, & B), color the US map such that no two adjacent states have the same color • Variables? • Domains? • Constraints? Overview 1

17. Example III: Map coloring (cont’d) Using 3 colors (R, G, & B), color the US map such that no two adjacent states have the same color Overview 1

18. { a, b, c } { a, b } { a, c, d } { b, c, d } Example IV: Resource Allocation What is the CSP formulation? Overview 1

19. { a, b, c } { a, b } { a, c, d } { b, c, d } Constraint Graph T1 { R1, R3 } T4   T6 { R1, R2, R3 }   { R2, R4 }       { R1, R3 }  { R2, R4 }   { R1, R2, R3 } T7 T3 { R1, R3 } T5 Example IV: RA (cont’d) Interval Order { R1, R3 } T1 { R1, R3 } T2 { R1, R3, R4 } { R1, R3 } T2 T4 T3 { R1, R2, R3 } T5 { R2, R4 } T6 { R2, R4 } T7 Overview 1

20. Example V: Puzzle • Given: • Four musicians: Fred, Ike, Mike, and Sal, play bass, drums, guitar and keyboard, not necessarily in that order. • They have 4 successful songs, ‘Blue Sky,’ ‘Happy Song,’ ‘Gentle Rhythm,’ and ‘Nice Melody.’ • Ike and Mike are, in one order or the other, the composer of ‘Nice Melody’ and the keyboardist. • etc ... • Query: Who plays which instrument and who composed which song? Overview 1

21. Example V: Puzzle (cont’d) • Formulation 1: • Variables: Bass, Drums, Guitar, Keyboard, Blue Sky, Happy Song • Gentle Rhythm and Nice Melody. • Domains: Fred, Ike, Mike, Sal • Constraints: … • Formulation 2: • Variables: Fred's-instrument, Ike's-instrument, …, • Fred's-song, Ikes's-song, Mike’s-song, …, etc. • Domains: • { bass, drums, guitar, keyboard } • { Blue Sky, Happy Song, Gentle Rhythm, Nice Melody} • Constraints: … Overview 1

22. Example VI: Product Configuration • Train, elevator, car, etc. • Given: • Components and their attributes (variables) • Domain covered by each characteristic (values) • Relations among the components (constraints) • A set of required functionalities (more constraints) • Find: a product configuration • i.e., an acceptable combination of components • that realizes the required functionalities Overview 1

23. Example VII: Cryptarithmetic puzzles • DX1 = DX2 = DX3 = {0,1} • DF=DT=DU=DW=DR=DO=[0,9] • O+O = R+10X1 • X1+W+W = U+10X2 • X2+ T+T = O + 10X3 • X3=F • Alldiff({F,D,U,V,R,O}) Overview 1

24. v1 < v2 Constraint Graph V1 {1, 2, 3, 4} { 3, 6, 7 } T1 V2 { R1, R3 } T4   T6 { R1, R2, R3 } v1+v3 < 9   v2 > v4 v2 < v3 { R2, R4 }       { R1, R3 } { 3, 4, 9 } { 3, 5, 7 } V3 V4  { R2, R4 }   { R1, R2, R3 } T7 T3 { R1, R3 } T5 Constraint types: examples • Example I: algebraic constraints • Example II: (algebraic) constraints of bounded difference • Example III & IV: coloring, mutual exclusion, difference constraints • Example V & VI: elements of C must be made explicit Overview 1

25. More examples • Example VII:Databases • Join operation in relational DB is a CSP • View materialization is a CSP • Example VIII: Interactive systems • Data-flow constraints • Spreadsheets • Graphical layout systems and animation • Graphical user interfaces • Example IX: Molecular biology (bioinformatics) • Threading, etc Overview 1

26. Outline • Motivating example, application areas • CSP: Definition, representation • Some simple modeling examples • More on definition and formal characterization • Basic solving techniques Overview 1

27. V1 a, b   V3 V2 a, c b, c  Representation (again) Macrostructure G(P): - constraint graph for binary constraints - constraint network for non-binary constraints Micro-structure (P): Co-microstructure co-(P): (V1, a ) (V1, b) (V2, a ) (V2, c) (V3, b ) (V3, c) no goods (V1, a ) (V1, b) (V2, a ) (V2, c) (V3, b ) (V3, c) Overview 1

28. v1 < v2 V1 {1, 2, 3, 4} { 3, 6, 7 } V2 v1+v3 < 9 v2 > v4 v2 < v3 { 3, 4, 9 } { 3, 5, 7 } V3 V4 Constraint arity I • Given P = (V, D, C ) , where • How to represent the constraint V1 + V2 + V4 < 10? Overview 1

29. Constraint arity II • Given P = (V, D, C ) , where Constraints: universal, unary, binary, ternary, … , global. A Constraint Network: Overview 1

30. Domain types • P = (V, D, C ) where • Domains: • Restricted to {0,1}: Boolean CSPs • Finite (discrete), enumeration works • Continuous, sophisticated algebraic techniques are needed • Consistency techniques on domain bounds Overview 1

31. Constraint terminology • Arity: • universal, unary, binary, ternary, …, global • Scope: • The set of variables to which the constraint applies • Definition: • Extension: all allowed tuples are listed • Constrained Decision Diagrams ([Cheng & Yap, AAAI 05]) • Intention: when it is not practical (or even possible) to list all tuples • Define types/templates of common constraints to be used repeatedly: linear constraints, All-Diff (mutex), Atmost, TSP-constraint, cycle-constraint, etc.) • Implementation: • predicate, set of tuples (list or table), binary matrix (bit-matrix), etc. Overview 1

32. Complexity of CSP Characterization: • Decision problem • In general, NP-complete by reduction from 3SAT Overview 1

33. Proving NP-completeness • Show that 1 is in NP • Given a problem 1 in NP, show that an known NP-complete problem 2 can be efficiently reduced to 1 • Select a known NP-complete problem 2 (e.g., SAT) • Construct a transformationf from 2 to 1 • Prove that f is a polynomial transformation (Check Chapter 3 of Garey & Johnson) Overview 1

34. What is SAT? Given a sentence: • Sentence: conjunction of clauses • Clause: disjunction of literals • Literal: a term or its negation • Term: Boolean variable Question: Find an assignment of truth values to the Boolean variables such the sentence is satisfied. Overview 1

35. CSP is NP-Complete • Verifying that an assignment for all variables is a solution • Provided constraints can be checked in polynomial time • Reduction from 3SAT to CSP • Many such reductions exist in the literature (perhaps 7 of them) Overview 1

36. Problem reduction Example: CSP into SAT (proves nothing, just an exercise) Notation: variable-value pair = vvp • vvp  term • V1 = {a, b, c, d} yields x1 = (V1, a), x2 = (V1, b), x3 = (V1, c), x4 = (V1, d), • V2 = {a, b, c} yields x5 = (V2, a), x6 = (V2, b), x7 = (V2,c). • The vvp’s of a variable  disjunction of terms • V1 = {a, b, c, d} yields • (Optional) At most one VVP per variable Overview 1

37. CSP into SAT (cont.) Constraint: • Way 1: Each inconsistent tuple  one disjunctive clause • For example: how many? • Way 2: • Consistent tuple  conjunction of terms • Each constraint  disjunction of these conjunctions  transform into conjunctive normal form (CNF) Question: find a truth assignment of the Boolean variables such that the sentence is satisfied Overview 1

38. Outline • Motivating example, application areas • CSP: Definition, representation • Some simple modeling examples • More on definition and formal characterization • Basic solving techniques • Modeling and consistency checking • Constructive, systematic search • Iterative improvement, local search Overview 1

39. How to solve a CSP? Search • 1. Constructive, systematic • 2. Iterative repair, local search Overview 1

40. Before starting search! Consider: • Importance of modeling/formulation: • To control the size of the search space • Preprocessing • A.k.a. constraint filtering/propagation, consistency checking • reduces size of search space Overview 1

41. 1 2 3 4 V1 V2 V3 V4 4 Rows 4 Column positions V11 V12 V13 V14 16 Cells V21 V22 V23 V24 V31 V32 V33 V34 V41 V42 V43 V44 Importance of modeling • N-queen: formulation 1 • Variables? • Domains? • Size of CSP? • N-queens: formulation 2 • Variables? • Domains? • Size of CSP? {0,1} Overview 1

42. 13 1- B: [ 5 .. 14 ] 14 C: [ 6 .. 15 ] 2- A: [ 2 .. 10 ] 2 C: [ 6 .. 14 ] 14 6 Constraint checking • Arc-consistency B A < B A [ 5.... 18] B < C [ 1.... 10 ] 2 < C - A < 5 3- B: [ 5 .. 13 ] [ 4.... 15] C Overview 1

43. Constraint checking • Arc-consistency: every combination of two adjacent variables • 3-consistency, k-consistency (k n) • Constraint filtering, constraint checking, etc.. • Eliminate non-acceptable tuples prior to search • Warning: arc-consistency does not solve the problem still is not a solution! Overview 1

44. Systematic search • Starting from a root node • Consider all values for a variable V1 • For every value for V1, consider all values for V2 • etc.. • For n variables, each of domain size d • Maximum depth? fixed! • Maximum number of paths? size of search space, size of CSP Overview 1

45. Systematic search (I)Back-checking • Systematic search generates dnpossibilities • Are all possibilities acceptable? • Expand a partial solution only when it is consistent • This yields early pruning of inconsistent paths Overview 1

46. Systematic search (II)Chronological backtracking What if only one solution is needed? • Depth-first search & Chronological backtracking • DFS: Soundness? Completeness? Overview 1

47. Systematic search (III)Intelligent backtracking What if the reason for failure was higher up in the tree? Backtrack to source of conflict !! • Backjumping, conflict-directed backjumping, etc. Overview 1

48. Systematic search (IV)Ordering heuristics • Which variable to expand first? • Heuristics: • most constrained variable first (reduce branching factor) • most promising value first (find quickly first solution) Overview 1

49. 3 1 2 2 2 4 1 3 4 1 3 1 4 1 2 4 4 1 1 2 3 3 2 2 Q Q Q Q Q Q Q 1 3 Solution! 26 nodes visited. Systematic search (V)Back-checking • Search tree with only backtrack search? Root node Q Overview 1

50. 2 1 1 3 4 3 4 2 Q Q Q Q Q Domain Wipe Out V3 Q Q Domain Wipe Out V4 Q Solution! 8 nodes visited. Systematic search (VI)Forward checking Search Tree with domains filter by Forward Check Root node Overview 1