Principles of Intelligent Systems Constraint Satisfaction + Local Search

1. Principles of Intelligent SystemsConstraint Satisfaction + Local Search Written by Dr John ThorntonSchool of IT, Griffith University Gold Coast

2. Outline • The Constraint Satisfaction Paradigm • Variables, Domains and Constraints • Example Problem Domains • Constraint Satisfaction Algorithms • Local Search Algorithms • Summary

3. The Constraint Satisfaction Paradigm • Knowledge represented by constraints • What is allowed or disallowed. • Transforms broad range of problems into a computable form • Standard representation means general purpose algorithms can be used • solvers ‘know’ how to solve problems • issue becomes how to model a problem

4. Variables, Domains and Constraints • A Constraint Satisfaction Problem (CSP) is: • a set of variables each with • a set of domain of values, and • a set of constraints that define which combinations of variable values are allowed • the task is to find a domain value for each variable such that all constraints are simultaneously satisfied

5. Problem Domains • N-Queens • Satisfiability Problems • Binary Constraint Satisfaction • Nurse Rostering • University Timetabling

6. Local Search Algorithms • Getting ‘stuck’ • Variable and Constraint-based Search • Random Walk • Tabu Search • NOVELTY and RNOVELTY • Constraint Weighting

7. Satisfiability Problems • Conjunctive Normal Form: Example {x Ú y Ú z} Ù {x Ú yÚz} Ù {xÚy} Ù {yÚz} • To Solve: • All clauses evaluate true • e.g.: x = false, y = true, z = true • As a CSP: • Each clause is a constraint, each literal is a variable with a {true, false} domain

8. Predicate to Conjunctive Form • Predicate: (X) (dog (X)  animal (X)) • Clause: ( dog(X) Ú animal(X)) • Clause is false only if  dog(X) is false and animal(X) is false, i.e. if X is a dog and X is not an animal - in other words if X is a dog then X is an animal

9. Binary Constraint Satisfaction • Two variable constraint representation • Non-binary to binary transformation • Phase transition: p2crit = 1 - m-2/p1(n - 1) where p1 = constraint density p2 = constraint tightness m = uniform domain size n = number of variables

10. Nurse Scheduling

11. Binary Constraint Representation Variable 1 0 1 2 3 4 0 1 Variable 2 2 3

12. Random Walk Local Search • if randomly generated probability < p: • Take best cost local move • else: • Take randomly selected local move

13. Tabu Search • if cost(best local move) < best cost yet: • Take best local move • else • Take best cost local move from the set of domain values that have not been used in the last n moves

14. The Eight Queens Problem Domain Variable Constraint

15. Constraint Satisfaction Algorithms • Backtracking • Algorithm • 8-Queens Example • 8-Queens Performance • Local Search • Algorithm • 8-Queens Example • 8-Queens Performance

16. Summary • Constraint Satisfaction • practical knowledge representation • allows general purpose approaches • Algorithms • Backtracking • Local Search

17. Backtracking Algorithm V= list of all variables vi (i = 1 to n) U = empty TryVariable() select next vi from Vand move to U for each domain value di of vi ifdi satisfies all constraints with U Value(vi) =di if(Vis empty) SaveSolution() else TryVariable() move vi from U backto V

18. Basic Local Search Algorithm assign a domain value di to each variable vi while no solution and not stuck and not timed out bestCost ; bestList ; for each variable vi | Cost(Value(vi)) > 0 for each domain value di of vi if Cost(di) < bestCost bestCost Cost(di); bestList di; else if Cost(di) = bestCost bestList bestList  di Take a randomly selected move from bestList

19. Backtracking Performance

20. Local Search Performance

21. Local Search Diagram

22. Try Queen 1 Backtrack and undo last move for Queen 4 Try next value for Queen 5 Still Stuck Undo move for Queen 5 no move left Try next value for Queen 4 Undo move for Queen 7 and so on... Try Queen 2 Try Queen 3 Try Queen 4 Try Queen 5 Stuck! Undo move for Queen 5 Try Queen 5 Try Queen 6 Try Queen 7 Stuck Again Eight Queens using Backtracking

23. Place 8 Queens randomly on the board 2 2 3 1 1 1 3 0 3 1 2 2 3 3 4 2 1 2 2 2 2 3 1 2 1 Answer Found Pick a Queen: Calculate cost of each move Take least cost move then try another Queen 1 2 1 2 1 3 2 3 1 4 1 5 1 0 1 1 1 3 1 4 1 0 1 4 1 4 1 4 1 2 1 3 1 3 1 1 4 2 3 1 3 1 3 2 4 2 3 2 1 2 2 2 2 3 2 2 2 3 2 2 1 4 3 1 3 1 1 2 2 2 1 1 2 1 2 3 3 2 1 3 3 3 3 3 2 4 3 0 3 2 Eight Queens using Local Search

24. Variable/Constraint-based Search • Variable-based search: • Randomly select variable in constraint violation • Try all domain values for that variable • Constraint-based search: • Randomly select a violated constraint • Try all domain values for all variables in the constraint that improve the constraint

25. NOVELTY • select best and second best cost moves that improve a violated constraint • if best cost move not most recent or p >= random value • select best cost move • else • select second best cost move

26. RNOVELTY • Same as NOVELTY except when best cost move is also most recent, then : • if second best cost - best cost <= c or p < random value • select second best cost move • else • select best cost move

27. Constraint Weighting • Add weight to all constraints violated at a local minimum: graph colouring example • Use these weights to calculate cost of subsequent moves • Adding weights changes shape of cost surface: “fills in” the minimum

28. b red b green b green cbd = 0 cbd = 0 cbd = w cab = 0 cab = w cab = 2w cbc = w cbc = 0 cbc = 0 a green a green a green d green d red d green ccd = 0 ccd = 0 ccd = w cac = 0 cac = 0 cac = 0 c red c red c red Constraint Weighting Example • Graph Colouring (a) Local minimum, cost 2w (b) After Weighting, cost 3w (c) Final solution, cost w