On the Integration of Constraint Propagation and Local Search Pedro Barahona

1. On the Integration of Constraint Propagation and Local Search Pedro Barahona Department of Computer Science Faculty of Science and Technology New University of Lisbon

2. Summary • Constraint Solvers for CSP and CSOP • Pure Methods (Propagation, Local Search, ...) • Hybrid Methods • Integration of Propagation & Local Search • Digital Circuits Testing • Handling Continuous Domains • Determination of Protein Structure • Conclusions

3. CSP and CSOP • CSP:Contraint Satisfaction Problems • Given variables V1 to Vn, find values in their respective domains D1 ... Dn that satisfy Constraints C1 to Cp. • Constraints Ci  D1 x D2 x ... x Dn • CSOP:Contraint Sat & Optimisation Problems • Given variables V1 to Vn, find values in their respective domains D1 ... Dn that satisfy Constraints C1 to Cp and, additionally, optimise some objective function F • Objective Function F: D1 x D2 x ... X Dn  R

4. Constraint Solving Methods • Algebraic Approach • Available in some specific domains • Examples: Boolean, Rational Numbers • Search Methods • General Purpose • Finite Domains, Integers, Real Numbers, Intervals, Sets, Strings, ... • Variety of Search Techniques • Constructive vs. Repairing

5. Constructive Methods • General Idea • Iterative narrowing of the possible values of the variables within their domains, until a completely characterised solution is found • Different Techniques for Narrowing • Assign Values to Variables (Finite Domains / Satisf.) • Split Current Domains (Continuous Domains / Optim.) • Key Issues • Early detection of dead ends • Efficient Backtracking

6. Repairing Methods • General Idea • Starting with a completely characterised solution, somehow unsatisfactory, search for a better solution in its neighbourhood. • Key issues • Selection of neighbour • Escaping from local optima • Search the whole solution space • Stopping criteria

7. Comparison of Methods • Constructive Methods • Complete • Inherently satisfaction methods • Optimisation considered in heuristics and deadends (BB) • More adequate for few and sparse feasible solutions • Repairing Methods • Incomplete • Adaptation of solutions from similar problems • Inherently optimisation methods • Satisfaction encoded in the optimising function • More adequate for many feasible solutions

8. 1 3 4 2 5 4 5 3 5 1 2 3 • Analyse dependencies • Backtrack in non-chronological order • “greatest of the least” Constructive Methods • Complete Search: Backtracking • Efficient Backtracking: dependency directed • Inneficiency: late detection of deadends

9. 1 1 1 2 1 2 • Propagate “Conflicts” • Avoid irrelevant backtracking 3 1 2 1 2 4 2 1 1 2 3 3 3 1 2 1 2 4 3 4 1 2 1 2 3 1 2 1 4 4 3 Constructive Methods • Complete Search: Backtracking + Propagation • Early detection of deadends - propagation • What kind of propagation?

10. Constructive Methods What kind of propagation • Trade-of: pruning vs. complexity

11. 0 1 0 0 1 1 X1 X2 X3 X4 0 0 0 0 1 1 1 1 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 Constructive Methods • Difficulty with Backtracking • Typically Chronological • Costly backtracking of early errors X

12. 0 1 X1 X2 X3 X4 0 0 1 1 0 0 0 0 1 1 1 1 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 x x 1010 1110 X2 X1 Repairing Methods • Even treatment of all variables 0010

13. Objective Function = 3 # Violations (attacks) Neighbourhood: Permutations Repairing Methods • Satisfaction via Optimization • Modeling issue: Definition of Neighbourhood

14. 2 3 1 3 0 Repairing Methods • Difficulty - Escaping from Local Optima • Restarts / Stochastic methods

15. Pure vs. Hybrid Approaches • Compete or Cooperate • Many Examples of Cooperation • Operations Research • Constraint Programming • Examples from Operations Research • Integer Programming with Linear Programming • LP provides Bounds for IP • Nonlinear and Linear Programming • Stepwise Linearization

16. Pure vs. Hybrid Approaches • Constraint Programming • Global Constraints improve Propagation • Graph Theory / OR Methods with Propagation • Integer Programming and Propagation • Cutting Planes (CPLEX and ECLiPse) • Benders Decomposition • Continuous Domains • Interval Arithmetic (Newton) • Quadratic Optimization • Propagation and Local Search

17. Experiments with CP and LS • Optimisation of Test Patterns (Digital Circuits) • Francisco Azevedo • Global-Hull Consistency (Continuous Domains) • Jorge Cruz • Determination of Protein Structure (NMR Data) • Ludwig Krippahl

18. Interaction of Constraint Programmingand Local Searchfor Optimisation Problems Francisco Azevedo & Pedro Barahona

19. Introduction • Scope: Optimisation problems in the field of Automatic Test Pattern Generation (ATPG) • Goal: For a given stuck-at fault, find a test (an input vector) with the maximum number of unspecified bits. • Approach: Integration of branch-and-bound CLP with Local Search (in a cycle CP <-> LS) with multi-valued logics • Key feature - Different encodings in CP and LS.

20. 5 valued logic to encode dependency/ignorance • Values D/ D encode dependency on a fault • Value X encodes “don’t know” situations G s-a-0 1 D (1/0) 1 G s-a-1 G s-a-0 1 0 D (0/1) 1 0 0 0 CP Modeling: 5-valued Logic • Encoding “stuck-at” faults (S-buffers)

21. X 0 0 D D D X X x CP Modeling: 5-valued Logic • Encoding normal behaviour in 5-valued Logic _ • Goal: An output D/D that maximises # X inputs

22. x x x Incompleteness of CP Modeling • Test ab=x0 detects b s-a-1 • 5 Valued Logic does not detect the fault

23. x Improving Completeness (1) • Naming unspecified values, including inversion parity, improves detection of faults

24. Multiple Sources of Unspecified Values A B Z=A.B 0 Arg 0 1 Arg Arg Arg Arg Arg id -0 id -1 0 id - p id - p id -0 A A B B Z x Improving Completeness (2) • Modelling still incomplete • final output should have value 1

25. Integration CP and LS • Constraint Propagation (weaker then full arc-consistency) finds solutions efficiently • (Still) Incomplete CP Model • Maximisation in usual Branch and Bound would still be incomplete • CP solutions as seeds for LS optimisation • LS uses a complete logic • Interesting neighbours heuristically detected with such logic

26. z z z x 0 0 1 0 0 1 y 0 1 1 (a) (b) (c) z z z 1 x 0 0 {x/ x-0, y/ y-0}:1 {}:0 {x/ x-0}:0 1 y 0 1 (b) (c) (a) Annotations in Logic for LS • Complete LS Model • Heuristic for “good” neighbours • Dependency on (single) unspecified values

27. 0/1 output does not depend on inputa , which may flip to x • Simulation Annotations in Logic for LS • Example • Local Search (over original test ab=00)

28. Evaluation of Maxx on ISCAS circuits • MTP100 and Maxx improvements over Atalanta Experimental Results (1)

29. MTP1000 and Maxx improvements over MTP100 Experimental Results (2) • Breakdown of Maxx Performance

30. Maintaining Global Hull Consistency with Local Search for Continuous CSPs Jorge Cruz & Pedro Barahona

31. Introduction • Scope: Continuous Constraint Satisfaction Problems for Decision Support • Goal: Find an enclosure of the solution space where a decisor may look for “good” solutions. • Approach: Maintaining Global Hull Consistency as the basic Constraint Propagation Method • Key Feature: Local Search in Enforcement Algorithms

32. [r1..r2] r F-interval R F [r..r] [f1 .. f2] F-box Canonical solution Under-Constrained CCSPs • Representation of Continuous Domains

33. Solving CCSPs • Branch and Prune algorithms • Prune: Constraint Propagation • Branch: Isolate Solutions

34. Basic CP Methods for CCSPs • Based on Interval Arithmetic • Hull-consistency • Decomposes constraints into sets of primitive constraints • x3+xy+y5=3  { Z1= X3 ; Z2= XY ; Z3= Y5 ; Z1+Z2+Z3 =3 } • Enforces consistency on the bounds of each domain • Box-consistency • Does not decompose constraints • Enforces consistency on domain bounds combining the Interval Newton method with spliting • F(x,y) = x3+xy+y5-3=0 Xn = xm - F(xm, Y)  Xn-1 • F’ (Xn-1,Y)

35. Higher Order Consistency • Basic Methods are often Insufficient • Higher Order Consistency Criteria • 3B- consistency, nB consistency (Hull) • if n-2 bounds are fixed then the problem is Hull-consistent • Bound-consistency, nBound consistency (Box) • if n-2 bounds are fixed then the problem is Box-consistent • However, these higher order methods may still be insufficient • Our Approach: Global Hull Consistency (GH)

36. Global Hull Consistency Each bound of the domain of each variable is justified by a canonical solution for the global set of constraints. • Decision Support Context (“many” solutions) • Requirement: search for extreme solutions

37. Hull Consistency Enforcing Algorithms • Several Algorithms were tried • OS1, OS2: Independent search for each bound of each variable • OS3: Alternate search for the bounds of all variables • TSA: Alternate search keeping some tree search structure of the domains • Common goal: search for canonical solutions • Alternatives: Pure splitting or local search

38. upper bound of x new point Newton’s vector Newton’s vector y new point minimization minimization initial point x canonical solution ? Local Search in GH Enforcing • Find a canonical solution within a box Stops when: converges find a solution

39. Constraints: F(X0)0 f(x,y)=0 F g(x,y)=0 F(X0+ X)=0 goal: X h(x,y)=0 Newton’s vector X 0 X= - J-1 F(X0) F(X0+ X)= F(X0)+J·X Jacobian Singular Value Decomposition Hull Consistency Enforcing Algorithms Newton’s Vector Current Point : X0 By the Taylor expansion (ignoring higher order terms):

40. Box-consistency x2 + y2  1 x2 + y2  2 3B-consistency Ø  Global Hull-consistency r=1.415 Box-consistency x2 + y2 + z2  2 x2 + y2 + z2  3 Global Hull-consistency 3B-consistency r=1 Ø  4B-consistency Experimental Results (1) • The need for Global Hull (toy examples):

41. 2B 3B GH x4 -1.001 .. 1.415 -0.056 .. 0.049 -0.004 .. 0.004 y4 0.585 .. 3.001 1.942 .. 2.047 1.996 .. 2.004 z4 -1.415 .. 1.416 -1.415 .. 1.415 -1.415 .. 1.415 x5 -0.415 .. 2.001 0.998 .. 1.002 0.999 .. 1.001 y5 -0.001 .. 2.001 0.999 .. 1.001 0.999 .. 1.001 z5 -1.415 .. 1.416 -1.415 .. 1.415 -1.415 .. 1.415 x6 -2.001 .. 2.001 -1.110 .. 1.053 -1.008 .. -0.992 y6 -0.001 .. 2.001 -0.894 .. 1.169 0.999 .. 1.001 z6 -2.001 .. 2.001 -1.483 .. 1.483 -1.420 .. 1.402 t (ms) 10 7 380 62 540 Experimental Results (2) • Determination of Protein Structure (6 atoms)  1 in << 7 secs

42. Time (s) Max Storage (F-boxes) kLS=10-1=10-2=10-3=10-1=10-2=10-3 2 n 2.99 11.37 600+30 47 60 OS2 y 14.97 38.46 600+14 16 24 3 n 41.01 150.21 359.6922 37 53 y 19.1475.98234.653 8 25 2 n 10.16 60.96 600+332 6786 76506 TSA y 44.79 47.59 600+221 621 16764 3 n 42.52 184.38 600+52 207 843 y 38.6397.88 275.8599 185 392 Experimental Results (2) • How useful is Local Search ?

43. PSICO Combining Constraint Programming and Optimisation to Solve Macromolecular Structures Ludwig Krippahl & Pedro Barahona

44. Introduction • Scope: Fast Interpretation of data collected in Nuclear Magnetic Resonance (NMR) experiments. • Goal: Determine Protein Structure from inter-atomic distances provided by NMR data. • Approach: Combining CP with local search optimisation (in continuous domains) • Key Feature: CP and Local Search acting on different models

45. Genes Structure Motion Regulation Function Catalysis Signaling Motivation (1) • Protein (amino acid) sequence, coded in the genes, determines its structure (shape) which, in turn, determines much of its functionality

46. What we want Glycin Leucin Tyrosine Basic Knowledge What NMR provides C N O H W Y L L H W S T Motivation (2) • NMR avoids crystalisation and other costly procedures, but requires fast data post-processing (to correct interpretation errors)

47. Phase 1: Establish Distance Constraints • From Basic Knowledge • From NMR Data In:| Xi - Xj |  UijOut:| Xi - Xj |  Lij Overall Method • Phase 2: Approximate Solution • Constraint Propagation • Phase 3: Repair Solution • Local Search

48. Domains modeled as 3-D boxes NoGood y x2 , y2 In Out Good Distance x1 , y1 Modeling allows “arc-consistency” stronger than “bounds-consistency” x a b c Phase 1: Domain Modeling

49. Out Constraint In Constraint Exclusion Zone Good d Good d Max d Good Min NoGood NoGood Intersection Exclusion Zone Phase 2: Constraint Propagation

50. Phase 3: Local Search (Optimisation) • Change representation to torsion angles (i.e the chemical bond allow some rotation) • Fit approximate solution from CP to the torsion angle model • Optimise overall constraint violation (Conjugate Gradient Method)