Solve Your Problem Faster - by changing the model

1. Solve Your Problem Faster - by changing the model Barbara Smith

2. Background Assumptions • A constraint programming tool providing: • a systematic search algorithm • combined with constraint propagation • a set of pre-defined constraints • A problem that can be represented as a finite domain constraint satisfaction or optimization problem • Caveat: the experiences described are based on ILOG Solver ERCIM/CologNet Workshop

3. What do I mean by Faster? • e.g.`Peaceable Armies of Queens’ problem • place m black & m white queens on a chessboard so that the black queens don’t attack the white queens, and maximize m • Optimal solution for an 11 x 11 board: ERCIM/CologNet Workshop

4. Results for 8 x 8 board Just changing the model makes a huge difference to the time to solve the problem ERCIM/CologNet Workshop

5. What are the options? • Given a CSP representation of the problem: • if there is symmetry in the CSP, eliminate it • find a related representation to use instead/as well • add variables – to express different aspects of the problem • add constraints • to relate new variables to old • to prune dead-ends earlier • change the search strategy • (other possibilities won’t be considered) ERCIM/CologNet Workshop

6. Symmetry in the CSP • A symmetry transforms any (full or partial) assignment into another so that consistency/inconsistency is preserved • Symmetry causes wasted search effort: after exploring choices that don’t lead to a solution, symmetrically equivalent choices will be explored • Especially difficult if there is no solution, or if all solutions are wanted, or in optimization problems ERCIM/CologNet Workshop

7. A = set of assignments made so far var = val var != val Symmetry Breaking During Search (SBDS) • See Gent & Smith, ECAI’2000 + g(var!= val) for any unbroken symmetry g, i.e. if g(A) is (or will be) true On backtracking to a choice point, add symmetry breaking constraints to the 2nd branch explored ERCIM/CologNet Workshop

8. Symmetries of ‘armies of queens’ • Variable s[i,j] represents the square i, j • The values are b, w or e (black, white or empty) • 7 chess board symmetries: x (horizontal reflection),y, d1, d2, r90, r180, r270 + bw(swap black & white queens) + 7 combinations • x(s[i,j]=v)  s[n-i+1,j]=v • bw(s[i,j] = v)  s[i,j] = v’ , (v’ = b if v = w, w if v = b, e if v = e ) • bw_x(s[i,j]=v)  s[n-i+1,j]=v’ , etc. • 15 symmetry functions are needed in all • each takes a constraint as input, e.g. s[i,j]=v, and returns the symmetric constraint, e.g. s[i,j] = v’ ERCIM/CologNet Workshop

9. s[1,1] != w x: s[8,1]!=w s[1,1]=w d1: s[1,1]!=w r90: s[1,8]!=w bw: s[1,1]!=b bw_x: s[8,1]!=b, etc. s[1,2]=w All symmetries swapping colours now broken on this branch s[1,2] != w Armies of Queens – 8 x 8 Board x: if s[8,1]=w then s[8,2]!=w , etc. ERCIM/CologNet Workshop

10. Results for 8 x 8 board ERCIM/CologNet Workshop

11. Optimizing SONET Rings • See Sherali & Smith*, ‘Improving Discrete Model Representations via Symmetry Considerations’ (*no relation) • Transmission over optical fibre networks • Known traffic demands between pairs of client nodes • A node is installed on a SONET ring using an ADM (add-drop multiplexer) • If there is traffic demand between 2 nodes, there must be a ring that they are both on • Rings have capacity limits (number of ADMs, i.e. nodes, & traffic) • Satisfy demands using the minimum number of ADMs ERCIM/CologNet Workshop

12. Example & Optimal Solution • Up to 7 rings, maximum capacity 5 ADMs • Ignore traffic capacity (for now) ERCIM/CologNet Workshop

13. A CSP Model • Variables: xij = 1 if node i is on ring j • At most 5 nodes on any ring: • If there is a demand between nodes k and l : • Minimize ERCIM/CologNet Workshop

14. Symmetry in the SONET CSP • The rings are indistinguishable (only numbered for the purposes of the CSP model) • We can eliminate the symmetry using SBDS just by describing all transpositions of pairs of rings: • e.g. r12(x[i,j] = v)  x[i,2]=v if j =1 x[i,1]=v if j =2 x[i,j]=v otherwise • That’s all – we can forget about symmetry (e.g. when choosing the variable ordering) ERCIM/CologNet Workshop

15. Three Alternative Models • Whether a given node is on a given ring: • xij = 1 if node i is on ring j • Which ring(s) each node is on: • Ni = set of rings node i is on • Which nodes are on each ring • Rj= set of nodes on ring j • In principle, any of these 3 sets of variables could be the basis of a complete CSP model ERCIM/CologNet Workshop

16. Dual Models from Boolean Variables • Given a CSP with Boolean variables xijkl…we can form anew set of variables with one less subscript: • e.g. xijkl…= 1 corresponds toyjkl… = i (an integer variable) or i  Yjkl… (a set variable), depending on whether one or several possible values i are associated with each combination of j,k,l,… • if the Boolean variables have n subscripts, we can derive n sets of dual variables ERCIM/CologNet Workshop

17. Which Model to Choose? • We don’t need to choose just one set of variables – we can use them all at once • We then need new channelling constraints to link the sets of variables: • (xij = 1) = (i  Rj) = (j Ni) • (see Cheng, Choi, Lee & Wu, Constraints, 1999) • But we should not combine all 3 complete models • Adding variables & channelling constraints doesn’t cost much – duplicated constraints do ERCIM/CologNet Workshop

18. Why add more variables? • Express each problem constraint in whichever way is easiest /most natural/ propagates best • e.g. gives better results than • (Often easiest /most natural/ propagates best are the same thing) • We can observe effects of search on different aspects of the model, & express them • develop implied constraints, search strategies, etc. ERCIM/CologNet Workshop

19. Possible Variables in the SONET Problem • Whether a given node is on a given ring  • Which ring(s) each node is on  • Which nodes are on each ring  • How many nodes are on each ring  • How many rings each node is on  • The total number of nodes on all rings  • The total of the number of rings each node is on  • Which demand pairs are on each ring • Which ring(s) each demand pair is on • How many rings are used • ……… ERCIM/CologNet Workshop

20. Choosing the Search Variables • We need to choose a set of variables such that an assignment to each one, satisfying the constraints, is a complete solution to the problem • Assume we pass the search variables to the search algorithm in a list or array • the order determines a static variable ordering ERCIM/CologNet Workshop

21. Possible Choices • Use just one set of variables, e.g. xij – the others are just for constraint propagation • Use two (or more) sets of variables (of the same type) e.g. Rj ,Ni • interleave them in a sensible (static) order • or use a dynamic ordering applied to both sets of variables • Use an incomplete set of variables first, to reduce the search space before assigning a complete set • e.g. decide how many rings each node is on (search variables |Ni|) and then which rings each node is on (xij) • All three possibilities are useful – the first will be used in the SONET problem (and later the third) ERCIM/CologNet Workshop

22. Implied Constraints • Constraints which can be derived from the existing constraints, and so don’t eliminate any solutions • We only want useful implied constraints: • they reduce search: i.e. at some point during search, a partial assignment must be tried which is inconsistent with the implied constraint but would otherwise not fail immediately • they reduce running time – the overhead of propagating extra constraints must be less than the savings in search ERCIM/CologNet Workshop

23. How to Find Useful Implied Constraints • Identify obviously wrong partial assignments that may/do occur during search • Try to predict them by contemplation/intuition • Observe the search in progress • Having many variables in the model enables observing/thinking about many possible aspects of the search • But we still need to check empirically that new constraints do reduce both search and running time ERCIM/CologNet Workshop

24. Implied Constraints: SONET • A node with degree in the demand graph > 4 must be on more than 1 ring (|Ni| > 1) • If a pair of connected nodes have more than 3 neighbours in total, at least one of the pair must be on more than 1 ring (|Nk|+|Nl| > 2) ERCIM/CologNet Workshop

25. Optimality Constraints • In optimizing, if we know that for any solution with a particular characteristic, there must be another solution at least as good, we can add constraints forbidding it • e.g. • no ring should have just one node on it • any two rings must have more than 5 nodes in total (otherwise we could merge them) • Derive these in the same way as implied constraints ERCIM/CologNet Workshop

26. Variable Ordering Heuristics • Armies of Queens: • Place a white queen next where it will attack fewest additional squares • SONET problem: • add a node to a ring with spare capacity; choose the node connected to most nodes already on the ring; of those, the node connected to most nodes still to be placed ERCIM/CologNet Workshop

27. Finding a Good Solution v. Proving Optimality • Armies of Queens • the heuristic finds an optimal solution for every case where this is known • but… it’s hopeless for proving optimality • the `anti-heuristic’ (place a white queen where it will attack most additional squares) is much better! ERCIM/CologNet Workshop

28. Finding a Good Solution v. Proving Optimality: SONET • The heuristic finds near-optimal solutions quite quickly, but is no good for proving optimality • How could we prove this solution is optimal: {2,3,4,9,12}, {1,3,7,8,10}, {4,5,6,7,10}, {1,8,11,12,13} ? • e.g. show that we cannot reduce the number of times that any of 3, 4, 7, 8, 12 appear without having another node appear twice instead • Introduce variables ni = |Ni| i.e. the number of rings that node i is on •  Two-stage search: find a good solution, then start search again, assigning ni variables first and then xij variables ERCIM/CologNet Workshop

29. What do I mean by Faster? • The basic model (just the xij variables, no symmetry-breaking) can only solve small problems (7 nodes, 8 demand pairs) • The final model can solve the problems in the Sherali & Smith paper (13 nodes, 24 demand pairs) • 3 full sets of variables + others • SBDS • implied constraints • variable ordering heuristic • two-stage search process ERCIM/CologNet Workshop

30. Some Advice • Eliminate symmetry • Use lots of variables, with channelling constraints • But don’t express the same problem constraints twice • Add constraints that make explicit what you know about satisfying/optimal solutions • But only if they reduce search and running time • Learn from solving the problem by hand and observing the search • If finding good solutions is easy and proving optimality is hard, consider using a different strategies for each stage ERCIM/CologNet Workshop

31. Conclusions • Can we list “10 Steps to Successful Modelling”? • New problems still often lead to new ideas about modelling • But some patterns do recur frequently • e.g. models representing dual viewpoints • We are beginning to automate some aspects of modelling • e.g. symmetry, implied constraints • Still a long way to go before building a good model of a problem is straightforward • e.g. we often can’t tell if model A is better than model B without trying them both • More research is still needed... ERCIM/CologNet Workshop

32. Selected References • Symmetry Breaking During Search in Constraint Programming, I. P. Gent and B. M. Smith, Proceedings ECAI'2000, pp. 599-603, 2000. • Models and Symmetry Breaking for `Peaceable Armies of Queens’,K.E Petrie, B. M. Smith & I. P. Gent, APES Report APES-50-2002, May 2002. • Improving Discrete Model Representations via Symmetry Considerations, H. D. Sherali & J. C. Smith, Man.Sci. (47) pp. 1396-1407, 2001. • Increasing Constraint Propagation by Redundant Modeling: an Experience Report, B. M. W. Cheng, K. M. F. Choi, J. H. M. Lee & J. C. K. Wu, Constraints (4) pp. 167-192, 1999.   ERCIM/CologNet Workshop