Improving Coalition Performance by Exploiting Phase Transition Behavior

1. 1 David W. Etherington Andrew J. Parkes CIRL, University of Oregon Improving Coalition Performance by Exploiting Phase Transition Behavior

2. 2 Administrative

3. 3 Subcontractors and Collaborators Subcontractors: None Collaborators: SNAP groups at ISI. Goal: develop an understanding of negotiation methods. This was achieved. Work on pseudo-Boolean encoding relevant to their phase transition studies. Plan to be able to integrate our work on robust solutions into SNAP/MAPLANT interactions. SRI group. Goal: understand and improve computational complexity of SRI negotiation methods. Have initial understanding of their handling of new targets. Plan to be able to utilize our previous results on robust interaction within their framework.

4. 4 Problem Description/Objective Develop lightweight, robust mechanisms, not subject to computational cliffs, to facilitate coordination of autonomous teams Challenges: strict real-time constraints stringent communication and coordination restrictions scaling Approaches: static: implement architectures guaranteed not to raise difficult problems dynamic: ensure that hard problems can be detected and made manageable on the fly Usual general framework. Will skip straight to technical.Usual general framework. Will skip straight to technical.

5. 5 Problem Description: Objective Model peaks and cliffs in computational/communication cost and develop mechanisms to help ANT systems avoid them theoretical and experimental results Develop infrastructure and tools to: detect infeasibility transitions by monitoring derived constraints & phase transition info relax constraints to avoid infeasibility develop resource-bounded distributed algorithms aggregate search information to guide ANT coalitions

6. Improving Coalition Performance by Exploiting Phase Transition Behavior

7. 7 Project Status Results: focused learning in common sublanguages for ANT negotiations showed existence of pre-phase transition cliffs use of robust partial solutions to reduce coupling in negotiation phase transitions indicate achievable robustness sequential search and derived constraints efficient parallel/distributed search Collaboration: preliminary analysis of SRI negotiation strategy possible application of robust partial solutions results on ISI�s approach, suggesting architecture changes cliff precursors in SNAP

8. 8 Project Status: Publications Scaling Properties of Pure Random Walk on Random 3-SAT. Andrew J. Parkes. Proceedings of the Eighth International Conference on Principles and Practice of Constraint Programming (CP2002). Published in Lecture Notes in Computer Science, LNCS 2470. Pages 708--713. Easy Predictions for the Easy-Hard-Easy Transition. Andrew J. Parkes. Eighteenth National Conference on Artificial Intelligence (AAAI-02) Likely Near-Term Advances in SAT Solvers. Heidi E. Dixon, Matthew L. Ginsberg, Andrew J. Parkes, at MTV-02. Inference methods for a pseudo-Boolean satisfiability solver. Heidi E. Dixon and Matthew L. Ginsberg. AAAI-02. Exploiting Solution Clusters for Coarse-grained Distributed Search Andrew J. Parkes. Proc. Distributed Constraint Reasoning, at the International Joint Conference on Artificial Intelligence (IJCAI-01) Distributed Local Search, Phase Transitions, and Polylog Time Andrew J. Parkes. Proc. Stochastic Search Algorithms, at IJCAI-01.

9. 9 Project Status The thrashing problem The need for higher-level frameworks than SAT/CSP e.g. pseudo-boolean (PB) representations Explanation-driven search PARIS coarse-grained distributed approaches Focus on part of work that has not been covered much in previous PI meetings, but that we believe raises critical issues, and is coming to fruition Issue today is some forms of thrashing and their avoidance, and associated representational and reasoning issues PB, and forms of search relevant locally -- system -- and globally -- system-of-systemsFocus on part of work that has not been covered much in previous PI meetings, but that we believe raises critical issues, and is coming to fruition Issue today is some forms of thrashing and their avoidance, and associated representational and reasoning issues PB, and forms of search relevant locally -- system -- and globally -- system-of-systems

10. 10 Thrashing �Internal� thrashing: ANT coalitions must recognize resource problems can thrash because further improvement is impossible but not recognized, and so do continual, but doomed, reallocation fix with sufficiently powerful internal reasoning �External� Thrashing: want effective coarse-grained distributed search (system-of-systems) can thrash by exchanging only point solutions fix by exchanging solution clusters, and explanations Have focussed on two forms of thrashing. First -- internal to a single system - thrash because can�t do better, but don�t know it Second, and later in talk is system-of-systems issues, we talked about this in previous meetings and papers, but will return to itHave focussed on two forms of thrashing. First -- internal to a single system - thrash because can�t do better, but don�t know it Second, and later in talk is system-of-systems issues, we talked about this in previous meetings and papers, but will return to it

11. 11 Failure to count causes thrashing Need to detect resource over-allocation requires counting infeasible in SAT/CSP representations Motivating Example: Pigeon-hole problems (PHPs) simplest form of resource contention can you put n pigeons in n-1 holes without sharing? shortest resolution proof is exponential BUT reasoning in SAT/CSP relies on resolution this is a form of thrashing because one do a lot better: Particularly irritating form of thrashing arises from inability to count, common issue with resource constraints. You can�t count easily in SAT, or CSP Simple motivating example, that probably often occurs in some hidden or embedded form -- PHP -- can you put n pigeons in n-1 holes, counting says obviously not. But standard SAT/CSP fail to recognize this. One can do a lot better Particularly irritating form of thrashing arises from inability to count, common issue with resource constraints. You can�t count easily in SAT, or CSP Simple motivating example, that probably often occurs in some hidden or embedded form -- PHP -- can you put n pigeons in n-1 holes, counting says obviously not. But standard SAT/CSP fail to recognize this. One can do a lot better

12. 12 Pseudo-Boolean Representation Pseudo-Boolean (PB): explicit arithmetic constraints linear inequalities on 0-1 variables e.g., 2x + y + z = 2 �either the radar or two SAM launchers must be struck� maximum constraint on flight hours, etc exponentially more concise than SAT Inference uses �cutting planes� linear combinations plus rounding PB is richer representation: Pigeon-hole problems become feasible polynomial proofs exist; exponentially faster than SAT But can PB be exploited as well as CSP/SAT? PB is well-known representation, and we have long advocated it because it is much more concise than SAT Inference in SAT uses resolution, equivalent in PB is cutting planes The combination is provably exponentially better than SAT. But can it be exploited? PB is well-known representation, and we have long advocated it because it is much more concise than SAT Inference in SAT uses resolution, equivalent in PB is cutting planes The combination is provably exponentially better than SAT. But can it be exploited?

13. 13 Explanation-Driven Search Conflict-driven search select and enforce branch variables analyze conflicts conflict explanations are derived constraints use derived constraints to drive backtracking Application to SAT produces CHAFF heavily used in verification community outperforms local search methods such as WSAT Application to pseudo-boolean gives PARIS richer representation & CHAFF-like performance hope to outperform WSAT(PB) despite being systematic Alejandro discussed use of local search to to PB, we have focused on complete search because it has the potential to produce explanations. At the cost of being MUCH harder to make it work well! In modern solvers the basic loop is driven by analysis of conflicts that arise after enforcing branch variables Doing this for SAT has produced CHAFF which is now heavily considered by the verification community -- all the people who used to use BDDs now also look for SAT solvers We applying it to PB -- want to get best of CHAFF speed and PB powerAlejandro discussed use of local search to to PB, we have focused on complete search because it has the potential to produce explanations. At the cost of being MUCH harder to make it work well! In modern solvers the basic loop is driven by analysis of conflicts that arise after enforcing branch variables Doing this for SAT has produced CHAFF which is now heavily considered by the verification community -- all the people who used to use BDDs now also look for SAT solvers We applying it to PB -- want to get best of CHAFF speed and PB power

14. 14 PARIS Overview Systematic PB solver Goals: combine speed of CHAFF with the power of PB learn to control reasoning in rich representations Advantage: supports powerful reasoning techniques Difficulty: direct generalization to PB didn�t work simple derived constraints don�t always drive a backtrack Lesson: Moving beyond SAT/CSP offers advantages but also new challenges So, like I said PARIS does PB, and with spped of a SAT solver. But an important reason is to study how to control reasoning in a representation that is the natural next step after CSP/SAT Next few slides I�ll show some pros and cons and results. Overall lesson is that �So, like I said PARIS does PB, and with spped of a SAT solver. But an important reason is to study how to control reasoning in a representation that is the natural next step after CSP/SAT Next few slides I�ll show some pros and cons and results. Overall lesson is that �

15. 15 Critical PB Advantage: �Piggybacking� Inference is by resolution, I.e., linear combination, but derived constraints can be more general than SAT equivalents, giving better pruning later in the search. Search State: { d=0, b=1, c=1, e=0, f=0 } Unsat constraints: a+b+c+d = 3  �a + e + f = 1 SAT: blame only { d=0, e=0, f=0 } generates only b v e v f PARIS: also blame { b=1, c=1, � } generates b+c+d+e+f = 3 equivalent to many clauses Learn not only about current failure but also related failures First, A MAJOR and hidden advantage of PB arises as a �side-effect� of the conciseness of the representation. During the conflict inference in the search, the derived constraints can have a much wider scope than if we were limited to SAT. First, A MAJOR and hidden advantage of PB arises as a �side-effect� of the conciseness of the representation. During the conflict inference in the search, the derived constraints can have a much wider scope than if we were limited to SAT.

16. 16 PARIS: Challenges and Lessons Inferred constraints not always unsat in PB, unlike SAT and cardinality. Lesson: use adjustable inference schemes that revert to cardinality when needed PB seems to have a much greater choice of, and sensitivity to, which explanations (constraints) are generated selection of conflict selection of conflict-analysis method tradeoffs between learning full PB and simpler cardinality Generating good explanations is hard. Impact: negotiation by argumentation A technical challenge that arose was that inferred constraints didn�t always drive backtracking. Fix we found was to consider a range of methods to deduce derived constraints. General is there are lots of choices in the process. Relevant to negotiation by argumentation issues Both are influenced by branch-variable selection A technical challenge that arose was that inferred constraints didn�t always drive backtracking. Fix we found was to consider a range of methods to deduce derived constraints. General is there are lots of choices in the process. Relevant to negotiation by argumentation issues Both are influenced by branch-variable selection

17. 17 ISI Instances SNAP: Allocate pilots to planes to ensure that training constraints are satisfied. Structure: Switch variables si � Is mission i selected? Other variables � resource usage, etc. What about performance so far? Considered simple instances obtained from ISI Basic structure is that of a switched problem.What about performance so far? Considered simple instances obtained from ISI Basic structure is that of a switched problem.

18. 18 PARIS on ISI Instances Consider two alternative heuristics: Both branch on switch variables before all others (VSIDS is a standard heuristic from CHAFF) Heuristic A: use static order to select switch Heuristic B: use VSIDS value to select switch If all switch variables are valued then use VSIDS to select non-switch variable. Found two successful heuristics for PARIS. Both take note of the structure of the problem. They branch on switch variables before all others.Found two successful heuristics for PARIS. Both take note of the structure of the problem. They branch on switch variables before all others.

19. 19 Progress on ISI Instance Sat: instance forced to have at least optimal value for objective Unsat: instance forced to have an impossibly good value Solving of the optimal case was now fast. Solving of the optimal case was now fast.

20. 20 Progress on ISI Instances Cannot compare with SAT approach as there is no sensible encoding. Available PB solvers could not handle this instance. Original PARIS took hours. Currently down to < 10 sec Lessons: PB, not just cardinality or SAT, was essential Branching on switch variables is a big help Our current version is a lot faster than anything that existed before, or our earliest attempts.Our current version is a lot faster than anything that existed before, or our earliest attempts.

21. 21 Exploiting PB�s Potential Experimental surprise: plain PARIS is much worse than it �should� be for some problems with embedded-PHP! straightforward translation of SAT approaches to PB can still thrash learning of explanations gets sidetracked need mechanism to guide learning Working hypothesis: Focusing learning on preferred variables will help make it coherent likely to be a general issue for higher-level representations Other lessons. Sometimes we don�t get embedded PHPs Underlying problem seems to be that attempting to generate explanations can get sidetracked. Working on controlling this by focussing the generation mechansims.Other lessons. Sometimes we don�t get embedded PHPs Underlying problem seems to be that attempting to generate explanations can get sidetracked. Working on controlling this by focussing the generation mechansims.

22. 22 Embedded PHP example State-transition �planning� problem Domain: Packages, planes and airports FLY, LOAD, UNLOAD actions Instances: n packages, but n-1 planes. Time constraint tight enough to generate embedded-PHP Domain is a simple planning problem. Set up to have an embedded PHP.Domain is a simple planning problem. Set up to have an embedded PHP.

23. 23 PARIS & Embedded PHP �prefer pa(*,*,2)� forces these to be branch variables first �PHP zchaff� gives performance on same size pure-PHP with SAT-based learning Top two lines are node counts with general branching. Much worse than it ought to be. Improves a lot if focus branching and learning onto critical variables. Still does not do as well as it ought! Understanding and fixing this is current research.Top two lines are node counts with general branching. Much worse than it ought to be. Improves a lot if focus branching and learning onto critical variables. Still does not do as well as it ought! Understanding and fixing this is current research.

24. 24 Suppose coalition C1 and C2 are interacting Previous idea was to avoid �external thrashing� by exchanging solution clusters over the public variables Coalition Communications Now return to issues of external thrashing -- system-of-systems issues. External thrashing can occur by exchanging just point solutions rather than something more general. Have previously talked about use of solution clusters. Now return to issues of external thrashing -- system-of-systems issues. External thrashing can occur by exchanging just point solutions rather than something more general. Have previously talked about use of solution clusters.

25. 25 Solution Clusters Distinguish public and private variables Communicate the projection of the constraints onto the public variables T'(x) := Exists y. T(x,y) Inside coalition: Generate initial solution Scan for variables whose values are inessential, and �unset� them Instead of total assignment T, send partial assignment residual constraints Specifically usually have notion of public and private. Want ot generate theory over the publcis and then had methodds to generate associated solution clusters.Specifically usually have notion of public and private. Want ot generate theory over the publcis and then had methodds to generate associated solution clusters.

26. 26 Problem: suppose C2 actually is unsat �Negotiation as argumentation� Even if we back off from unsat until we just reach sat, it can be hard to control generation of constraints for communication Want finer grain control than just value of objective function � to get easy results first Coalition Communications In practice often need to do opt not just decision. Want to combine with reasoning and solution clusters. Could just back off on global objective. Believe need finer control in order that can generate easy-to-find explanations first. Negotiation as argumentation approach Note on last point here we might be differing from USC -- want to discuss this with them.In practice often need to do opt not just decision. Want to combine with reasoning and solution clusters. Could just back off on global objective. Believe need finer control in order that can generate easy-to-find explanations first. Negotiation as argumentation approach Note on last point here we might be differing from USC -- want to discuss this with them.

27. 27 Coalition Constraints Optimization use switches for subgoals potential changes to nominal demands means should never just have T� == unsat Goal sensitivity & finer control communications should say something directly about subgoals Newer formulation: T'(s,x) := Exists y. T(s,x,y) Finding T� is significant computational issue Communication can be based on derivation of upper and lower bounds on T' in incremental fashion incrementally increase number of si set to true (goals attainable) Seems to be a relationship between effective calculation of T�, and the issues of focusing learning in PARIS Believe focusing learning on preferred variables will again help maintain coherence Current work is suggestion of relevant forumulation, and the assocaited computational issues. Specifically introduce explicit vars for each subgoal, not just the entire objective. Projection onto public then includes these switch vars. Goal Is to be able to do generate appropriate solution clusters and constraints. Believe that are relations between the underlying search problems. Typically expect T'(si=false,x) is weak set of constraints T'(si=true,x) is �bottom� Current work is suggestion of relevant forumulation, and the assocaited computational issues. Specifically introduce explicit vars for each subgoal, not just the entire objective. Projection onto public then includes these switch vars. Goal Is to be able to do generate appropriate solution clusters and constraints. Believe that are relations between the underlying search problems. Typically expect T'(si=false,x) is weak set of constraints T'(si=true,x) is �bottom�

28. 28 Summary PARIS: fast SAT-style solver, using PB constraints for the counting needed for resource constraints, etc. Learning of derived constraints has potential to allow explanation of decisions (unlike hill-climbers) intended use is for interactions between ants or coalitions Structure of problems�decision variables vs. auxiliary�can be exploited to significantly improve solution times Coalition communications: Extending previous solution clusters methods to account for �switched structure� of problems over-constrained nature of problems need for appropriate derived constraints and �solution clusters� in anytime fashion Focused learning needed for both

29. 29 Project Plans Release generic PARIS Release version tailored to issues in ANTS switched nature of constraints (incremental) generation of explanations over public/switch variables and hiding of private variables Testing on ANTS problem Document key ideas in a generic form for use in other representation schemes (e.g., MAPLANT�s constraint programming language)

30. 30 Project Plans: Distributed Computing Distributed PARIS: modify to interact with other solvers tackling other parts of the problem, with limited communications Metrics: handle problems too large for centralized search show advantages of interactions based on negotiation over ownership of variables show advantages of interaction based upon exchange of robust solutions

31. 31 Project Plans: Theory/Experiment Explore focused learning/common languages Follow up on synergy between robust solutions and negotiation Understand non-PT cliffs genesis? prediction methods Metrics: reduction in negotiation effort improved solution quality/predictability ability to predict cliffs

32. 32 Project Schedule and Milestones Accomplishments: Develop efficient and effective constraint-reasoning and dependency methods Develop theory and experiments of how robust solutions affect coalition interaction within coarse-grained distributed search Develop parallel complexity theory relevant to fine-grained distributed ANTS problems Delayed: Use of dependency methods and robust solutions to dynamically adjust coalitions, and control search.

33. 33 Technology Transition/Transfer Exploring potential use of PARIS solver in microprocessor verification. Existing SAT solvers are already being used in that field and using pseudo-Boolean rather than SAT should allow larger problems to be solved. Paper presented at �Microprocessor Testing and Verification; MTV�02�.)

34. 34 Program Issues Difficulty collaborating with terminating projects.