Multi-Agent-Based Search for Resource Allocation Problem

1. Characterizing the Behavior of a Multi-Agent-Based Search by Using it to Solve a Tight, Real-world Resource Allocation Problem Hui Zou and Berthe Y. Choueiry Constraint Systems Laboratory Department of Computer Science and Engineering University of Nebraska-Lincoln {hzou|choueiry}@cse.unl.edu

2. Introduction Search algorithms: systematic or iterative repair Complex, real-world optimization problems • Systematic search thrashes • Local search gets stuck in ‘local optima’ • Remedial: random walk, breakout, restart strategies, etc. Multi-agent-based search [Liu & al. AIJ 02] • Also an iterative repair technique • provides us with a new way • Advantages & shortcomings via a practical application

3. Background - GTA Graduate Teaching Assistants (GTA) problem: In a semester, given • a set of courses • a set of graduate teaching assistants • a set of constraints that specify allowable assignments Find a consistent and satisfactory assignment of GTAs to courses • Types of constraints: unary, binary, non-binary • Each course has a load, indicates weight of the course • Each GTA has a (hiring) capacity, limits max. load Detailed modeling in [Glaubius & Choueiry ECAI 02 WS on Modeling]

4. Background - GTA (cont’) • In practice, this problem is tight, even over-constrained • Our goal: ensure GTA support to as many courses as possible Problem size: B – boosted to make problem solvable O – original, not necessary solvable

5. Background - GTA (cont’) Optimization criteria: • Maximize the number of courses covered • Maximize the geometric average of the assignments wrt the GTAs’ preference values (between 0 and 5). Problem: • Constraints are hard, must be met • Maximal consistent partial-assignment problem (MPA-CSP?) • Not a MAX-CSP (which maximizes #constraints satisfied)

6. Background - MAS for CSPs • Multi-Agent System: agents interact & cooperate in order to achieve a set of goals • Agents: autonomous (perceive & act), goal-directed, can communicate • Interaction protocols: governing communications among agents • Environment: where agents live & act • ERA [Liu & al. AIJ 2002] • Environment, Reactive rules, and Agents • A multi-agent approach to solving a general CSP • Transitions between states when agents move

7. Background - ERA’s components ERA=Environment + Reactive rules + Agents Environment:a n×mtwo-dimensional array • n: the number of variables (agents) • m: the maximum domain size, |Dmax| • e(i, j).value: domain value of agent i at position j • e(i, j).violation: violation value of agent i at position j • Zero position: where e(i, j).violation=0 When all agents are in zero position, we have a complete solution Example:

8. Background - ERA’s components ERA=Environment + Reactive rules + Agents Reactive rules: • Least-move: choose a position with the min. violation value • Better-move: choose a position with a smaller violation value • Random-move: randomly choose a position Combinations of these basic rules form different behaviors.

9. Background - ERA’s components ERA=Environment + Reactive rules + Agents Agents:a variable is represented by an agent At each state, an agent chooses a position to move to, following the reactive rules. The agents keep moving until all have reached zero position, or a certain time period has elapsed. All agents in zero position Some agents in zero position Assignments are made only for agents in zero position

10. Background - ERA vs local search ERA operates by local repairs, how different is it from local search? ERA • Each agent has an evaluation function • At each state, any agent moves wherever it desires to move Control is localized: Each agent is in pursuit of its own happiness Local search with min-conflict • One evaluation function for the whole state (cost), summarizes the quality of the state • At each state, few agents are allowed to move (most unhappy ones) Control is centralized: towards one common good

11. 2 0 2 0 2 1 3 2 2 Init Eval (agent Q1) Move (agent Q1) Eval (agent Q2) 2 1 2 1 1 1 3 1 0 0 Eval (agent Q3) Move (agent Q3) Eval (agent Q4) Move(agent4) Background - Example ( ERA ) 4-queen problem

12. Background - Example (ERA vs. Local search) ERA – any agent can kick any other agent from its position Local search with min-conflict –cannot repair a variable without violating a previously repaired variable

13. Empirical study - In general • Apply ERA on GTA assignment problem: 0. (Test & understand the behavior of ERA) • Compare performance of: • ERA: FrBLR • LS: hill-climbing, min-conflict & random walk • BT: B&B-like, many orderings (heuristic, random) • Observe behavior of ERA on solvable vs. unsolvable problems • Observe behavior of individual agents in ERA • Identify a limitation of ERA: deadlock phenomenon • 8 instances of the GTA assignment problem

14. Unassigned Courses Unassigned Courses Unassigned Courses Solution Quality Solution Quality Solution Quality CC (×108) CC (×108) CC (×108) Unused GTAs Unused GTAs Unused GTAs Available Resource Available Resource Available Resource Unassigned Courses Unassigned Courses Unassigned Courses Unassigned Courses Unassigned Courses Unassigned Courses Unassigned Courses Solution Quality Solution Quality Solution Quality Solution Quality Solution Quality Solution Quality Solution Quality Unused GTAs Unused GTAs Unused GTAs Unused GTAs Unused GTAs Unused GTAs Unused GTAs CC (×108) CC (×108) CC (×108) CC (×108) CC (×108) CC (×108) CC (×108) Solution Quality Solution Quality Solution Quality Solution Quality Solution Quality Solution Quality Solution Quality Unused GTAs Unused GTAs Unused GTAs Unused GTAs Unused GTAs Unused GTAs Unused GTAs CC (×108) CC (×108) CC (×108) CC (×108) CC (×108) CC (×108) CC (×108) Solution Quality Solution Quality Solution Quality Solution Quality Solution Quality Solution Quality Solution Quality Available Resource Available Resource Available Resource Available Resource Available Resource Available Resource Available Resource Unassigned Courses Unassigned Courses Unassigned Courses Unassigned Courses Unassigned Courses Unassigned Courses Unassigned Courses Available Resource Available Resource Available Resource Available Resource Available Resource Available Resource Available Resource Unassigned Courses Unassigned Courses Unassigned Courses Unassigned Courses Unassigned Courses Unassigned Courses Unassigned Courses # Courses # Courses # Courses # Courses # Courses # Courses # Courses Ratio= Ratio= Ratio= Ratio= Ratio= Ratio= Ratio= Unused GTAs Unused GTAs Unused GTAs Unused GTAs Unused GTAs Unused GTAs Unused GTAs Available Resource Available Resource Available Resource Available Resource Available Resource Available Resource Available Resource CC (×108) CC (×108) CC (×108) CC (×108) CC (×108) CC (×108) CC (×108) Original/Boosted Original/Boosted Original/Boosted Original/Boosted Original/Boosted Original/Boosted Original/Boosted # GTAs # GTAs # GTAs # GTAs # GTAs # GTAs # GTAs Total Capacity Total Capacity Total Capacity Total Capacity Total Capacity Total Capacity Total Capacity Total Load Total Load Total Load Total Load Total Load Total Load Total Load Solvable? Solvable? Solvable? Solvable? Solvable? Solvable? Solvable? Empirical study 1- Performance comparison Original/Boosted Solvable? # GTAs # Courses Total capacity (C) Total load (L ) Ratio= C \ L • - Only ERA finds complete solutions to all solvable instances • On unsolvable problems, ERA leaves too many unused GTAs • LS and BT exhibit similar behaviors Observations:

15. Empirical study 2- Solvable vs unsolvable • Observation: • Number of agents in zero-position per iteration • ERA behavior differs on solvable vs. unsolvable instances ERA performance on solvable problems ERA performance on unsolvable problems

16. Empirical study 3- Behavior of individual agents • Instances • solvable • unsolvable • Motion of agents • variable • stable • constant • Observations:

17. Empirical study 4- Deadlock Observation: ERA is not able to avoid deadlocks and yields a degradation of the solution on unsolvable CSPs. • Each circle corresponds to a given GTA • Each square represents an agent • A blank squares indicate that an agent is on a zero-position • The squares with same color indicate agents involved in a deadlock

18. Discussion + advantages – shortcomings

19. Dealing with the deadlock • Possible approaches: • Direct communications, negotiation mechanisms • Hybrids of search • Global control • Conflict resolution • Experiments: • Enhancing ERA with global control • Don’t accept a move that deteriorates the global goal • Lead to local-search-like behavior (i.e., local optima) • ERA with conflict resolution • add dummy resources • find a complete solution when LS and BT fail • remove dummy assignments, solutions are still better

20. Future research directions • Enhance ERA to handle optimization problems • Test approach using other search techniques • BT search: Randomized, credit-based • Other local repair: squeaky-wheel method • Market-based techniques, etc. • Validate conclusions on other CSPs • random instances, real-world problems • Try search-hybridization techniques References: R. Glaubius and B.Y. Choueiry, Constraint Modeling and Reformulation in the Context of Academic Task Assignment. In Workshop Modeling and Solving Problems with Constraints, ECAI 2002. J. Liu, H. Jing, and Y.Y. Tang. Multi-Agent Oriented Constraint Satisfaction. Artificial Intelligence, 136:101-144, 2002.

21. Questions