The story of distributed constraint optimization in LA: Relaxed

1. The story of distributed constraint optimization in LA:  Relaxed Master’s Thesis Steven Okamoto Computer Science Department University of Southern California December 8, 2003

2. Motivation: Role allocation in multiagent teams • Large teams of agents, robots, people important in many domains • Key challenge: Allocate large number of roles to agents • Dynamic environment (roles change) • Heterogeneous roles (agents have differing capabilities to perform roles) • Role constraints (some roles require other roles to be performed)

3. The Problem • Distributed constraint-based reasoning important for role allocation • Key weaknesses of existing distributed constraint-based role allocation algorithms: • Time: must run to completion • Communication • Dynamism: run again, many role changes • Simplified agents and roles: • Only perform one task • No interactions between roles

4. LA-DCOP • Low communication, approximate DCOP • Key Ideas: • Thresholds: global probabilistic information, improve output • Tokens: reduces communication, highly distributed • Potential tokens: AND-constrained roles • Key results • Empirical evaluation using abstract simulator: higher output, lower communication than competition • Calculation of best threshold in simple cases • Key limitations • Inappropriate for domains requiring strict optimality

5. Outline • Generalized Assignment Problem • Natural representation of role allocation problem • Extended Generalized Assignment Problem • Extends GAP to allow dynamism and constraints among roles • Distributed Constraint Optimization Problem • LA-DCOP algorithm • Theoretical calculation of thresholds • Empirical results

6. Example: E = {e1, e2} R = {r1, r2, r3, r4} A = e1 performs r1 e2 performs r2 and r3 no one performs r4 Generalized Assignment Problem - GAP • Known to be NP-complete • Set of agents E, set of roles R • Capability: ability to perform roles Cap(ei, rj) →[0, 1] • Resources: each agent has 1.0 resources Resources(ei, rj) →[0, 1] • Allocation matrix A = (ai,j) ai,j = 1 if ei is performing rj 0 otherwise

7. reward for all roles performed by ei Agents do not exceed resources total reward over all agents No role is performed by more than one agent GAP – The Goal • Goal is to find A that maximizes total reward: • While respecting constraints:

8. number of constrained roles being performed = 1 if rm is being performed sum for all constrained roles number of constrained roles Extended GAP – E-GAP • Dynamism: E, R, Cap, Resources indexed by time • Coordination Constraints: # • AND – all roles in a set must be simultaneously performed • Ex. Big fire requires multiple firefighting roles • Delay Cost • Find sequence A of allocations

9. Reward for performed roles Penalty for unperformed roles Roles are never performed by more than one agent at a time Agents never exceed resources E-GAP – The Goal • Find A that maximizes • While respecting constraints

10. Distributed Constraint Optimization Problem - DCOP • Set of variables V: agents • Can have multiple values • Values: roles • Objective function: total reward • Cost functions: reward, delay cost, resources, single agent allocation • Complete graph

11. LA-DCOP • Low communication, approximate DCOP • Create a token for each role • Token required to perform role • Distribute tokens to agents • Agents pass tokens around • Keep token: 2 levels 1. Thresholds: DCSP 2. Kickout: locally optimal • Potential tokens: AND constraints

12. Threshold Kickout Kickout LA-DCOP - Example Threshold = 0.7 Only one role at a time A: 0.2 B: 0.8 A: 0.8 B: 0.9 A A B A: 0.8 B: 0.6 A: 0.9 B: 0.4

13. LA-DCOP - Retainers • AND constrained: deadlock, starvation • Inefficient for agents to wait for coalition • Potential tokens • Promise to perform role when coalition formed: agent is “retained” • Agent can do other roles in the meantime • Algorithm: • Give real tokens to “owner” • Distributes potential tokens • Monitors for coalition formation • “Locks” coalition by sending out real tokens

14. Agent algorithm Check threshold Enter retainer for potential token Add real token Compute locally optimal allocation Keep only retained tasks that would perform Retainer lock

15. Run by the owner of AND-constrained set of roles Distribute tokens Wait for at least one token for each task Choose best agents from among retainers AND Owner Algorithm

16. Steady state – output stops changing Simple case: M roles N agents K types of roles, chosen randomly Static environment Independent, uniform(0, 1) capabilities Roles require 0.25, 0.5, or 0.75 resources EU(T) = E(#tasks executed|T)  E(capability|T) Picking Thresholds – Steady State

17. Only increasing part of function Maximum at this value Picking thresholds

18. How well does the prediction match? Predicted max

19. High is good Low is good Comparison to other algorithms Communication Output • DSA: Distributed, approximate • Greedy: Greedy auction with centralized auctioneer

20. 10% of agents have non-zero capability for each type Threshold and dynamics

21. Retainers

22. Summary • Thresholds • Increase output: higher output than competition • Calculate offline from probabilistic information • Tokens • Reduce communication: orders of magnitude • Potential tokens: AND constraints • Trades off optimality for efficiency

24. Calculating E(#tasks executed) • Allocate roles using fewest resources first • Z = #tasks requiring each resource amount = M/3 • Nx = #agents required to execute all tasks requiring x resources • N0.25 = Z / 4 • N0.5 = Z / 2 • N0.75 = Z • Ncapable = E(#capable agents)

25. Calculating threshold • Capable agent has at least one capability > T EU(T) = E(#capable agents)  E(#tasks per capable agent)  E(capability of capable agent) = E(#tasks executed)  E(capability of capable agent) • E(#capable agents) = (1 – TK)  N • E(capability of capable agent) = (1 + T) / 2 Pr(all caps<T)

26. Calculating E(#tasks executed) • If Ncapable < N0.25 E(#tasks executed) = 4 * Ncapable • Else if Ncapable < (N0.25 + N0.5) E(#tasks executed) = Z + 2 * (Ncapable – N0.25) • Else if Ncapable < (Z / 4 + Z / 2 + Z) E(#tasks executed) = Z + Z + Ncapable – (N0.25 + N0.5) = (5/4) * Z + Ncapable • Else E(#tasks executed) = M