Combining Job and Team Selection Heuristics

1. Combining Job and Team Selection Heuristics Chris L. D. Jones and K. Suzanne Barber

2. Selfish Agents Making Strategic Decisions Goal is to maximize profit by maximizing its own estimated payoff function Selfish Agents

3. Defining the Environment Selfish Agents Non-zero probability that subtasks will be added or subtracted from a job Dynamic Environments Unenforceable Contracts Agents face no penalty from de-committing from a team

4. Scenario: Freelancers on the Internet However, lack of a single needed skill can break entire enterprise Selfish agents have the opportunity to create a lucrative website In a dynamic environment, elements of the solution change during development Creating the website involves multiple elements: hardware, software, content and advertising Because contracts are unenforceable, some freelancers quit without warning

5. Selfish Agents and Unenforceable Contracts • Selfish agents exchange goods or services without an enforcement mechanism if exchange parameters are static[Sandholm and Lesser, 1995] • Agents may form institutions or coalitions based on static common goals [Gaertner et al, 2008] • Selfish agents utilize concepts such as the Core, Kernel, and Shapley value to navigate static coalitional games • [Myerson, 1991; Davis et al, 1963; Shapley, 1997] Selfish Agents • Bounded search may be used to search through static coalition space of selfish agents [Sandholm et al, 1999; Rahwan et al, 2007] Dynamic Environments Unenforceable Contracts All approaches rely on valuations and planning associated with static problems

6. Dynamic Environments and Unenforceable Contracts • Cooperative agents may be reassigned roles and tasks within a team as circumstances change [Tambe et al, 2000; Nair et al 2003] • Agents cooperate to solve distributed constraint satisfaction problems [Scerri et al, 2005; Modi et al, 2001] • Cooperative agents work in highly dynamic Robocup Rescue domain [Nazemi et al, 2005; Nair et al, 2001; Lau et al 2005] Selfish Agents All approaches rely on cooperative agents maximizing team’s utility over their own Dynamic Environments Unenforceable Contracts

7. Selfish Agents in a Dynamic Environment • Selfish agents in dynamic environments utilize contingency contracts to guard against specific events [Raiffa, 1982; Faratin and Klein, 2001] Selfish Agents • Leveled commitment contracts allow agents to leave a team by paying a penalty [Sandholm and Lesser, 1996; Sandholm et al, 1999; Andersson et al, 2001] Dynamic Environments Unenforceable Contracts • Central fault-tolerance frameworks can enforce penalties on agents which fail to provide contracted services [Smith, 1980; Dellarocas and Klein, 2000; Patel et al, 2005] All approaches rely on some form of contract enforcement between agents

8. A Gap in the Prior Work Selfish Agents • Trust and/or reputation information may be used to find agents less likely to defect [Fullam, 2007] • Agents may follow societal norms which minimize defection [Oren et al, 2008] Dynamic Environments Unenforceable Contracts Selfish agents in a dynamic environment with unenforceable contracts

9. Strategies to Maximize Payoff Agent needs to select a profitable job to work on Agent needs to select a capable team to work with Selfish agents in a dynamic environment with unenforceable contracts Agent combines a job selection heuristic with a team selection heuristic to form a profit-maximizing strategy

10. Building Strategies by Combining Heuristics • Combining a job selection heuristic with a team selection heuristic produces a strategy • Two job selection heuristics and five team selection heuristics gives us ten possible strategies • Previous simulation work created multiple classes of agents each of which executed a different strategy

11. Job Selection Heuristics • Greedy job heuristic • Selects job most profitable to foreman while taking completed work into account • Lean job heuristic • Selects job which can be completed the quickest

12. Team Selection Heuristics • Null team heuristic • Randomly selects team from top-ranked job • Fast team heuristic • Minimize time to completion • Redundant team heuristic • Maximize number of duplicate skills • Auxiliary team heuristic • Maximize number of unused skills • MinPartnerteam heuristic • Minimize number of partners

13. Benefits of Heuristic-based Strategies • Strategies allow selfish agents to make immediate estimates of how their actions will effect their utility • Immediate estimates of job worth can be used since no decommitment penalties are possible • Estimates of team worth based on how well teams may adapt to dynamic circumstances • Use agent simulation to test relative utility of team formation strategies [Jones and Barber, 2007] • Previous work did not explore different mechanisms for utilizing heuristic information • How should job and team selection heuristics be combined in a strategy?

14. Approach: Separate Job/Team (SJT) Formation Foreman Agent Job selection heuristic selects the most attractive jobs Information about job heuristic value does not affect team selection process Team selection heuristic ranks the most attractive teams for all selected jobs Therefore, SJT may prefer more robust teams at the expense of agent profit Foreman agent selects the top-ranked team and sends request to form team Agents respond to request based on job heuristic

15. Approach: Combined Job/Team (CJT) Formation Foreman Agent Job selection heuristic selects the most attractive jobs Normalized heuristics are multiplied together, so that job heuristic information influences team selection process Combined job and team selection heuristics rank the best job/team assignments for all selected jobs Robust teams are therefore not selected at the expense of agent profit Foreman agent selects the top-ranked job/team pairing and sends request to form team Agents respond to request based on combined job and team heuristics

16. Experimental Parameters 2500 agents in simulation Simulation tests increasingly dynamic environments by changing required subtasks

17. Credit Earned at 0% dynamicism CJT agents equal or exceed SJT agents by statistically significant margins Credit Earned Agent Strategies

18. Credit Earned at 25% dynamicism GMP strategy works best in relatively static environments Credit Earned Agent Strategies

19. Credit Earned at 50% dynamicism Credit Earned Agent Strategies

20. Credit Earned at 75% dynamicism Credit Earned GA strategy works best in relatively static environments Agent Strategies

21. Credit Earned at 100% dynamicism Credit Earned The CJT advantage over SJT agents continues over all sampled dynamicism values Agent Strategies

22. Jobs Completed at 0% Dynamicism Attempted jobs successfully completed CJT likewise has a statistically significant advantage over SJT in percentage of jobs successfully completed Agent Strategies

23. Jobs Completed at 25% Dynamicism Attempted jobs successfully completed Agent Strategies

24. Jobs Completed at 50% Dynamicism Attempted jobs successfully completed Agent Strategies

25. Jobs Completed at 75% Dynamicism Attempted jobs successfully completed Agent Strategies

26. Jobs Completed at 100% Dynamicism Attempted jobs successfully completed The CJT advantage over SJT agents in jobs completed continues for all sampled dynamicism values CJT LA job completion improves markedly Agent Strategies

27. Conclusions and Future Work • Simultaneous usage of job and team selection heuristics improves credit earned and jobs completed • Mechanism works in dynamic environments where thousands of selfish agents work without enforceable contracts • Future work: • Dynamic weighting of job and team-selection heuristics • Development of theoretical framework for determining job and team selections

28. THANK YOU!QUESTIONS?

29. References • Kraus, S., O. Shehory, et al. (2003). Coalition formation with uncertain heterogeneous information, ACM Press New York, NY, USA: 1-8. • Tambe, M., D. V. Pynadath, et al. (2000). Building dynamic agent organizations in cyberspace. 4: 65-73. • Sandholm, T. W. and V. R. Lesser (1995). Equilibrium Analysis of the Possibilities of Unenforced Exchange in Multiagent Systems, University of Massachusetts at Amherst, Computer Science Dept. • Myerson, R. B. (1991). Game theory: analysis of conflict, Harvard University Press. • Davis, M. and M. Maschler (1963). THE KERNEL OF A COOPERATIVE GAME, DTIC Research Report AD0418434. • Shapley, L. S. (1997). A VALUE FOR n-PERSON GAMES, Princeton University Press. • Sandholm, T., S. Sikka, et al. (1999). Algorithms for optimizing leveled commitment contracts: 535-540. • Rahwan, T., S. D. Ramchurn, et al. (2007). Near-optimal anytime coalition structure generation: 2365-2371. • Gaertner, D., Rodrigez, J. A., et al. (2008. Agreeing on Institutional Goals for Multi-agent Societies. • Nair, R., M. Tambe, et al. (2003). Role allocation and reallocation in multiagent teams: towards a practical analysis, ACM Press New York, NY, USA: 552-559. • Nazemi, E., M. Faradad, et al. (2005). SBCe_Saviour Team Description. Tehran, Iran, ShahidBeheshti University: 6. • Nair, R., T. Ito, et al. (2001). Task Allocation in the RoboCup Rescue Simulation Domain: A Short Note, Springer. • Lau, N., L. P. Reis, et al. (2005). FC Portugal 2005 Rescue Team Description: Adapting Simulated Soccer Coordination Methodologies to the Search and Rescue Domain. • Scerri, P., A. Farinelli, et al. (2005). Allocating tasks in extreme teams, ACM Press New York, NY, USA: 727-734. • Modi, P. J., H. Jung, et al. (2001). A dynamic distributed constraint satisfaction approach to resource allocation, Springer. • Raiffa, H. (1982). The art and science of negotiation, Belknap Press of Harvard University Press Cambridge, Mass.

30. References • Faratin, P. and M. Klein (2001). Automated Contract Negotiation and Execution as a System of Constraints, MIT, Cambridge. • Sandholm, T. W. and V. R. Lesser (1996). Advantages of a leveled commitment contracting protocol: 126-133. • Sandholm, T., S. Sikka, et al. (1999). Algorithms for optimizing leveled commitment contracts: 535-540. • Andersson, M. R. and T. W. Sandholm (2001). Leveled commitment contracts with myopic and strategic agents, Elsevier. 25: 615-640. • Smith, R. G. (1980). The contract net protocol. 29: 1104-1113. • Dellarocas, C. and M. Klein (2000). An experimental evaluation of domain-independent fault handling services in open multi-agent systems: 95-102. • Patel, J., W. T. L. Teacy, et al. (2005). Agent-based virtual organisations for the Grid, IOS Press. 1: 237-249. • Fullam, K. (2007). Learning Trust Decision Strategies in Emerging Reputation Networks. • Oren, N., Luck, M., et al. (2008). An Argumentation Inspired Heuristic for Resolving Normative Conflict. • Jones, C. L. D. and K. S. Barber (2007). Bottom-up Team Formation Strategies in a Dynamic Environment: 60-74. • Jones, C. L. D. and K. Barber (2008). Combining Job and Team Selection Heuristics. 2008 AAMAS Workshop on Coordination, Organization, Institutions and Norms. Lisbon, Portugal, ACM. • Sutton, R. S. and A. G. Barto (1998). Reinforcement Learning: An Introduction, MIT Press. • Klusch, M. and A. Gerber (2002). Dynamic coalition formation among rational agents. 17: 42-47. • Lin, C., S. Hu, et al. (2007). An Anytime Coalition Restructuring Algorithm in an Open Environment, Springer. 4681: 80.

31. BACKUP SLIDES

32. Causes of Agent and Job dynamicism • Changes to Job requirements • Bounded rationality • Incomplete information • Inherent environmental dynamics • Changes to Team membership • Agent failure • Agent defection • Changes to current job requirements make job less attractive • Changes to alternate job requirements makes job more attractive • Teammate defection decreases robustness of current team, likelihood of expected payoff

33. Domain Assumptions • Agents are multiskilled • Single-skilled agents would be unable to provided the reserve of redundant skills needed • Contractless environments feature non-transferable utility • With transferable utility, contracts become possible • Quality and Timeliness are not represented • Both could probably be represented as different types of subtasks, e.g. subtask requiring 90% QoS is different than subtask requiring 50% QoS • Quality and Timeliness are both likely to be domain dependent – not a huge difference between 80% and 100% QoS in ditch digging, but huge difference in brain surgery

34. Comparison to Trust work • This work is complementary to trust • Can be used when trust information is unavailable or unreliable • Can be used when environmental dynamics make agents change their trustworthiness over time • Work is supplementary to trust • Can be used in addition to trust, to compensate for non-trustworthy agents • Can be used when trust system is bootstrapping, and agents need to explore space of likely untrustworthy agents

35. Foreman flowchart

36. Worker flowchart