Agent-Organized Networks for Dynamic Team Formation

1. Agent-Organized Networks for Dynamic Team Formation Marie desJardins mariedj@cs.umbc.edu Matt Gaston mgasto1@cs.umbc.edu Multi-Agent Planning and Learning Laboratory (MAPLE) Department of Computer Science and EE University of Maryland Baltimore County

2. Overview • Introduction and Motivation • Team Formation • Agent-Organized Networks • Experimental Results • Related Projects: • Connections Model AONs • AONs for Production and Exchange • Stable Team Formation • Future Work and Conclusions Agent-Organized Networks

3. Introduction and Motivation Agent-Organized Networks

4. Introduction: Multi-Agent Systems • Agent: Autonomous, intelligent software system. • Physical (robot, autonomous vehicle, mobile sensor) or virtual (search, travel planning, trading / e-commerce, information retrieval) • MAS: “Community” of agents – competitive or cooperative • Connections form a “social network” of agents Agent-Organized Networks

5. Why Adapt? • Multi-agent systems are growing in popularity and size • Technologies like the Semantic Web support the deployment and evolution of large-scale, dynamic multi-agent systems • Agent cognitive capacities are limited, preventing all agents from knowing/interacting with all other agents • Previous findings suggest that network structure plays an essential role in understanding team formation dynamics in multi-agent systems • Identifying the “best” network structure is difficult or impossible to do a priori • Solution: Agent-Organized Networks Agent-Organized Networks

6. Team Formation[AAMAS 2005] Agent-Organized Networks

7. Multi-Agent Team Formation Model • Agents must form teams to complete tasks • Agent states: • Uncommitted • Committed • Active • Tasks are advertised to the network of agents • A valid team: • Connected path in network • Task skill requirements met • Formed within time constraints 1 1 2 2 2 2 2 3 4 1, 1, 2, 3 2, 2 1, 2, 2, 2, 4 BLOCKED! Agent-Organized Networks

8. Multi-Agent Team Formation Model • Some model details • Parameters • Number of agents: N • Skill diversity:  • Task introduction interval:  • Team/task size: T • Advertisement duration:  • Task duration:  • Network structure • Organizational Efficiency 1 1 2 2 2 2 2 3 4 efficiency = # of tasks successfully completed total # of tasks advertised Agent-Organized Networks

9. Team Joining Strategy Considering each task in random order... With some initiation probability, start a new team if needed: Always join a team if it’s already been started, and it needs your skill. Agent-Organized Networks

10. Agent-Organized Networks Agent-Organized Networks

11. Agent-Organized Networks • Definition: An agent-organized network (AON) is an organizational network structure, or agent-to-agent interaction topology, that is the result of local rewiring decisions made by the individual agents in a networked multi-agent system. • Design considerations: • Local perception of global performance • Adaptation triggers • Rewiring strategies • Evaluation metrics: • Learning rate • Stability • Structural properties of resulting networks Agent-Organized Networks

12. Structure-Based Adaptation • Adapt based on preferential attachment • Natural network formation process that leads to scale-free networks • Adaptation trigger (random): • Probability of adaptation for each uncommitted agent: 1/N • Rewiring strategy: • Disconnect from a random neighbor • Connect to some neighbor’s neighbor with probability Agent-Organized Networks

13. Performance-Based Adaptation • Adaptation trigger: • Adapt if performance drops below neighbors’ average performance: • Rewiring strategy: • Drop the lowest-performing neighbor: • Add a connection to the highest-performing neighbor ak of the highest-performing neighbor al: Agent-Organized Networks

14. Results Agent-Organized Networks

15. Experimental Setup • Initial network structure: Random geometric graph • Randomly place agents in a unit square • Connect agents that are closer than d units apart • Use the minimal d that guarantees all neighbors have at least one edge • Run team formation with no adaptation to establish baseline • Run with each adaptation strategy separately • Results are an average of 50 runs Agent-Organized Networks

16. Results: Summary • Significant performance improvement (over baseline) for both AON methods Agent-Organized Networks

17. Stability of Networks • Structure-based AONs outperform performance-based AONs, but result in substantially more rewirings • Performance-based AONs are more efficient (“better value” if adaptation cost is in similar units to performance measure) Agent-Organized Networks

18. Evolution of the Network: Structure-Based • Converges to a network with hub structure and short average path length Agent-Organized Networks

19. Evolution of the Network: Performance-Based • Convergence to short-average-path-length structure happens more slowly • Qualitatively similar structure to strategy-based (but in this case not by design!) Agent-Organized Networks

20. Connections Model AONs[AAAI 2005 Workshop on Multi-Agent Learning] Agent-Organized Networks

21. The (Symmetric) Connections Model • Symmetric when ij=  and cij = c for all i and j • 0 <  < 1 is the value of a relationship, discounted by distance • c is the cost of a direct connection (Jackson & Wolinsky 1996; Jackson 2002) Agent-Organized Networks

22. Dynamic Network Formation in SCM Based on pairwise stability (Watts 2001): • At each iteration: • Two agents meet (are selected) at random (synchronous) • If they have a connection, they remove the connection if at least one of them benefits -- unilateral deletion • If the do not have a connection, they add a connection if it is mutually beneficial -- bilateral creation But . . . Agent-Organized Networks

23. Experiment: Watts Dynamic Network Formation  = 0.9, c = 0.8, optimal = 7878.42 Agent-Organized Networks

24. A (Simple) Multi-Agent Learning Approach • Goals: • Eliminate need for “global” knowledge • Eliminate need for “global” computation • Maintain bilateral network formation (agents agree to create link) • Follow dynamic network formation process of Watts • On-line learning • Approach • Stateless Q-Learning (Claus & Boutilier 1998) • A = { add, delete, nothing} • Agents add connection if both have largest Q value for add (bilateral) • Agents remove connection if one has largest Q value for delete (unilateral) • Reinforcement signal comes from omniscient oracle (!) Agent-Organized Networks (AONs) “Distributed Annealing” Agent-Organized Networks

25. Experiment: Learning to Form Networks  = 0.9, c = 0.8, optimal = 7878.42 Adaptive Learning Rate: Win or Lose Fast (WoLF) (Bowling & Veloso 2002) Agent-Organized Networks

26. Experiment: Adding an Unselfish Agent  = 0.9, c = 0.8, optimal = 7878.42 Agent-Organized Networks

27. AONs for Production and Exchange[AAAI 2005] Agent-Organized Networks

28. A Model of Production and Exchange • n agents in an artificial economy with two goods • Each agent i possesses g1iunits of good 1 and g2i units of good 2 • Each agent is a producer of either good 1 or good 2 • At each iteration of the model, the agents are selected in random order and choose between initiating trade with another agent or producing their respective good in order to maximize utility • Agent utility: fully rational behavior (Wilhite 2001: 2003) Agent-Organized Networks

29. Push Referral AON Strategies • Random referral: agent selected randomly from Nj(i) • Degree referral: • Production referral: Definition: Assuming that agent i is adapting its connection to agent j, a push referral is a local rewiring by i from j to an agent in Nj(i) Agent-Organized Networks

30. Results degree referral random selection production referral n = 400, q = 30,  = 0.05 • =  = 0.1 • =  = 0.1 initialized values to 1 Agent-Organized Networks

31. Stable Team Formation[AAAI 2004 Workshop on Team/Coalition Formation] Agent-Organized Networks

32. Economic Model of Team Formation • Share-based scheme for pay-off distribution • Team’s revenue is stored in “team account” • Team members get shares for joining and working • Share value = team account / # outstanding shares • Agents bound to the team by a contract • Joining Shares, Sjoin : sign-on bonus • Commission, Scomm: shares given to the agent for every task completed by the team in which the agent actively participates • Dividend, Sdiv: shares given to the agent for every task completed by the team in which the agent does not participate (Dividend < Commission) • Penalty, p: the amount to be paid to the team when leaving the team Agent-Organized Networks

33. Results: Effect of Deadlines Agent-Organized Networks

34. Results: Stable vs. Dynamic Agents Agent-Organized Networks

35. Conclusions and Future Work Summary: • AONs based only on local knowledge can improve team formation in networked MAS • AON ideas can also be applied to other MAS domains and models • Stability can be achieved through a contractual model of team formation Future Work: • Quantitative analysis of post-adaptation network structures • Learning individual agent team selection strategies • [JAAMAS 2006] • Skill placement and replacement for dynamic team formation Agent-Organized Networks

36. Questions? Agent-Organized Networks