Lecture VI: Collective Behavior of Multi-Agent Systems II: Intervention

1. Lecture VI:Collective Behavior of Multi-Agent Systems II:Intervention Zhixin Liu Complex Systems Research Center, Academy of Mathematics and Systems Sciences, CAS

2. In the last lecture, we talked about Collective Behavior of Multi-Agent Systems I: Analysis

3. In the last lecture, we talked about • Introduction • Model: Vicsek model

4. Multi-Agent System (MAS) Autonomy: capable ofautonomous action Interactions: capable of interacting with other agents • MAS • Many agents • Local interactions between agents • Collective behavior in the population level • More is different.---Philp Anderson, 1972 • e.g., small-world, swarm intelligence, panic, phase transition, coordination, synchronization, consensus, clustering, aggregation, …… Examples: Physical systems Biological systems Social and economic systems Engineering systems … …

5. r Alignment:steer towards the average heading of neighbors A bird’s neighborhood : heading of agent i v: the constant speed of birds r: radius of neighborhood Vicsek Model(T. Vicsek et al. , PRL, 1995) http://angel.elte.hu/~vicsek/ xi(t) : position of agent i in the plane

6. Neighbors: • Heading: r A bird’s neighborhood • Synchronization: There exists θ, such that • Position: Alignment: steer towards the average heading of neighbors Vicsek Model http://angel.elte.hu/~vicsek/

7. In the last lecture, we talked about • Introduction • Model • Theoretical analysis • Concluding remarks

8. The Linearized Vicsek Model A. Jadbabaie , J. Lin, and S. Morse, IEEE Trans. Auto. Control, 2003.

9. Theorem 2(Jadbabaie et al. , 2003) Joint connectivity of the neighbor graphs on each time interval [th, (t+1)h] with h >0 Synchronization of the linearized Vicsek model Related result: J.N.Tsitsiklis, et al., IEEE TAC, 1984

10. Random Framework Random initial states: 1) The initial positions of all agents are uniformly and independently distributed in the unit square; 2) The initial headings of all agents are uniformly and independently distributed in [-+ε, -ε] with ε∈(0, ). The initial headings and positions are independent.

11. Theorem 7 High Density Implies Synchronization For any given system parameters and when the number of agnets n is large, the Vicsek model will synchronize almost surely. This theorem is consistent with the simulation result.

12. Theorem 8 High density with short distance interaction Let and the velocity satisfy Then for large population, the MAS will synchronize almost surely.

13. Three Categories of Research on Collective Behavior

14. Three Categories of Research on Collective Behavior • Analysis Given the local rules of the agents, what is the collective behavior of the overall system ?(Bottom Up) • Design Given the desired collective behavior, what are the local rules for agents ? (Top Down) • Intervention Given the local rule of the agents, how we intervene the collective behavior? J.Han, M.Li, L.guo, JSSC,2006

15. r A bird’s Neighborhood Alignment: steer towards the average heading of neighbors Simulation Result Example 1:Synchronization Q: Under what conditions such a system can reach consensus?

16. Example 2: Escape Panic Fire, panic Normal, no panic

17. Three Categories of Research on Collective Behaviors • Analysis Given the local rules of the agents, what is the collective behavior of the overall system ?(Bottom Up) • Design Given the desired collective behavior, what are the local rules for agents ? (Top Down) • Intervention Given the local rule of the agents, how we intervene the collective behavior? J.Han, M.Li, L.guo, JSSC,2006

18. Example 1:Formation control How we design the control law of each plane to maintain the form ?

19. Example 2: Swarm Intelligence(Marco Dorigo et al., 2001-2004) www.answers.com/topic/s-bot-mobile-robot

20. Example 3:Distributed Control in Boid Model Each agent is described by a double integrator (Newton's second law of motion ): where xi, viand uirepresent the position, velocity and the control input of the agent i. Goal: 1) Avoid collision 2) Alignment 3) Cohension What information can be used to design the controller? The position and velocity of neighbors R. Olfati-Saber, IEEE Trans. Auto. Control ,2006.

21. Algorithm Controller design: where A=[aij(q)] is the adjacency matrix, (·) is the action function, isσ-norm, and Neighbor graph • Theorem 1: • If the neighbor graphs are connected at each time instant. Then • The group will form cohesion. • All agents asymptotically move with the same velocity. • No interagent collisions occur.

22. Three Categories of Research on Collective Behaviors • Analysis Given the local rules of the agents, what is the collective behavior of the overall system ?(Bottom Up) • Design Given the desired collective behavior, what are the local rules for agents ? (Top Down) • Intervention Given the local rule of the agents, how we intervene the collective behavior? J.Han, M.Li, L.guo, JSSC,2006

23. Intervention Example 1:Can we guide the birds’ flight if we know how they fly ?

24. Example 2:Leadership by Numbers Couzin, et al., Nature, Vol. 433, 2005 The larger the group is, the smaller the leaders are needed.

25. Example 3:CockroachJ.Halloy, et al., Science, November 2007

26. III. InterventionGiven the local rule of the agents, how we intervene the collective behavior? • The current control theory can not be applied directly, because • It is a many-body self-organized system. • The purpose of control aims to collective behavior. • Not allowed to change the local rules of the existing agents; • Distributed Control: special task of formation, • Pinning Control: Networked system, imposed controllers on selected nodes

27. Intervention Via Soft Control

28. Soft Control • The multi-agent system: • Many agents • Each agent follows the local rules Autonomous, distributed • Agents are connected, the local effect will affect the whole. From Jing Han’s PPT

29. u(t) y(t) Soft Control an associate of a person selling goods or services or a political group, who pretends no association to the seller/group and assumes the air of an enthusiastic customer. • The “Control”: • No global parameter to adjust • Not to change the local rule of the existing agents; • Put a few “shill” agents to guide (seduce) • Shill: is controlled by us, not following the local rules, is treated as an ordinary agent by other ordinary agents • The power of shill seems limited The ‘control’ is soft and seems weak From Jing Han’s PPT

30. U(t) y(t) Soft Control Key points: • Different from distributed control approach. Intervention to the distributed system • Not to change the local rule of the existing agents • Add one (or a few) special agent – called “shill” based on the system state information, to intervene the collective behavior; • The “ shill” is controlled by us, but is treated as an ordinary agent by all other agents. • Shill is not leader, not leader-follower type. • Feedback intervention by shill(s). This page is very important! From Jing Han’s PPT

31. There Are Lots of Questions … • What is the purpose/task of control here? • Synchronization/consensus • Group connected / Dissolve a group • Turning (Minimal Circling) • Lead to a destination (in a shortest time) • Avoid hitting an object • Tracking • … • In what degree we can control the shill? (heading, position, speed, …) • How much information the shill can observe ? (positions, headings, …) • … From Jing Han’s PPT

32. A Case Study • Problem statement: • System:A group of n agents with initial headings i(0)[0, ); • Goal: all agents move to the direction of  eventually. • Soft control: Design oneshill agent based on the agents’ state information. • Assumptions: • The local rule about the ordinary agents is known • The position x0(t) and heading 0(t) of the spy can be controlled at any time step t • The state information (headings and positions) of all ordinary agents are observable at any time step From Jing Han’s PPT

33. Neighbors: • Heading: r • Position: A bird’s Neighborhood • Synchronization: There exists θ, such that Alignment: steer towards the average heading of neighbors Vicsek Model http://angel.elte.hu/~vicsek/

34. A Case Study • Problem statement: • System: A group of n agents with initial headings i(0)[0, ); • Goal: all agents move to the direction of  eventually. • Soft control: Design oneshill agent based on the agents’ state information. • Assumptions: • The local rule about the ordinary agents is known • The position x0(t) and heading 0(t) of the shill can be controlled at any time step t • The state information (headings and positions) of all ordinary agents are observable at any time step From Jing Han’s PPT

35. The Control Law u Control the Shill agent From Jing Han’s PPT

36. Control the Shill agent Theorem 4: For any initial headings and positionsi(0)[0, ), xi(0)R2, 1 i  n, the update ruleand thecontrol law uβwill lead to the asymptoticsynchronizationof the group. It is possible to control the collective behavior of a group of agents by a shill. J.Han, M.Li, L.guo, JSSC,2006

37. Simulation

38. An Alternative Control Law otherwise where Result: The control law ut will also lead to asymptotic synchronization of the group.

39. Simulations Switching between u and ur Control Law u

40. Remarks on Soft Control • It is not just forthe above model Can be applied to other MAS ,e.g., • Panic in Crowd • Evolution of Language • Multi-player Game • …… • “Add the special agent(s)” is just one wayShould be other ways for different systems: • Remove agents • Put obstacle • … … We need a theory for Soft Control ! From Jing Han’s PPT

41. Intervention Via Leader-Follower Model (LFM)

42. Example 1:Leadership by Numbers Couzin, et al., Nature, Vol. 433, 2005 The larger the group is, the smaller the leaders are needed.

43. Leader-Follower Model Problem statement: • System: A group of n agents; • Goal: All agents move with the expected direction eventually. • Intervention by leaders: Add some information agents-called “leaders”, which move with the expected direction.

44. Ordinary agents Information agents Leader-Follower Model • Key points: • Not to change the local rule of the existing agents. • Add some (usually not very few) “information” agents – called “leaders”, to control or intervene the MAS; But the existing agents treated them as ordinary agents. • The proportion of the leaders is controlled by us (If the number of leaders is small, then connectivity may not be guaranteed). • Open-loop intervention by leaders.

45. Leader agents (labeled by ): Heading: Position: Mathematical Model Ordinary agents (labeled by 1,2,…,n): Neighbors: Position: Heading:

46. Simulation Example N=1000

47. Q: How many leaders are required for consensus/synchronization?

48. Assumption on the initial states Random Framework 1) The initial positions of all agents are independently and uniformly distributed in the unit square. 2) The initial headings of the agents are uniformly and independently distributed in [-π, π), and the initial headings of the leaders are. The headings and the positions are mutually independent.

49. If i ~ j Adjacency matrix: Otherwise Degree: Degree matrix: Weighted adjacency matrix: Weighted degree matrix: Leader degree matrix: Average matrix: Weighted average matrix: Some Notations

50. Some Notations (cont.) Laplacian : L(0)=D(0) – A(0) “Normalized Laplacian” : Spectrum : “Spectral gap”: where