RoboCup: A Case Study in Multiagent System

1. Vahid Mokhtari RoboCup: A Case Study in Multiagent System

2. Trends 1 What is an Agent? 2 Multiagent System 3 3 Case study in RoboCup 4 4 Contents

3. Trends in History of Computing

4. From Programming Perspective

5. What is an Agent?

6. Agent and Environment

7. Environment • Accessible vs. Inaccessible - can the agent “see” everything? • Deterministic vs. Non-deterministic - do actions have guaranteed effect? • Static vs. Dynamic - does the environment change on its own? • Discrete vs. Continuous - is the number of actions and percepts finite?

8. What is an Intelligent Agent?

9. Examples of Intelligent Agents • Assistant agent in MS Office • Trading agents • Web spiders • Computer viruses • Characters in computer games

10. Agents vs. Objects

11. Agent Oriented Architectures

12. Agent Types (1)

13. Agent Types (2)

14. Multiagent System (MAS)

15. The Fully General Multiagent System

16. Why MAS? • Some domains require it • Parallelism • Robustness • Scalability • Simpler programming • To study intelligence • Geographic distribution • Cost effectiveness

17. MAS Research Area • Distributed Computing: Processors share data, but not control. Focus on low-levelparallelization, synchronization. • Distributed AI: Control as well as data is distributed. Focus on problemsolving, communication, and coordination. • Distributed Problem Solving (DPS): Task decomposition and/or solution synthesis. • Multiagent Systems (MAS): Behavior coordination or behavior management.

18. MAS Taxonomy

19. Issues in Building MAS

20. Homogeneous Non-Communicating Multiagent Systems • Several different agents with identicalstructure (sensors, effectors, domain knowledge, and decision functions). • Different sensor input and effectors output. • Situated differently in the environment and they make their own decisions regarding which actions to take.

21. Heterogeneous Non-Communicating Multiagent Systems • Agents are situated differently in the environment • Different sensory inputs and different actions

22. Homogeneous Communicating Multiagent Systems • Agents are identical that they are situated differently in the environment • Agents can communicate together directly

23. Heterogeneous Communicating Multiagent Systems • Different sensory data, goals, actions, and domain knowledge

24. Learning Opportunities

25. Importance of MAS • Research in “Distributed AI” started over 30 years ago, but only in the mid of 1990s has it become a major research trend in AI. • Now the main conference (AAMAS) attracts around 800 submissions (of which 20-25% get accepted) each year. • In addition, there are dozens of smaller workshops and conferences. it’s a large, young and dynamic research community

26. RoboCup Case study in Multiagent System

27. What is RoboCup?

28. Domain Characteristics

29. The Standard Problem

30. RoboCup Soccer • Distributed • Multiagent • Teammates and adversaries domain • Partial world view • Noisy sensors and actuators • Real-time

31. Applied Machine Learning

32. Reinforcement Learning

33. The Agent-Environment Interface • Agent and environment interact at discrete time steps: t=0, 1, 2, … • Agent observes state at step t: st S • Produces action at step t: at A(st) • Gets resulting rewards: rt+1 R • And resulting next step: St+1

34. General Process of RL

35. Action Selection Policies

36. Exploration and Exploitation

37. Subtask of RoboCup Soccer

38. SARSA (State-Action-Reward-State-Action) • SARSA is a learning algorithm in the reinforcement learning area of machine learning. • On-policy learning method, • It learns state-action values (Q values).

39. SARSA Algorithm

40. Mapping Keepaway to SARSA

41. Hand-coded Algorithm

42. Result keepers hold the ball for about 8.2 seconds on average keepers hold the ball for about 12 seconds on average

43. Summary

44. References

45. Other Articles

46. Thanks For Your Attention