Autonomous Learning Agents for Decentralised Data and Information Networks (ALADDIN)

1. RISE 2008, January 7−8, Spain Autonomous Learning Agents for Decentralised Data and Information Networks (ALADDIN) www.aladdinproject.org Dr. David Nicholson BAE Systems Bristol, UK David.Nicholson2@baesystems.com

2. Facts & Figures • £5.5M funding (plus £1M in kind) • 2/3 BAE Systems; 1/3 EPSRC • Started 1st October 2005 • Duration 5 years (3 + 2) • Funded manpower: • Research Fellows: 600 person months • Programmers: 100 person months • 1 lecturer: Alex Rogers • 13 PhD students

3. The Team School of Electronics and Computer Science (Lead: Prof. Nick Jennings, Director) Dept. Electrical and Electronic Engineering (Lead: Prof. Erol Gelenbe) & Dept. Mathematics (Lead: Prof. David Hand) Department of Engineering Science (Lead: Prof. Stephen Roberts) Department of Mathematics (Lead: Dr. David Leslie) Advanced Technology Centre Integrated System Technologies (Programme management, research, exploitation)

4. Overall Aims • Develop techniques, methods and architectures for modelling, designing and building decentralised systems that can bring together information from variety of heterogeneous sources in order to take informed action • Take total systems view on information and knowledge fusion and consider feedback between sensing, decision making and acting in such a system • Achieve these objectives in environments in which: • Control is distributed. • Uncertainty, ambiguity, and bias are endemic. • Multiple (self-interested) stakeholders with different aims and objectives are present. • Resources are limited and continually vary during system’s operation. • Timeliness of action is important • Demonstrate applicability in domain of disaster management

5. A2 A1 A3 A6 A4 A5 Conceptual Underpinning multi-agent processes: interacting with each other and their environment

6. Technical Approach • Building Individual Agents • Information fusion • Bayesian inference • Decision theory • Reinforcement learning • Building Multi-Agent Systems • Multi-agent systems • Game theory/Mechanism design • Mathematical modelling of collective behaviour Principled combination Total systems view Demonstrable applicability

7. Research Themes • Individual Actors • Design of actors that can perform effectively in dynamic uncertain and multi-actor environments • Multiple Actors • Way in which individual actors can interact in flexible ways to achieve individual and collective goals • Architectures • Development of efficient and effective system architectures • Applications • Development of disaster management demonstrators

8. Applications

9. Urban rescue Situational awareness Information Agent Demo Vignettes Building evacuation

10. Situational Awareness • Information agents to provide real-time environmental information around a disaster area: • Weather, temperature, visibility, noise, gas detection etc. • Exploit sensors already in the environment. • Built into an urban environment. • Scattered around a disaster area. • Access information through current internet protocols. Information Agent on Tablet

11. Solent Weather Sensors • Analogue for future deployed sensors. • Wind speed, direction, air temperature, air pressure, tide height etc. • 4 sensors in the Solent. • Updates every 5 minutes. • Data accessed through HTTP.

12. Current Implementation • Data deployed on sensor websites in RDF format. • Information agent collects data via HTTP - parses, stores and queries sensor reading using standard semantic web technology (Jena). • Backgound mapping data collected in real-time from GoogleMaps. <sit:Reading rdf:about="&sit;sotonmet/winddir/reading/20060704113000" rdfs:value="235" sit:datetime="2006-07-04T11:30:00"> <sit:sensor rdf:resource="&sit;sotonmet/winddir"/> <sit:unit rdf:resource="&sit;degrees"/> </sit:Reading> Information Agent

13. Demo Screenshot

14. RoboCup Rescue • Initiated in the late 90s following the Hanshin/Awaji earthquake • Short term: support decision making in multi-agency response • Long term: realise human-robot response • Open-source – universities, individuals • Yearly competition • Agent strategies • Simulators / Infrastructure

15. RoboCup Rescue Simulator • Models the aftermath of an earthquake • Simulators – simulate civilian behaviour, fires, blockades. building collapse, traffic • Agents – ambulance / police / fire • Agents have to rescue civilians / clear blockades / put out fires

16. Platoon Agents • Fire Brigades • put out fires • have limited water tank capacity • can collaborate to extinguish more quickly • Ambulances • dig out civilians who are buried under rubble (takes a number of cycles)‏ • carry victims to a refuge • can collaborate to dig out more quickly • Police Force • can remove obstructions on roads

17. Extended Simulator Simulators model other actors (civilians) and evolving world state. Viewers can access global world view from kernel. Fire Viewer Kernel Simulators Civilian Blockade Omni-agent connects with kernel as a viewer. Omni-agent Maintains compatibility with new versions of kernel, viewers and simulators. Can completely configure and tailor sensing and communication. Can extend and customise rescue scenarios. Omni-agent launches agents and all actions, sensing and communication passes through it. Actions forwarded to the kernel. Agents Agents Agents Agents Agents Agents

18. Extended Capability Autorun, Strategies and Map Generation Automatically launch strategies. Run repeatable experiments. Store statistics from different runs. Communication System Implement completely configurable communication network. Broadcast, peer-to-peer. Lossy communication and corrupted messages. Bandwidth constraints and communication black-holes. Agent Sensing Abilities Noisy sensing, different agent capabilities. Clustering Automatic map clustering tools for implementing hierarchical strategies.

19. Demo Screenshots

20. Building Evacuation

21. Architectures

22. Disaster Management Pyramid Disaster Management Enterprise Disaster Management Services Response Agencies Response Resources Non-Sentient Elements

23. Strategic Planning • global goals • constraints information • Multi-Agent Decision-Making • local goals • resources • coordination RT2 information • Info Fusion • context • reasoning • assessment information and decisions reports and estimates information RT1 • Data Fusion • alignment • estimation • combination information • Data Collection • uncertain • ambiguous • incomplete

24. Decentralisation storage Info source peer-to peer communication network Info source Info source inference and decision cycle

25. Research Focus: How to conflate functions and manage interactions to achieve global goals in a dynamic and uncertain environment ? • Strategic Planning • global goals • constraints information • Multi-Agent Decision-Making • local goals • resources • coordination RT2 information • Info Fusion • context • reasoning • assessment information and decisions centralised reports and estimates information RT1 • Data Fusion • alignment • estimation • combination intra-function decentralised inter-function decentralised information • Data Collection • uncertain • ambiguous • incomplete fully decentralised

26. Individual Actors

27. Aims • Model the world view of an individual agent in the face of: • incomplete data • delayed data • bandwidth restrictions which then serves as basis on which: • to choose optimal action from a finite set of actions • Taxonomy, experiments, evaluation of methods from sensors or other agents

28. Gaussian Processes missing correlated delayed active data selection.

29. Exploit Correlation

30. Faulty Sensors • Faulty sensors • 2 out of 6 sensors output temperatures 10% too high • Faulty observations can spread, reinforce and thus create havoc in decentralised estimation networks ALADDIN fusion algorithm basic fusion algorithm

31. Multiple Actors

32. Optimisation Problem • NL number of incidents • Ij number injured at incident j • NU number of emergency units • Ci capacity of emergency unit i • Tij response time for unit i to incident j 3 4 Ij=5 5 Ci = 3 j i Tij

33. Auctions? • Desirable Properties • Efficient means of dynamically allocating limited resources • in the presence of multiple stakeholders • with minimal communication requirement • Auctions employed in a number of places: • Resource Allocation: Spectrum, Mining Tracts, Rare Items • Industrial Procurement • Novel applications • Format • Agent submit bids • Auctioneer calculates winners (allocation) and payment

34. Distributed Auctions

35. Agent 1 Agent 2 Task1 Agent 3 Agent 4 Coalition 3 Agent 5 Coalition formation process Task 2 Coalition 1 Agent 6 Coalition 2 Coalition Formation in Multi-Agent System

36. The guarantee provided by our algorithm on the quality of its solutions (21 agents) Algorithm Portion of the space searched

37. Achievements • Good level of collaboration • 2 project workshops • Winner of 1st and 2nd International Competitions on Agent Trust and Reputation (2006, 2007) • Winner of Robocup Rescue Infrastructure Competition (2007) • Winner of International Trading Agent Competition on Market Design (2007). • 25 publications/submissions in journals, conferences and workshops • Including number of multi-institution ones • 60 deliverables produced

38. Outreach • Number of keynotes at international conferences • Organised Workshop on Game Theory and Probabilistic Inference at NIPS (Rezek & Rogers). • Organised 1st International Workshop on Agent Technology for Disaster Management at AAMAS-2006 (Jennings & Ramchurn) • Organised special track on Autonomous Agents for Data and Information Fusion at Fusion-2006 conference (Johnston & Jennings) • Organised agentcities workshop on Advanced Technologies for Disaster Management (Ramchurn & Jennings) • Invited by EPSRC to present Aladdin at International Review of ICT (Jennings, Rogers & Ramchurn) • Organised 1st International Workshop on Agent-Based Sensor Networks at AAMAS-2007 (Rogers & Dash). • Participated in technical committee of Robocup Rescue 07 (Ramchurn)

39. Conclusions • Exciting & challenging research agenda • Bringing together number of disciplines to produce end-to-end solutions to complex problems • Fundamental research in basic techniques for individual and multiple actor systems • Systems research on how to combine distinct components • Demonstrations of technologies in disaster management scenarios