Distributed Perception Networks

1. Distributed Perception Networks Roald Hopman Brammert Ottens Patrick de Oude Gregor Pavlin Intelligent Autonomous Systems University of Amsterdam

2. Overview • Introduction • Diagnostic Fusion of Heterogeneous Information (Brammert) • Belief Propagation in DPN Structures (Patrick) • DPN Architecture (Roald) • Cougaar Yellow Pages (Hakan) • Demos (Patrick, Gregor, Hakan) • Sensory Infrastructure (Peter)

3. Introduction: Project Status

4. Introduction: Breakthrough Aspects • Model-driven DPN self-configuration principles • No centralized configuration control required • Local ontologies encode possible fusion structures • Distributed Belief Propagation Algorithm • No centralized fusion control required • No complete knowledge of the fusion system required • Asynchronous fusion approach • Basic DPN System Design • Complete separation of SW implementation and fusion functionality

5. Diagnostic Fusion of Heterogeneous Information Brammert Ottens bottens@science.uva.nl

6. Diagnostic Fusion • What kind of problems do we want to solve? • We have large sets of information providers: • Sensors observing some events (temperature, gas, …) • Humans • Databases • We are interested in hidden causes of observable events • We reason about the likelihood of hypotheses corresponding to potential causes of observed symptoms. • We reason about the time after the hidden cause has occurred. • The scores assigned to hypotheses are basis for decision making. • Diagnostic fusion: we use causal models to reason about hidden events • Causal models provide explicit mapping between the symptoms and possible causes

7. Examples • A tube in a plant containing a toxic liquid rupture. The liquid escapes and evaporates in the air creating toxic fumes. These fumes can be detected by sensors and the cause should be identified and located as soon as possible. • Fire spreads in a room and we use measurements from the local smoke detectors, thermometers and cameras, which can detect flames. • A police commander needs information on the “people density” in different locations that he cannot observe directly, in order to plan a local evacuation. We can use reports from other people in the field (e.g. police officers) and sensors (e.g. cameras and microphones) that can detect a crowd. • Identification of significant /dangerous events in complex systems, such as aircraft propulsion, nuclear power plants, etc.

8. Causal Models and Domains • Many fusion problems can be described through causal models (e.g. a pipeline leak in area X causes high gas Y concentration in area X). • Causal models provide rigorous mapping from heterogeneous observations to hypotheses about hidden events. • Models must capture different uncertainties : • Incomplete knowledge of the modeled domain, such as relevant concepts, relations, etc. • Non-deterministic phenomena (e.g. noisy sensor measurements) • Partial observability

9. Simple Fusion Models • Fusion Context • We reason about EXISTENCE of hidden events => Concepts are represented through binary nodes (true/false). • We obtain large amounts of independent and heterogeneous pieces of evidence • Very heterogeneous information sources are available (e.g. humans via PDAs, different types of sensors, databases, etc.) • Each fusion model is used for belief updating of a single hypothesis • A significant class of domains can be described through (quasi) static models • Models feature topologies with a significant portion of independent branches

10. Simple Fusion Models (2) • Quasi static variables (green nodes) • Dynamic variables (red nodes)

11. Implications • Quasi static causal models can easily be decomposed and distributed within agent societies. • Cooperative Distributed Problem Solving => Multi Agent Systems. • Simple and efficient inter-agent belief propagation algorithms exist.

12. Belief Propagation in DPN-structure Patrick de Oude poude@science.uva.nl

13. Monolithic Networks

14. Distributed Networks

15. Problems • No centralized knowledge of the information sources and the fusion structure prior to operation. • No centralized fusion control. • Asynchronous sequences of evidence. • Classical belief propagation algorithms cannot be applied for the inter-agent belief propagation without knowing the complete BN topology and synchronization.

16. Features • Simple, but useful for a significant class of problems • No centralized knowledge of the BN structure required • Consistent belief updating with asynchronous evidence => no centralized fusion control required • Limited to BNs with significant portion of independent branches • Limited DPN topology • Simple Tree Topology (Maximum one parent) • Arbitrarily complex structures within agents • Standard propagation algorithms are used for the reasoning in each agent.

17. Inter-Agent Messages • Partial fusion results are treated as soft evidence in higher level agents. • Partial fusion results correspond to factors in Bayesian conditioning equations. • Asynchronous Message passing.

18. Propagation Algorithm • Two types of agents • Agents with dynamic variables • Agents with quasi-static variables • The type of agent dictates the way agents are updated • Fusion Results Independent of the Updating Sequence • No central Fusion Control

19. Updating with Dynamic Nodes • Initially all weights are instantiated uniformly (i.e. 0.5) • If new evidence is received then calculate the new weight by: • Normalize the newly calculated fusion weights • Replace the old saved weight with the new weight • Send wnew(X) to higher level agent

20. Updating with Quasi-Static Nodes • Initially all weights are instantiated uniformly (i.e. 0.5) • If soft evidence is received at C or D save this information at that leaf node • Instantiate all leaf nodes with current values then run arbitrary belief propagation algorithm • Normalize results and save at node B • Send calculated result to higher level agent or if current agent is top level agent output results to end-user

21. Complex Model

22. DPN Architecture Roald Hopman rhopman@science.uva.nl

23. DPN Features • Graceful degradation • Agent Migration • Simple specification of fusion capabilities/properties • World model-driven Design: distributed partial world models • Runtime self-configuration: Network Assembly top-down • Asynchronous information fusion: Information flow bottom-up

24. SW Layer DPN HW Layer DPN Architecture • Hardware Layer: Network of Sensor Suites and Computers dispersed through arbitrarily large areas. • Software Layer: Network of agents implements the functionality; i.e. fusion of very heterogeneous data and information, accuracy monitoring, etc.

25. Types of Agents • Fusion Agent • Higher level agents that will fuse information from lower levels • Sensor Agent • Getting values from sensors and do data interpretation • Yellow Pages Agent • Register services of all agents

26. Agent Components • Blackboard • Plugins: • Communication Engine • Handles all internal and external communication • Acquaintances Model: contains agent-ids that need services from this agent • Reasoning Engine • Responsible for all the reasoning • Reasons with arbitrary local BNs (Junction Tree-based approach) • Local World Models • Arbitrary Bayesian Networks • XML format.

27. External Messages • Yellow Pages – register all services of agents • Service Discovery • Set up contract: Contract Net Protocol • Send the requested information findServiceRequest findServiceRequestResponse CallForProposal Bid FusionContract fusionResult

28. Internal Messages • getFusionResult (1) • fusionRequest (2) • giveFusionResult (3) • sendFusionResult (4)

29. World Model-Driven Design • All fusion and sensor agents use the same basic SW components. • Functionality of the fusion system is determined through partial world models, which are provided to the different agents. • World models (BNs) can be designed with the help of common graphical editors such as JavaBayes => domain experts can easily specify the functionality without programming. • Implications • Modification of functionality does not require recompilation => avoid SW faults • New agents can join a complex DPN system at runtime. • Domain experts can configure a DPN system without SW experts

30. Self-Configuration

31. Scenario Test Case • Purpose • Demonstrate the ability to find relevant sources of information • Fusion of heterogeneous information • Preconditions • Police PDAs in area X register at DPN yellow pages upon entering that area. • Camera and Microphone DPN sensor agents registered at DPN YP. • Information Gateway available. • Postconditions • Relevant information sources are found and integrated into DPN • Fusion results are delivered

32. Scenario Test Case Info Gateway YP

33. Scenario Test Case

34. Future Work • Clean up current code. • Graphical representation. • Model Checking (verify that there are no cycles in distributed BNs). • Implement Identification of potentially unreliable model components and information providers. • Integration with PDAs. • Integration with SNE. • Implementation of DPN-Sensor interfaces. • Improved information provider discovery and Yellow Pages updating. • Enhanced belief propagation algorithm (allow limited loops in DPN structures). • Semi-automated decomposition of monolithic BNs.