Dr. Dana Ulery, Program Manager

1. Context Powered BORG Fusion Research Dr. Dana Ulery, Program Manager

2. Outline • BORG Challenge & Overall Goal • Situation Semantics • Knowledge as a Situated Process • BORG Approach & Project Objectives • Conclusion ARL Knowledge Fusion COE Program

3. BORG: Battlefield Organic Robotic Grid Technology centerpiece for the ARL Knowledge Fusion COE Program BORG R&D environment implemented at KIMCOE at MSU Technology partners include Clark Atlanta, U of MD, Monmouth, PNNL, & U of VA. ARL Knowledge Fusion COE Program

4. Situational Understanding Adversarial Reasoning Battle Management Action Aids Lifting the Fog of Terrorism & War • Quick Access to • Actionable Information • Multiple, Multimedia Information Sources ENVIRONMENT • Mobile • High Stress • Dynamic • Uncertain • Real-time ARL Knowledge Fusion COE Program

5. Real-Time Attack Questions • Am I under attack? • What is the nature of the attack? (who, where, what) • What is the impact? (damage, response) • What are they trying to do? • What is their next action? • What action should I take now? • What action should I take longer term? Currently, answering these questions is a major challenge at all levels of command. ARL Knowledge Fusion COE Program

6. Real-Time Attack Answers • Sharp and focused • Quickly understood • Critical information available “at a glance” • Fused from multiple, global & local key sources • No information overload can be tolerated • In-depth data on demand only The BORG program strives to prove that Attack Answer tools can be automated using fusion methodologies & techniques. ARL Knowledge Fusion COE Program

7. Linguistic Situation Semantics • Situation Semantics: a branch of linguistic semantics • Concerned with how we understand each other in natural conversation • Concerned with the way we use social knowledge to communicate Early Situation Semantics (Barwise & Perry, 1983) & other mathematically-based semantic theories: • Assume words, phrases, sentences have unique or canonical meanings • Relate situations to objects: for a given situation s and an object a, a is a • constituent of s or it is not. Deficiencies of the Barwise-Perry Model A. Is there any water in the refrigerator? B1. Yes – the refrigerator coils have a lot of condensation built up. B2. No – but there’s some lemonade. B3. Yes – there’s a bottle of water with lemon in it to cover up the taste of rust from the pipes. Every speech act occurs in a context, with a background shared by speaker and hearer (Winograd, 1985; Winograd & Flores, 1986) ARL Knowledge Fusion COE Program

8. Modern Theory of Meaning Modern Situation Theory (Keith Devlin, 1990): 2 Notions of Meaning abstract meaning: Answers the question “What does this sentence s mean in general” where s is taken out of context. meaning-in-use: Answers the question “What does this sentence s mean as it is being used in this instance?” where s is in a particular context. Situations are not “set of objects” or “regions of space-time”, but fundamental entities in their own right. This is a significant shift from the linguists’ classical structural approach to language to the social scientists’ and psychologists’ procedural approach. The sociolinquist observes that language users rely on common cultural experiences, shared knowledge of language, and various interpretive strategies to “understand” how to interpret what is said. ARL Knowledge Fusion COE Program

9. Knowledge as a Situated Process • Situated Process: recent extension of Knowledge Based Systems technologies • in KM, AI, CS, organizational behavior, psychology, cognitive and social science. • Knowing is viewed as a community process that people collectively do, • rather than a product an individual possesses. • Emphasizes the role of interaction and context in intelligent behavior. • Reinterprets the technologies of symbolic representation (Suchman, 1987; • Lave, 1988, 1993): • denies classical AI assumption of pre-existing internal representation • representation is moved to the cultural level, coordinated and constrained by interacting neural and social structures. • Organizational behavior cannot be reduced to a function of autonomous decisions (Gammack & Stephens, 1997). ARL Knowledge Fusion COE Program

10. Implications for Knowledge Fusion Current State: Information Fusion primarily combines research from: • the engineering sciences dealing with sensors & complex dynamical systems • the computer sciences, especially AI Future State: Knowledge Fusion must have stronger links to: • linguistics • cognitive science • psychology • social sciences ARL Knowledge Fusion COE Program

11. BORG Approach to Knowledge Fusion Considers Knowledge Fusion as a Situated Army Process and considers the prospects for developing Situational Understanding based on this view and Situation Semantics theory. • Situation Semantics and Situated Process are considered in project & system design. • Projects are linked by Army context and through systems integration. • An early BORG project defined and developed two Army contexts for BORG technology projects. • significant tool for defining and studying interactions and relationships • strong aid for mapping research to problem ARL Knowledge Fusion COE Program

12. BORG Project Objectives • Create an automated environment for researching Situational Understanding in an Army context. • Show that real-time Attack Answer tools can be automated using fusion technologies. • Include modern Situation Theory and Situated Process concepts to offer new perspectives in fusion research for Army applications. • Successfully transition Attack Answer tools to Army demonstrations. ARL Knowledge Fusion COE Program

13. BORG Research Areas of Special Interest • Internet Technologies – a socially based shared memory • Ontologies – advances to accommodate situation semantics & processes • Multimedia Interfaces – diagram interpretation & generation • Information Understanding – evolved and understood in a context • Stories – narratives that structure histories to fit current situations ARL Knowledge Fusion COE Program

14. BORG Army Context Project • Wargaming methodology was adapted to focus on fusion research technology issues. • 2 detailed scenarios • developed by US intelligence experts • battlefield • terrorist • Project is complete & documented. ARL Knowledge Fusion COE Program

15. Scenario: Civil War and Peninsular Conflict • Kim Jong-il dies suddenly. • Over next several months: • Small group of more moderate leaders announce their assumption of control of the central government. • Moderate’s View: Detrimental to survival of the country to continue hardened stance against the West and South. • Moderate’s Intentions: Open a more active dialogue. • Status: Unable to gain complete control or vows of loyalty from all Party members loyal to Kim Jong-il’s legacy, and ominously, from some of the more hard-line elements within the military. • Hardliners oppose moderates. • See their positions of relative privilege and power being placed in jeopardy. • Supporters: Include the Navy Chief of Staff, who has gained the backing of several senior officers in Pyongyang military headquarters, and two corps commanders, one near the DMZ and one north of Pyongyang. ARL Knowledge Fusion COE Program

16. NATIONWIDE SECURITY ALERT • 3 days ago: FBI issued chemical, biological, or radiological (CBR) attack alert. • Attack is imminent. • Threat information obtained from Guantanamo Bay detainee. • Cousin is local “middleman” for catastrophic event planning • Incident in Northeast US and fairly soon. “MIDDLEMAN” ARL Knowledge Fusion COE Program

17. Conclusion The BORG program strives to prove that Attack Answer tools can be automated using fusion methodologies& techniques. BORG considers Knowledge Fusion as a Situated Army Process and considers the prospects for developing Situational Understanding based on this view and Situation Semantics theory. Multiple BORG Projects are in progress. ARL Knowledge Fusion COE Program

18. References Barwise, J. and Perry, J. Situations and Attitudes, Bradford Books, MIT Press (1983). Devlin, K. Extending Barwise and Perry’s Relational Theory of Meaning, presented at the Jon Barwise Memorial Conference, Stanford University, 2003. http://www.cs.tcd.ie/Tim.Fernando/B/DevlinPaper.pdf Gammack, J. G. and Stephens, R. A. Knowledge Acquisition as a Situated Process: Implications for Information Systems Design, in Proceeding 3rd Australian Knowledge AcquisitionWorkshop at 10th AJCAI, 1997. http://www.cse.unsw.EDU.AU/~timm/pub/aka97/papers/ Winograd, T. Moving the Semantic Fulcrum, Linguistics and Philosophy 8(1), 1985. Winograd, T. and Flores, F. Understanding Computers and Cognition, Ablex Publishing, 1986. ARL Knowledge Fusion COE Program