Fusion Technology Development for Urban/Asymmetric Warfare: Deductive and Inductive Approaches

1. Fusion Technology Development for Urban/Asymmetric Warfare: Deductive and Inductive Approaches Rakesh Nagi Department of Industrial Engineering and Center for Multisource Information Fusion (CMIF) University at Buffalo, State University of New York at Buffalo nagi@buffalo.edu November 30, 2005

2. Introduction: Problem Space Overview • Representative modern-day asymmetric problems: • Urban warfare/Critical Infrastructure Attacks • Improvised Explosion Devices (IEDs) • Dirty bomb (Radiological weapon) http://www.defendamerica.mil/ *http://icasualties.org/oif/IED.aspx

3. Introduction: Problem Space Overview • An unexploded IED sits on the hood of a car as American soldiers investigate at the site of an April attack on a U.S. convoy north of Baghdad. (By Khalid Mohammed -- Associated Press)

4. Basic Questions • How do we understand and classify these asymmetric threat? • How do we develop Knowledge/Information Fusion Technology to assist the analyst? • How do we test the efficacy of the Fusion Technology to thwart intended attacks?

5. Outline • Introduction: Problem Space Overview • Fusion Research Approach • Problem and Domain Understanding • Hybrid Deductive + Inductive 1. Problem and Domain Research • Part 1.A: Scenario/Use Case • Part 1.B: Operational Net Assessment • Part 1.C: Ontology 2. Deductive or Model-based research • Part 2.A: Information Fusion Engine for Real-time Decision Making (INFERD) • Part 2.B: Graph Matching 3. Inductive or Data Mining and Knowledge Discovery research • Part 3.A: Semantic Networks (SNePS) • Part 3.B: Graph Data Mining 4. Integrated Software Architecture • Discussion

6. Taxonomy of Asymmetric Warfare Problems Ref: “Asymmetric Warfare: A Conventional Classification Approach to Understanding the Unconventional” CUBRC Report July 2005.

7. High Uncertainty Deductive + Inductive High Dimensionality High Risk Introduction: Problem Space Overview • General Characteristics: • Knowledge/Model-based approach viable for parts but not all aspects of these problems • Observability spotty, ambiguous • Extensive data base requirements • High collateral damage environments

8. Hybrid deductive and inductive approach due to high uncertainty in these environments Integrated Software System Fusion Research Approach Realistic and problem-oriented approach Research Approach Problem and Domain Understanding Deductive SA/IA Model/Algorithmic Inductive SA/IA Information Fusion Engine for Real-time Decision Making (INFERD) SNePS KRR System Scalable Scenario Evidential Framework For ONA Graph Matching Graph Data Mining Ontology Adaptive: Hybrid Inferencing Insightful: Multi-perspective Interoperable: Onto-grounded Multi-perspective and insightful “gaming” approach Interoperable and formally designed world views

9. Deductive Approach Integrated Simulation Software Environment Domain Study Historical Cases SMEs Scenario Development Ontology Development Forensic Methods: e.g., Graph Matching SA/IA FUSION Technology

10. Inductive Approach Knowledge Reasoning Integrated Simulation Software Environment Knowledge Reasoning and Representation System Domain Study Historical Cases SMEs Ontology translation to KRR system Scenario Development Text Mining for Lexicon Generation Ontology Development Data Mining Forensic Methods: e.g., Graph Matching SA/IA FUSION Technology Integrated Software System

11. Integrated Software System Fusion Research Approach Research Approach Problem and Domain Understanding Deductive SA/IA Model/Algorithmic Inductive SA/IA Information Fusion Engine for Real-time Decision Making (INFERD) SNePS KRR System Scalable Scenario Evidential Framework For ONA Graph Matching Graph Data Mining Ontology

12. Scenario/Use Case Motivation for Scenario Development • To satisfy Use Case Requirements: • Use Case Requirements • Representative of modern-day military and/or security threat • Scaleable to other “genre’s” of the Scenario • To motivate fusion technology development • Sufficiently complex to: • Further test and develop existing fusion capability • Motivate new, innovative fusion capabilities • To provide basis for demonstration of ONA process • Enough observational basis to allow multi-perspective fusion-based inferencing *Credits: Dr. James Llinas and Justin Yates, CUBRC/CMIF

13. Scenario/Use Case Research approach • Survey classes of “typical” modern-day threats • Select a genre that is representative of a sufficiently wide class of problems • Explore authoritative operational literature so Use Case script is defendable/plausible • Frame script representation for both operational understanding and use by fusion processes • Cannot overemphasize the role of SMEs

14. Scenario/Use Case: Example • Genre: “Coordinated small unit attack on Critical Infrastructure point target” • Meta-genre: “Opportunity-constrained Threat” • Phases: • Reconnaissance • Subtle, covert data gathering by Red • Intell reconn by Blue • Pre-Mission • Solidification of Red plans to point of initial positioning • Mission • Execution of point attack • Cyberattack included • Movement to contact • Execute diversion • Attack task execution • Immediate post-attack actions • Pursuit • Red dispersal, movement to escape • Blue coordinated pursuit

15. The Nature of Critical Infrastructure Entities* * GAO Report to Congress, “CRITICAL INFRASTRUCTURE PROTECTION”, GAO-05-434, May 2005

16. Addtl Extensibility • Analogous also to Base Defense • US bases on foreign soil—Bases in Theater • See Joint Pub 3-10.1 • Joint Tactics, Techniques, and Procedures for Base Defense

17. Base Defense Threats* Small Unit Ops • * Joint Pub 3-10.1 Joint Tactics, Techniques, and Procedures for Base Defense

18. Scenario/Use Case • Specific Case: Coordinated Insurgent attack on an Infrastructure Facility to extract Fissile Materials for Use in WMD • Urban location • Typical of Critical Infrastructure Facilities • Secure Research Institute (Typical of other Infrastructure-embassies, Govt offices, etc) • Coordinated, Multi-jurisdictional Defense and Pursuit • Facility security staff (private, contractor-type)—Local Police—Natl Intell

19. Cases of Fissile Material Diversions* *from: “International Terrorist Threat to Nuclear Facilities”, Braun, C., et al, Amer. Nuclear Soc Winter 2002 Mtg, Washington DC, Nov 2002

20. Link to Scenario Operation Kharkiv Defense

21. Integrated Software System Fusion Research Approach Research Approach Problem and Domain Understanding Deductive SA/IA Model/Algorithmic Inductive SA/IA Information Fusion Engine for Real-time Decision Making (INFERD) SNePS KRR System Scalable Scenario Evidential Framework For ONA Graph Matching Graph Data Mining Ontology

22. The Operational Net Assessment Concept* Adaptive ISR Sensor Management Embedded MULTI-PERSPECTIVE Gaming Estimated BLUE FUSION PROCESS and Red Blue View COA Blue Self-awareness Route of Blue and Vulnerabilities Estimation Inherent Blue View Nominated BLUE Threat of Red Action Cmdr's ISR or Guidance ASSETS Resource Utilization Blue View Perceived Threat of ( Red View of Blue ) by Red Effects Analysis Blue View Red's Self- of ( Red View of Red ) awareness Contextual Data and Information * Biggie, J., Operational Net Assessment” brfg, JFCOM J9, Nov 2003, http://www.mors.org/meetings/decision_aids/da_pres/Biggie.pdf

23. Biggie, J., Operational Net Assessment” brfg, JFCOM J9, Nov 2003, http://www.mors.org/meetings/decision_aids/da_pres/Biggie.pdf

24. PMESII System Behavior Models Economic/ Infrastructure Social/ Culture • Political “PMESII” Evidence Space Political/ Religious • Military • Economic Model/Activity Interaction • Social Information • Information • Infrastructure Regular Military ONA—Evidential Growth Requirement • Much more Holistic View • Much better Adversarial Insight • Technology Challenges: • -- Increased Combinatorics, hypothesis mgmt • -- Development and integration of large a priori info: Data base mgmt • -- Testing and Validation

25. Integrated Software System Fusion Research Approach Research Approach Problem and Domain Understanding Deductive SA/IA Model/Algorithmic Inductive SA/IA Information Fusion Engine for Real-time Decision Making (INFERD) SNePS KRR System Scalable Scenario Evidential Framework For ONA Graph Matching Graph Data Mining Ontology

26. What Is Ontology? Ontology • Philosophy • Theory-based: • Formal Ontology • Logical Theory • Doctrine of Hylomorphism • Mind-Body Problem […] • Information Sciences • Application-based: • SUO • Species of OWL • IDEF5 • Ontolingua • Protégé 2000 […] Both Are Needed • Objs (X, Y,…) • Attributes (p, q,…) • Relations • Events Ontology Epistemology Epistemic states: X’s belief in Y (KR)

27. Ontology Development Methodology 1. DEVELOP A DOMAIN LEXICON 0. Utilize Text-Mining Software for Term Extraction The lexicon should contain a sufficiently large sample of terms which represent those items found within a given domain. 2. DEVELOP UPPER-ONTOLOGY CATEGORIES Mine useful terms from electronic documents for use in constructing the initial domain lexicon. Upper-level categories contain highly abstract metaphysical items. 3. INTEGRATE DOMAIN-SPECIFIC CATEGORIES Domain-specific categories contain those (L1/L2 Fusion) from within a specific spatio-temporal domain. 4. Merge Ontology With Cognitive Work Analysis CWA’s provide user-centric and functionalistic domain info. 5. FORMALIZE ONTOLOGICAL RELATIONS Map relations between upper- and lower-level ontological items. 6. Integrate Ontology with KR Tool for Reasoning Over Ontological Relations 7. CODE INTO COMPUTATIONAL LANGUAGE Develop a computational language which captures necessary relations. 8. DEVELOP METHOD FOR EVALUATION Reason over items and relations within the ontology to aid in improved discovery of relations. Test ontology to assure its consistency and completeness. This process assures the ontology remains relevant for a variety of applications. *Eric Little, CUBRC/CMIF

28. Lexical Terms Of Interest Manual Checking E-Documents DATA/TEXT MINING • nouns • verbs • Etc Methodology (cont.) STEP 1 STEP 0 Domain Lexicon (alphabetized & organized) STEP 2 LEXICON CONSTRUCTION Define Formal Relations Define Formal Relations Upper-Ontology Evaluation Procedure STEP 4 SNAP Ontology (Spatial Items) SPAN Ontology (Temporal Items) Knowledge Representation Tool Reasoning over Ontology Domain Ontology (Organized Domain Lexicon) STEP 7 STEPS 5 & 6 STEP 3

29. Exemplary Skeletal Ontology Model From Domain Terminology Situational Item (Attack on Nuclear Facility including cyber attack) Spatial Items (SNAP) Temporal Items (SPAN) Dependent Item Temporal Region Processual Entity Independent Item Scattered Substance Quality Process Disconnected Times of Reports Damage Facility Infiltration Civil Infrastructure Agent Plan Reconnaissance Facilities Civilian System Updates Mission Building Properties Affected Unaffected Connected Pursuit Size, Materials, etc Non-combatants. Nuclear Facility Instance Cyber Attack Capacity Affected Attack event at a given time Recon Cyber System Facilities Privilege Escalation Security Combatants Police Unaffected Interval Intrusion Friendly Forces Cyber Police Facility Attack event over time Setting Blue Base Transportation Systems Time of Day Road, Bridge, etc. Day of Week Road, Bridge, etc. Time of Year Fluid Temporal Boundary Situation = Spatial + Temporal Components: Must Be Modeled Independently Enemy clustering Growing Shrinking

30. 2. Deductive Research

31. Integrated Software System Fusion Research Approach Research Approach Problem and Domain Understanding Deductive SA/IA Model/Algorithmic Inductive SA/IA Information Fusion Engine for Real-time Decision Making (INFERD) SNePS KRR System Scalable Scenario Evidential Framework For ONA Graph Matching Graph Data Mining Ontology

32. Minimize Apriori Knowledge Target Graph Generation Level 4 Automatic Adaptive Learning Efficient Deployment SME Real-Time SA/IA Visualization Level 2/3 Level 0/1 Completeness Multiple Formats INFERD: Technology Approach Initial Knowledge of Domain and Objectives Sensor Location and Settings Physical/Virtual Domain of Interest Sensors Type 2 INFERD Sensors Type n Sensors Type 1 SIGINT, COMINT, HUMINT sensed data Decision Maker Graph Matching (Batch) Cleansing, Filtering and Homogenizing Data Data Graph Generator *Credits: Dr. Moises Sudit, CUBRC/CMIF

33. INFERD: Why a new approach? Time Occurrences Event Rule Based INFERD Probabilistic Parametric Graph Matching Real-Time Forensics

34. System Architecture: INFERD + Graph Matching INformation Fusion Engine for Real-time Decision-making (INFERD) Truncated Graph Matching Heuristic

35. General Hierarchical Fusion Framework (INFERD) • Graphical representation of the elements that make up a Template Graph (Attack Track) • Each Template Node is composed of a Feature Tree • Each Feature Node is asserted via an L0/1 fusion method on sensory data.

36. INFERD: L2 Computational Technology • Depth of Template Measurement – measures longest path • Breadth of Attack Measurement - measures how much of the entire possible scope of the template has already taken place, • Reliability of Attack Measurement - measures how sure we are that this particular template is actually happening(Information Theory – Shannon)(Generalized Entropy – Tsillas)

37. INFERD: Success in Asymmetric Domains • BDA (Army) • Chem/Bio (DTRA) • IED Detection (CACI) • Urban Warfare (MIT LL, LMCO)

38. INFERD Capability in UW Video shows a U.S. strike on Taliban forces in Afghanistan. Video taken from Predator UAV – used for reconnaissance to provide real-time images for attacks. • Shows people and vehicles moving around • Shows series of buildings including a mosque • Can hear discussions between aircrews and ground controllers concerning targeting • They take special care as to protect the mosque • INFERD is placed over the video using a grid-like overlay to show its ability to represent the situation at hand.

39. Demo: INFERD in UW Situational Assessment

40. Power Mean INFERD and Predator’s Video Grid-box #27, Time interval #141 CF = 0.625 WA 0.75 0.25 0.5 1.0 max min 0.5 WA target structure auto. weap. civilian collateral 0 0 1.0 1.0 0.5 0.25 0.25 people man. weap. vehicle 1.0 0 0

41. Integrated Software System Fusion Research Approach Research Approach Problem and Domain Understanding Deductive SA/IA Model/Algorithmic Inductive SA/IA Information Fusion Engine for Real-time Decision Making (INFERD) SNePS KRR System Scalable Scenario Evidential Framework For ONA Graph Matching Graph Data Mining Ontology

42. Graph Matching: Objectives • Find the “best” match to a template in the Data graph • Attributed Graph Structure G = (V, E, Av, AE) • Where V - the set of nodes; E - the set of arcs; • Av- the set of node attributes; Av- the set of arc attributes.

43. Graph Matching: 1-Hop Neighbor Matching • Algorithm • Step 1: Compute a node score, denoted as Cij , for each node in the template graph to each node in the data graph. • Step 2:Compute the scores, denoted as Wij , for the 1-Hop neighbors of each root node pair. • The score is given by  Cij + (1-) Wij “” is the Score vs. Topology Parameter.

44. Graph Matching: Truncated Greedy Algorithm • k0= 3 • ki = 3 • i= 7 •  = 4

45. 3. Inductive Research

46. Integrated Software System Fusion Research Approach Research Approach Problem and Domain Understanding Deductive SA/IA Model/Algorithmic Inductive SA/IA Information Fusion Engine for Real-time Decision Making (INFERD) SNePS KRR System Scalable Scenario Evidential Framework For ONA Graph Matching Graph Data Mining Ontology

47. What is SNePS? • A Logic- and Network-based Knowledge Representation, Reasoning, and Acting system. • In existence/development for 30 years, with participation by more than 64 people. • (Academic, not commercial, software.) • As expressive as higher-order logic • but designed for commonsense reasoning. • Broad set of logical rules of inference • but incomplete to improve tractability. • Inconsistency detection and Belief Revision • for Multisource Information Fusion. • Efficient, path-based reasoning. • Useful for ontological reasoning and graph-matching. • To make use of non-logic-based computations such as efficient mathematical calculations. • Integrated reasoning/acting for intelligent agents. • Clean syntax/semantics • for reasoning rules, • acting rules, • and policies, that connect them. *Credits: Dr. Stuart Shapiro, CSE

48. SNePS: Sample Applications (Prototypes) • Neurological diagnosis (1984-86) • An expert system for fault diagnosis (1986-88) • CUBRICON: A multi-modal intelligent user interface to a tactical Air Force mission planner (1988-94) • Foveal Extravehicular Activity Helper-Retriever robot(1992-96) • An unexploded ordinance recovery robot (1996-2001) • Truth Maintenance in Data Fusion for Situation Assessment (1998-2003) • Intelligent agents in a Virtual Reality drama (2003-present)

49. Inductive Research Part A: SNePS Research Objective • To deploy SNePS as a KRR (Knowledge Representation and Reasoning) tool • For combined representing and reasoning about • The Ontology • The Data Graphs and Template Graphs • To make the SNePS representations available • For data mining • Induction and testing of new templates

50. Integrated Software System Fusion Research Approach Research Approach Problem and Domain Understanding Deductive SA/IA Model/Algorithmic Inductive SA/IA Information Fusion Engine for Real-time Decision Making (INFERD) SNePS KRR System Scalable Scenario Evidential Framework For ONA Graph Matching Graph Data Mining Ontology