520 likes | 690 Views
Fusion for the Army Knowledge Fusion Research Workshop. Dale A. Walsh Principal Engineer The MITRE Corporation Fusion SME for Army DCS G-2 October 2004 (Version 2004.02). Definition(s) of Fusion. JCS Pub 1-02
E N D
Fusion for the ArmyKnowledge Fusion Research Workshop Dale A. Walsh Principal Engineer The MITRE Corporation Fusion SME for Army DCS G-2 October 2004 (Version 2004.02)
Definition(s) of Fusion • JCS Pub 1-02 The process of examining all sources of intelligence and information to derive a complete assessment of an activity. • Textbook A process dealing with the association, correlation, and combination of data and information from single and multiple sources to achieve refined position and identity estimates, and complete and timely assessments of situations and threats, and their significance. The process is characterized by continuous refinements of its estimates and assessments, and the evaluation of the need for additional sources, or modification of the process itself, to achieve improved results. (Llinas/Hall) • Textbook, Simplified A process of combining data or information to estimate or predict entity states. (Steinberg/White) • Army’s “Usable” A series of processes performed to transform observational data into more detailed and refined information, knowledge, and understanding.(Walsh/Jones) These processes, by their very nature, will involve both automated processes and human cognition. Also included as part of fusion are the databases, human interfaces and information portrayal, and the control and feedback of the fusion process.
Fusion “Truisms” • Fusion is not new; automating some of it is • Fusion is a “critical enabler” • Fusion is driven by the Commander’s Priority Intelligence Requirements (PIRs) • Fusion takes place simultaneously at all echelons • Fusion facilitates Actionable Intelligence
Definitions • The JDL Fusion Model – A taxonomy and concept initially developed in the 1980’s by the then-Joint Directors of (DoD) Laboratories • Designed to allow the diverse fusion efforts to have a common language and framework – has worked very well to that purpose • Is NOT a system architecture, notional or otherwise • Is NOT intended to define problem sets that clearly vary from armed service to armed service • The “boundaries” between fusion “levels” are not intended to be completely distinct (so some processes may touch adjoining levels) • Is now shepherded by a panel of “graybeards” who still debate and modify the model on occasion
Level 1 Fusion Level 2 Fusion Level 3 Fusion Level 0 Fusion Sources & Sensors Human Computer Interaction Level 5 Fusion Level 4 Fusion Databases (Fusion & Support) The Data Fusion Model(from Joint Directors of Laboratories, as modified)
What do these levels do for me? • Level 1 fusion tells you what is physically out there • Level 2 fusion tells you how they’re working together and what they’re doing now • Level 3 fusion tells you what it all means, what will happen next, and how it affects your plans • Level 4 fusion tells you what you need to do to improve the information from levels 1-3 • Level 5 fusion allows you to look at and interact with the information from levels 1-4
Entities vs. Fusion Levels 1-3 We have taken the types or classes of entities and other knowledge goals and categorized them into “tiers”. On one end, Tier 1 entities are ones that can (in general) be observed directly; conversely, Tier 4 entities (really knowledge goals more than entities) are items of knowledge that can never be directly observed but can only be developed by cognitive analysis. Entity Tiers Tier 1 - Direct Observables (equipment, facilities, physical infrastructure) Tier 2 - Reportable Items (lower echelon units, individuals, specific events) Tier 3 - Conclude Existence (upper echelon units, organizations, comms nets, interrelationships between other entities) Tier 4 - Post-analysis (plans/courses of action, intentions, behavior at both the enterprise and sub-enterprise level)
Entities vs. Fusion Levels 1-3 Battlefield entities, and further knowledge and understanding about them and their actions/intents, are developed in Fusion Levels1, 2, and3only. Levels0 and 5 are simply pre- or post-cursors to that goal, not “lower” or “higher” levels of fusion. Level4 is the feedback loop that uses what is learned in Levels 1-3 to drive improvements in processing (or collection tasking) Entity Tiers Tier 1 - Direct Observables (equipment, facilities, physical infrastructure) Tier 2 - Reportable Items (lower echelon units, individuals, specific events) Tier 3 - Conclude Existence (upper echelon units, organizations, comms nets, interrelationships between other entities) Tier 4 - Post-analysis (plans/courses of action, intentions, behavior at both the enterprise and sub-enterprise level)
Level 2 Fusion Level 3 Fusion Entities vs. Fusion Levels 1-3 Knowledge is developed by continuous cycling of current knowledge and incoming data through Levels 1, 2, and 3 - and this cycle can occur at each echelon. Entity Tiers Level 1 Fusion Tier 1 - Direct Observables (equipment, facilities, physical infrastructure) Tier 2 - Reportable Items (lower echelon units, individuals, specific events) Tier 3 - Conclude Existence (upper echelon units, organizations, comms nets, interrelationships between other entities) Tier 4 - Post-analysis (plans/courses of action, intentions, behavior at both the enterprise and sub-enterprise level) Nouns --> Verbs --> Sentences --> Paragraphs
Current Fusion “State of the Art” • Level 0 fusion handled by sensor communities • Generating objects is not automated for all domains • Level 1 fusion largely conquered - automated correlation and update techniques in use for 10+ years • Level 2 fusion is largely unautomated, yet current and future quantities of observations call for some levels of machine assistance • Automated processes need “templates” for aggregation rules • Level 3 fusion will remain the G2/G3 analysts’ domain for the near future • Level 4 fusion is handled by the collection management arena • Level 5 fusion is a “visualization thing”; user interaction for feedback/control is at various depths
How much Automation? How much Cognition? • The desired or achievable balance between machine and Soldier has led to much debate • An Army Working Group will be stood up to try and address how much automation is “enough” • Four Modes of Operation have been defined: • Manual: No applications software to help • Computer Assisted: Tool Box of applications • Semi-Computer Controlled: Manned Assembly Line (series of applications run under user supervision) • Total Computer Controlled: Automatic Assembly Line
Notional Fusion Informational Flow Pixels, waveforms, grunts/squeals, etc. Entity Extraction Sensing/ Reporting Entities without Entities w/tech data Entities w/tech data Domain-specific Correlation (Level 1) Entities Eqmt/Unit/Facility/ Organization/Individual/ Event Entities w/o tech data Entities Generic Correlation (Level 1) Hypothesized Current Enemy COA(s) Correlated Events Correlated Objects Tgt COP SA Current Behavior Assessment (Level 2C) Event/Activity Aggregation/ Analysis (Level 2B) Object Aggregation/ Analysis (Level 2A) Aggregated objects Note: Decomposing Level 2 into 2A/B/C is an Army-unique concept to permit more detailed analysis of the classic L2 notions SU RE
Intelligence Domains All Source (Integrating) Domain Human Domain Signatures Domain Signals Domain Imagery Domain Visible COMINT Externals Soldiers as Sensors MASINT (exc. Imagery) Infrared COMINT Internals CI/HUMINT Radar (incl. SAR, MTI) ELINT Multi/Hyper Spectral Open Source Multi-INT (Intra-domain) Exchanges
An Analogy(Thanks to Rex Williams, USAIC&FH DCD) • The COP is like taking a snapshot of a soccer game: it will show you where the players and the ball are at that point in time, but it doesn’t show the flow of the game • The Intel Running Estimate (RE) is like a video of a soccer game: it shows the flow of the game and allows the viewer to project how they think the game is going and will go
030440Z - Seismic detection 030440Z - Acoustic detection 030441Z - Tracked vehicle on move 030443Z - Acoustic detection 030444Z - Tracked vehicle westbound 030445Z - MTI reports 8 vehicles 030447Z - UAV Video shows tanks on road 030448Z - Radio intercept of 3rd Tank BN 03 Sep 0445Z 03 Sep 0450Z 03 Sep 0448Z 03 Sep 0441Z 03 Sep 0444Z 03 Sep 0440Z Looks like the 42nd Tank BDE is going to attack our communications facility at Bradlehofer There’s some vehicles moving west... There’s 8 tanks moving west... 3rd Tank BN is moving west Elements of the 42nd Tank BDE are moving west The 42nd Tank BDE is moving towards our position There’s some vehicles out there... There’s something out there...
Now, A Word from Our Sponsor While the last slide showed a very realistic operational example, fusion in today’s Army cannot be constrained to the simple problems of observing, analyzing, and predicting movement of conventional units. The tasks of fusing information regarding non-linear battlespaces, terrorism cells, and other 21st century matters must also be added to fusion’s “plate”.
Level 5 Analysts Level 3 Level 4 Warfighter Level 2 Level 1 Data Repository COP/RE, BCS Level 0 Solid lines – primary data flow Dashed lines – alternate data flow – human interaction – displays, reports, etc. Sensors An Army Operational Extension to the JDL Data Fusion Model
UE UA Fusion from “Space to Mud” Organic/ Nat’l Sensing Home Station (HSOCs, IDC) EAC CORPS DIV BDE BN CO PLT SQD Σ DCGS-A Organic Sensors Tactical Overwatch Σ FCS with DCGS-A embedded software Organic Sensors Σ OFW Organic Sensors Σ
Tactical Overwatch Concept • Overwatch is a new, advanced concept for providing dedicated, focused intel to operationally engaged UA units (BDE/BN) • Overwatch “spotlights tactical forces with the full power of the theater, joint, and national set of intel capabilities” • Overwatch provides the UA high-level fusion along with collection management and targeting support • Overwatch provides sustained culturally-aligned global awareness conducted daily (and operates in peacetime and during war) • Overwatch provides support to engaged UAs while anticipating transitions and future operations • MI Brigades (as UEy) are the platforms from which Overwatch is executed, with INSCOM’s IDC acting as a hub for all Overwatch activities
Fusion for Dismounted Squads • Fusion Goals • Locally-focused Level 1 fusion on organic sensing to produce the local COP/SA • Integration (and resolution) of the SA/SU products from FCS/UA above • Semi-autonomous Level 4 asset management • Soldier-supportive visualization/reporting • Organic Sensing • EO/IR, acoustic/magnetic/seismic, CBRN, soldiers • Efforts • Objective Force Warrior, Warriors’ Edge (ARL)
Fusion for Unit of Action • Fusion Goals • Regionally-focused Level 1 fusion on a huge multi-INT organic sensing flow, leading to generation of COP/SA • Integration and deconfliction of the SA from squads below and the SA/SU of tactical overwatch support from above • Effective cross-cuing to maximize utility of organic sensors • Level 2/3 fusion to see the aggregated red and blue picture and then develop predictions on how/where the focused conflict might develop (ie, SU) • Intelligent management of “steerable” collection assets • Seamless use of fusion products between ISR and C2 • Organic Sensing • EO/IR/SAR, video, LADAR, acoustic, seismic, magnetic, radars, GMTI, CBRN, Met, EMTI • Efforts • FCS with DCGS-A embedded software, FBKFF STO (ARL/I2WD, Level 2/3)
Fusion for Unit of Employment • Fusion Goals • Globally-focused Level 1 fusion within each discipline feeding an all-source fusion process which also leverages COP/SA/RE/SUs from below/above/joint • Level 2/3 fusion to aggregate forces then predict future developments, put them in context with terrain/weather/plans, and develop knowledge about intended behavior/driving forces and their impact • Joint operations (true at all echelons, but critical here) • Organic Sensing • COMINT, ELINT, IMINT, MASINT, HUMINT, OSINT, Soldiers • Efforts • DCGS-A, FBKFF STO, DARPA RAID (Level 3)
Fusion for Overwatch • Fusion Goals • Extract relevant information from the large “take” available at the overwatch level • Make that information usable and actionable • Provide strategic-level SA/SU • Provide tactical overwatch support (to echelons as low as UA) in a timely, adjudicatable form • Provide models, guidance, context information, etc. to support fusion processing at all echelons • Sensing • Vast amounts of strategic/national intel • Some organic sources • Efforts • INSCOM’s IDC work and research, NGIC
Fusion “Longpoles” (Page 1) • ATR/AiTR - The quantity of imagery and video is exploding, yet the ability to extract objects and information from those feeds in an automated way trails badly. Techniques like change detection are available but not implemented on a large scale. • Cross Domain - Intelligence domains breed classification domains, which restrict the timely flow of critical intelligence to those who need it. The resolution of this issue is as much political/procedural as it is technical. Intelligent agents can work to provide critical information while still protecting sources and methods - if their implementations are approved for use. • Sensor Webs - A simple web of 20 deployable MASINT sensors can produce an overwhelming amount of raw data - so much that it becomes critical for Level 1 fusion first be accomplished within the web. This fusion must still produce a product that can be efficiently fused with other info/product held upstream.
Fusion “Longpoles” (Page 2) • Text/Speech Translation - Translation of printed, handwritten, and spoken languages is needed to begin to exploit the vast amounts of available non-formatted information sources. Translation support must be able to have a small footprint at SQD (for limited capabilities) and support high volumes for overwatch. • Text/Speech Exploitation - More robust capabilities are needed to first parse (given post-translation grammatical constructs of each language) and then to extract/make sense of “free text” data. Eventually the ability to infer meaning into jargon, etc. needs to be explored. • Scalability of Level 1 - While ASAS and TES provide very viable (and yet different) Level 1 correlation capabilities today, the sensing capabilities of an FCS UA will increase the input stream 100-fold. The scalability of current approaches/software has not been determined.
Fusion “Longpoles” (Page 3) • Distributed Level 1 Architecture, with Tactical Overwatch - Given the required analytical timelines and the realities of future communications throughputs, Level 1 fusion cannot be centralized for a UE or even a UA. Distributing fusion within a UA implies that issues of adjudication between varying non-synchronized views must be dealt with, and only in a cohesive metadata/data tagging environment. This problem has not be solved nor even sufficiently researched to date. The ability to effectively cross-cue from both single domain and all source results will also be critical. And the addition of tactical overwatch products into the fusion at a UA will complicate a distributed fusion architecture even more. • Reconstruction of Functional Networks - The application of varied techniques to sort through quantities of information and discover/confirm/assess the non-obvious, n-deep interrelationship networks is not widespread, nor is it readily manageable by average analysts. This capability is critical for many current operations.
Fusion “Longpoles” (Page 4) • Level 2 Fusion at Tactical Echelons - Given insufficient staffs at UA echelons, the need to perform Level 2 fusion analysis with sufficient automated support on the constrained problem view of the tactical commander is critical. • Level 2 Fusion at Overwatch Echelons - At higher echelons, the ability to constrain the problem view must be lifted. More robust Level 2 capabilities, especially for urban, asymmetric, stabilization, and other types of operations, will need to be developed. • Red Models - In order to support Level 2 fusion at both tactical and overwatch echelons, a significant amount of prerequisite knowledge of enemy doctrine/beliefs/structure/methods/etc. will need to be first developed and then formulated for use by automated fusion processes. • Simplified User Interfaces - Tactical users see/sense much information, yet have no simple, quick way of inputting that information into the fusion process. Similarly, they need easy, understandable access to the results and products of fusion.
Mapping “Longpoles” to Echelons ü ü ü SCI plus SBU Coll ü ü ü ü ü ü ü ü ü ü ü ü ü ü ü ü ü ü ü ü ü ü ü ü ü ü
Level 1 Fusion Flowchart HCI Alignment Correlation Reports/ Observations • COP • Targets/BDA • Sit Alarms • Level 2 Processes Spatial Alignment Candidate Retrieval Correlated Entities Temporal Alignment Candidate Scoring Combination (State Estimation) Identity Alignment Candidate Assessment Determination of “best” Resultant record Technical Alignment(s) Knowledge Bases New, add Already seen
Type of |Entity Type | Process| Classes of Knowledge | INT |U Q F I E O | Speed | Loc Mvmt Time Type ID Comp Status Intent| COMINT Ext x fast COMINT Int x x x x x x slow ELINT x fast IMINT EO/IR x x x * * med IMINT MTI x fast HUMINT EPW x x x x x x slow MASINT ASM x x fast MASINT MS/HS x x x slow OSINT x x x x x x slow ZSU-23-4, seen by EO and MTI med EO and MTI and ELINT med EO and MTI and ELINT and COMINT Int slow The Advantage of Multi-INT(a notional example, not anyone’s official chart of sensor capabilities!)
Part Four: Level 2 Fusion Concepts and Architecture(with thoughts on Level 3)(Version 2)(still a work in progress)
170,000 reports => 105 tracks • Level 1 fusion correlates inputs and produces a set of tracks 105 tracks => 15 “units” 15 “units” => a BN-sized force • Level 2 first associates and aggregates those tracks into forces moving to contact this way threatening our forces here • and those forces into force structures • then proposes that the force is exerting (L2) /will exert (L3) itself in some way • which will threaten our forces and require the battle to be “shaped” Level 2/3 Fusion at UA Scenario taken from USAIC&FH MAPEX
Why Level 1 Fusion Isn’t Enough If only Level 1 fusion is done, there is no roll-up of entities, and the Bde COP carries all the “little things”. With only L1: 500+ icons With L1 and L2: >50 icons
Mid 1980’s Mid 1990’s 2000-2001 Dec 2001 Jan 2002 June 2002 Oct 2002 Level 1 Fusion Level 2 Fusion Level 3 Fusion Level 0 Fusion Sources & Sensors Human Computer Interaction Level 5 Fusion Level 4 Fusion Databases (Fusion & Support) Level 2/3 Fusion State-of-the-art Level 2 - Situation Refinement (how they work together and what they’re doing) Level 3 - Threat Refinement (what it means and how it affects our plans) JDL Fusion model proposes concept of Levels 2 and 3 ASAS makes small inroads to automating some Level 2 fusion FCS/OF concept emerges - trades armor for ISR/Knowledge (aka fusion) DARPA Level 2/3 Symposium - sets an 8-year path for research (RAID planned for FY05 start) G-2 says 8 years too long; develop ‘03 STO Notional L2/3 Architecture presented Joint I2WD/ARL FBKFF STO approved
Level 2 Fusion Flowchart(a work in progress) HCI “Linked” Correlated Entities Correlated Simple Objects Aggregated, linked set of objects Object Aggregation Event Derivation Aggregate Analysis Correlated Complex Objects Event Reporting “Linked” Events/Activities Event Aggregation Deception Assessment Red Models Event Interpretation & Analysis Learning Hypothesis Assessment Hypothesis Generation Knowledge Bases Context Interpretation Behavior Hypotheses (large and small) Environment Hypotheses of the current behavior/ motive/objectives of the enterprise and its constituents, with some sense of validity/score of each viable hypothesis
“Level 2B” - Event/ Activity Aggregation/ Analysis Level 2, Decomposed “Level 2A” – Object Aggregation/Analysis HCI “Linked” Correlated Entities Correlated Simple Objects Aggregated, linked set of objects Object Aggregation Event Derivation Aggregate Analysis Correlated Complex Objects Event Reporting “Linked” Events/Activities Event Aggregation Deception Assessment Red Models Event Interpretation & Analysis Learning Hypothesis Assessment Hypothesis Generation Knowledge Bases Context Interpretation “Level 2C” – Current Behavior Assessment Behavior Hypotheses (large and small) Environment Hypotheses of the current behavior/ motive/objectives of the enterprise and its constituents, with some sense of validity/score of each viable hypothesis
Some Object Aggregation Techniques • Template-based • Example: Observations of lower-tier units or equipment spawn inferred units of one higher echelon • Traffic Analysis • Example: Piecing together hundreds of observations of emissions to reconstruct a communications network • Geolocation/Movement • Examples: a unit that stays at a facility may be using the facility; a unit whose equipment is moving west is likely also moving west • Other Data Mining • Example: What n-deep relationships can two people have if they talk to the same people or share the same religious center?
“3 Events” Event: one object performing one simple* action (* be at, move, shoot, emit, etc.) Activity “Trees” of Events Tank #3713 moving N Tank #3717 moving N Tank #3719 moving N +association of these three tanks to the 33 TK CO
“Event Cluster” Event Cluster: complex object performing one action ADA at Bridge #36 (Conf 90%) Comms between 33 TK CO and ADA unit (Conf 95%) Activity “Trees” of Events Tank #3713 moving N Tank #3717 moving N 33 TK CO moving North (Conf 85%) Tank #3719 moving N +association of these three tanks to the 33 TK CO
“Activity” 33 TK CO to conduct river crossing operation Bridge #36 approx. 1345Z (Conf 73%) Activity: upper ‘tree’ construct built from multiple objects performing a complex action (derived from a set of related simple actions) ADA at Bridge #36 (Conf 90%) Comms between 33 TK CO and ADA unit (Conf 95%) Activity “Trees” of Events Tank #3713 moving N ACTIVITY MODEL #121 – River Crossing Unit moving towards river Bridge on unit’s probable path ADA deployed at/near bridge Tank #3717 moving N 33 TK CO moving North (Conf 85%) Tank #3719 moving N +association of these three tanks to the 33 TK CO √ √ √
Level 2 Fusion Flowchart(a work in progress) HCI “Linked” Correlated Entities Correlated Simple Objects Aggregated, linked set of objects Object Aggregation Event Derivation Aggregate Analysis Correlated Complex Objects Event Reporting “Linked” Events/Activities Event Aggregation Deception Assessment Red Models Event Interpretation & Analysis Learning Hypothesis Assessment Hypothesis Generation Knowledge Bases Context Interpretation Behavior Hypotheses (large and small) Environment Hypotheses of the current behavior/ motive/objectives of the enterprise and its constituents, with some sense of validity/score of each viable hypothesis
Critical point: As is true between Levels 1, 2, and 3, the interaction between Levels 2A, 2B, and 2C are not linear nor necessarily sequential. Knowledge is developed by a continual and ad hoc application of all these levels of fusion processes.
Level 3 Fusion Takes the Course of Action hypotheses from Level 2C, vetted by the evidence from Levels 1, 2A, and 2B, and projects future Courses of Action hypotheses and assesses them vis-à-vis the capabilities, vulnerabilities, opportunities, environment, etc. of both red and blue. (A notional architecture remains to be tackled.)
Enabling Technologies • Ontologies provide organization of the terms and concepts, to permit clearer cross-understanding of what things are • Intelligent Agents are software that can, given a set of actions/reactions, provide automation to tasks such as smart dissemination, alert/alarms, sanitization/declassification, etc. • Knowledge Bases/Management are advanced IT techniques for capturing and manipulating the “higher level” abstractions of information (environment, belief, intent, etc.) • Service-based Architectures are the emerging means to support net-centric operations; they imply that the processes will be able to function as invokable services hosted within some network space • Distributed Fusion Architectures are approaches to implementing fusion processing across a multi-echelon net of fusion process services without the creation and use of a central fusion node (to date, this is a very immature field of study)
Army Goals for Level 2 Fusion • “Unit of Action” Level 2 goals Given that we can sense where the threat is... • Where is it going and what are its near-term goals? • When will they see us and decide to shoot? • “Unit of Employment” Level 2 goals • Who are the players and where are they operating? • What are the large-scale objectives? • Why are they doing what they are doing? BOTTOM LINE: We cannot hope to completely automate these goals… but given the vast amounts of observations that will be collected, we need to find ways to automate hypothesizing knowledge from amidst the information
Army Fusion Efforts • Program Development • Objective Force Warrior • Future Combat System • DCGS – Army (incl. ASAS, TES, CHIMS, CGS) • INSCOM/Information Dominance Center • Research • Warrior’s Edge/Horizontal Fusion Initiative • Warrior’s Lens – ARL 6.1 Fusion Work Group • SIBRs • Adv Viz Support to Fusion; Ontology-based Fusion Model; VMTI Tracker • STOs/ATDs/ACTDs • FBKFF, Eye in the Sky, Netted Sensors, BTRA, Gnd Station Tech Testbed • Several smaller studies on specific topics • Urban issues, MASINT phenomenologies, etc. • AIS/Univ Buffalo IR&D on Graph Theory • Cooperation across DoD • DoD-level fusion community symposia and coordination • Leverage of DARPA efforts (RAID, DDB, DTT, AIM, etc.)