Topic 11: Level 2

1. Topic 11: Level 2 David L. Hall

2. Topic Objectives • Introduce the concept of Level 2 processing • Survey and introduce methods for approximate reasoning • Introduce concepts in probabilistic reasoning and fusion (e.g., Bayes, Dempster-Shafer, etc) • Describe challenges and issues in automated reasoning • Note: this topic will focus on report-level fusion; Topic 12 will introduce reasoning methods such as rule-based systems & intelligent agents

3. Level 2 Processing (Situation Refinement)

4. LEVEL TWO PROCESSING SITUATION ASSESSMENT • OBJECT AGGREGATION • Time relationship • Geometrical proximity • Communications • Functional dependence EVENT/ACTIVITY AGGREGATION • CONTEXTUAL INTERPRETATION/FUSION • Environment • Weather • Doctrine • Socio-political • MULTI-PERSPECTIVE ASSESSMENT • Red/Blue/White Level Two Processing:Situation Assessment

5. Shortfalls in L2/L3 Fusion Research From Valet, L., et al: “A Statistical Overview of Recent Literature in Information Fusion”, FUSION2000, Paris, France, July 2000 (~85% of pubs reviewed at L1) From Nichols, M., “A Survey of the Current State of Data Fusion Systems”, OSD Decision Support Ctr. presentation at SPAWAR San Diego, CA, May 2000. (17 of 100 systems with any L2, L3 at all, mostly basic/simple techniques.)

6. Type of Inference Applicable Techniques High - Threat Analysis - Knowledge-Based Techniques - Expert Systems- Scripts, Frames, Templating - Case-Based Reasoning - Genetic Algorithms - Situation Assessment - Decision-Level Techniques - Neural Nets- Cluster Algorithms - Fuzzy Logic INFERENCE LEVEL - Behavior/Relationships of Entities INFERENCE LEVEL - Estimation Techniques - Bayesian Nets - Maximum A Posteriori Probability (e.g. Kalman Filters, Bayesian) - Evidential Reasoning - Identity, Attributes and Location of an Entity - Existence and Measurable Features of an Entity Low - Signal Processing Techniques Hierarchy of Inference Techniques

7. Examples of Data Fusion Inferences

8. Comments on L-2 and L-3 Techniques • Reasoning for level-2 and level-3 processing involves context-based reasoning and high level inferences • Techniques are generally probabilistic and entail representation of uncertainty in data and inferential relationships • Methods represent knowledge at the semantic level • Rule-based methods • Graphical representations • Logical templates, cases, plan hierarchies, agents & others

9. SYMBOLIC PROCESSING TECHNIQUES Pattern Matching Inference Search Knowledge Representation APPLICATION AREAS Knowledge Acquisition Computer Vision Machine Translation Expert Systems Robotics Planning Learning Natural Language Processing Text Understanding Automatic Programming Speech Understanding Intelligent Assistance Elements of Artificial Intelligence

10. Challenges in Symbolic Reasoning • Computer Inferencing Challenges • Lack of real-world knowledge • Inability to deal with the perversity of English or other languages • Requirement for explicit knowledge representation & reasoning methods • Computer advantages • Processing speed & power (use of physics-based models) • Unaffected by fatigue, emotion, bias • Machine learning from large data sets • Human Inferencing Capabilities • Continual access to multi-sensory information • Complex pattern recognition (visual, aural) • Semantic level reasoning • Knowledge of “real-world” facts, relationships, interactions • Use of heuristics for rapid assessment & decision-making • Context-based processing

11. Categories of Representational, Decomposition Techniques

12. Major reasoning approaches Knowledge representation • Rules • Frames • Scripts • Semantic nets • Parametric Templates • Analogical methods Uncertainty representation • Confidence factors • Probability • Dempster-Shafer evidential intervals • Fuzzy membership functions • Etc. Reasoning methods & architectures • Implicit methods • Neural nets • Cluster algorithms • Pattern templates • Templating methods • Case-based reasoning • Process reasoning • Script interpreters • Plan-based reasoning • Deductive methods • Decision-trees • Bayesian belief nets • D-S belief nets • Hybrid architectures • Agent-based methods • Blackboard systems • Hybrid symbolic/numerical systems

13. Declaration of Identity (sensor A) Sensor A • Decision-Level Identity Fusion • Voting methods • Bayes method • Dempster- Shafer’s method Entity, target or activity being observed Declaration of Identity (sensor B) Sensor B Fused Declaration of Identity Declaration of Identity (sensor N) Sensor N Decision-level identity fusion In the last lecture we addressed the magic of pattern recognition! This represents a transition from Level-1 identity fusion to Level-2 fusion related to complex entities, activities, events; the reasoning is performed at the semantic (report) level.

14. Classical Statistical Inference • Based on empirical probabilities • Definitions: • statistical hypothesis: statement about a population which based on information from a sample, one seeks to support or refute • statistical test: a set of rules whereby a decision on H is reached • Measure of test accuracy: • a probability statement re: the decision when various conditions in population are true

15. Classical Statistical Inference • Test logic: • Assume null hypothesis is true (HO) • Examine consequences of HO true in sampling distribution for the statistic • If observations have high P of occurring, data do not contradict HO • Otherwise data tend to contradict HO • Level of Significance: • Define probability level that is considered too low to warrant support of HO if P (obs. data/ HO true) <  - > reject HO

16. E2 E1 ELINT COLLECTOR E2 Forward Edge of Battle Area (FEBA) • Type 1 and Type 2 radars exist on a battlefield • These radars are known to have different PRI • ability • Problem: Given an observed PRI have we seen a • Type 1 or Type 2 radar? Emitter Identification: Example Note: During this presentation we will use an example of emitter identification, e. g. for situation assessment related to a DoD problem. However this can be translated directly into other applications such as medical tests, enfironmental monitoring, or monitoring complex machines;

17. A measure of the probability that radar class 2 will use a PRI in the interval PRIN≤ PRI ≤ PRIN+1 E1 (Radar Class 1) E2 (Radar Class 2) Probability density function E1 (Radar Class 1) E2 (Radar Class 2) Pulse repetition-interval (PRI) Probability density function PRIN PRIN+1 PRI PRIC Classical Inference for Identity Declaration: Example

18. Issues with Classical Probability Requires knowledge of a priori probability density functions (Usually) applied to a hypothesis and it’s alternate Does not account for a priori information about the “likelihood in nature” of a hypothesis being true (Strictly speaking) classical probability can only be used with repeatable experiments

19. Can be used on subjective probabilities • Does not necessarily require sampling, etc. • Statement: • If H1, H2 --- Hi represent mutually exclusive and exhaustive hypotheses which can explain an event, E, that has just occurred • Then • And • Nomenclature • P(Hi/E) = a posteriori probability of Hi true given E • P(Hi)= a priori probability • P(E/Hi) = probability of E given Hi true P(E/Hi) P(Hi) P(Hi/E) = P(E/Hi) P(Hi) i P(Hi) = 1 exhaustivity i Bayesian Inference

20. P(PRI0  EMITTER X) P(EMITTER X) P(EMITTER X  PRI0) = • In the case of multiple measurements, P(EMITTER X ) P(PRI0EM X) P(F0 EM and PRI0) P(EMITTER X  PRI0 and F0) = P(EMITTER X) ------------ P(EMITTER Y) ------------ P(EMITTER Z) ------------ P(PRI0)  EMITTERi) P(EMITTERi) i • Does not require sampling distributions • Analyst can estimate P(PRI/E) • Analysts can include any knowledge they have pertaining to • the relative numbers of emitters Bayes Form: Impact on the Examples of Emitter Identification

21. Major Development Issue: ability to model/establish P(D/T) as a function of range, SNR, etc. • Note: Columns of declaration matrix are mutually • exclusive, exhaustive hypotheses that explain • an observation Concept of Identity Declaration by a Sensor

22. Based on empirical probabilities derived from tests, we have for a sensor [P(Di/Oj)] i = 1, n j = fixed jP(Di/Oj) = 1 Then Then construct probability matrix for each sensor • D • D1 • D2 • D3 • n 1 2 3 --- H [P(Di/Oj)] • NOTES: • 1. Column SUM =1 • Row SUM  1 • 2. n (not necessarily) = m Sensor 3 Sensor 2 Sensor 1 Identification, Friend, Foe Neutral (IFFN) Bayesian Example

23. P(Oj)[P(D1Oj) P(D2Oj) … P(DkOj) … ]  P(Oj)[P(D1Oj) P(D2Oi) P(D2 Oj) … P(DnOi) ] i IFFN Bayesian Example continued • Given multiple evidence (observations) Bayes rule allows fusion of declarationsfor each object (i.e., hypothesis) • P(OjD1  D2  ...) = • Individual sensors provide (via a declaration matrix), P(DiOj) • Bayes rule allows conversion of P(DiOj) to P(OjDi) • Multiple sensors provide: {P(O1D1), P(O2D1), … P(OjD1) …} FROM SENSOR 1 {P(O1D2), P(O2D2), … P(OjD2) …} FROM SENSOR 2 • • •

24. SENSOR #1 Observables  Classifier  Declaration P(D1Oj) D1 Bayesian Combination Formula Decision Logic SENSOR #2 ETC. P(D2Oj) • MAP • Threshold • MAP • etc. D2 P(OjD1 D2  Dn) • Select highest • value of P(Oj) Fused Identity Declaration j = 1, … M SENSOR #n ETC. P(DnOj) Dn • Fused probability of object j, given D1, D2 …, Dn • Transformation from • observation space to • declaration • Uncertainty in • declaration as • expressed in a • declaration matrix Summary of Bayesian Fusion for Identity

25. Bayesian Inference • The good news • Allows incorporation of a priori information about P(Hi) • Allows utilization of subjective probability • Allows iterative update • Intuitive formulation • The bad news • Requires specification of “priors” • Requires identification of exhaustive set of alternative hypotheses, H • Can become complex for dependent evidence • May produce “idiot” Bayes result

26. PROPOSITIONS ABOUT THE EXHAUSTIVE POSSIBILITIES IN A DOMAIN DISTRIBUTION OF BELIEF EVIDENCE  BELIEF  (OVER) Dempster-Shafer Theory • Arthur Dempster (1968):Generalization of the Bayesian approach • Glen Shafer (1976):Mathematical theory of evidence • Basic issue: Manner in which belief, derived evidence, is distributed over propositions (hypotheses)

27. Distribution of Belief • Operational Mode • Humans seemingly distribute belief (based on evidence) in a fragmentary way, thus, in general, for evidence, E, and propositions, A, B, C, we will have: M(A) = measure of belief that E supports A exactly M(AB) = measure of belief assigned to the disjunction, which includes A etc.

28.  PR(A) = M(, 2)  A Probability and Belief PR(A) + PR(~A) = 1 [SPT(A), PLS(A)] • Probabilities for propositions are induced by the mass distribution • Bayesian mass distributions assign only to the set of single mutually exclusive and exhaustive propositions (in  only). • With the D-S approach, belief can be assigned to a set of propositions that need not be mutually exclusive. This leads to the motion ofevidential interval

29. Example of probability mass assignment • A single dice can show one of six observable faces (these are the mutually exclusive and exhaustive hypotheses) • The number showing on the dice is • 1 • 2 • 3 • 4 • 5 • 6 • Propositions can include; • The number showing on the dice is even • The number showing on the dice is 1 or 3 • … • The number showing on the dice is 1 or 2 or 3 or 4 or 5 or 6 (the “I don’t know” proposition); The set of hypotheses is; Θ = {1,2,3,4,5,6} The set of propositions is; 2Θ= {1, 2,3,4,5,6, 1 or 2, 1 or 3, 2 or 3, 3 or 4, ……. 1 or 2 or 3 or 4 or 5 or 6}

30. SPT(A) = M(, 2)   A PLS(A) = 1 - SPT(~A) = 1 - M(, 2) • and SPT(A) + SPT(~A)  1 • Uncertainty (A) = PLS(A) - SPT(A) • If for all A, U(A) = 0  Bayesian   A Probability and Belief Formulae Adapted from Greer, Thomas H., “Artificial Intelligence: A New Dimension in EW”, Defense Electronics, October, 1985, pp. 108-128.

31. Plausibility Evidential Interval .25 .85 Refuting Evidence Supporting Evidence Support and Plausibility • Support • The degree to which the evidence supports the proposition • The sum of the probability masses for a proposition and its subjects • Plausibility • The extent to which the evidence fails to refute a proposition • P(A) = 1 - S(~A) = 1 • Examples • A(0,0)  S(A) = 0 no supporting evidence • P(A) = 0  S(~A) = 1 evidence refutes A • A(.25, .85)  S(A) = .25 S(~A) = .15 Adapted from Greer, Thomas H., “Artificial Intelligence: A New Dimension in EW”, Defense Electronics, October, 1985, pp. 108-128.

32. Belief Distribution • SAM-X 0.3 • SAM-X, TTR 0.4 • SAM-X, ACQ 0.2 • UNKNOWN 0.1 D-S Threat Warning Sensor (TWS) RF PRF TWS • Then, the evidential intervals are; • SPT (SAM-X, TTR)= 0.4 • PLS (SAM-X, TTR) = 1-SPT (SAM-X, TTR) • = 1-SPT (SAM-X, ACQ) • = 1-0.2 • = 0.8 • (SAM-X, TTR) = [0.4, 0.8] • Similarly, • (SAM-X)= [0.9, 1.0] • (SAM-X, ACQ) = [0.2, 0.6] Dempster-Shafer Example Adapted from Greer, Thomas H., “Artificial Intelligence: A New Dimension in EW”, Defense Electronics, October, 1985, pp. 108-128.

33. Composite Uncertainty:Two Source Example

34. Dempster Rules of Combination • The product of mass assignments to two propositions that are consistent leads to another proposition contained within the original (e.g., m1(a1)m2(a1) = m(a1)). • Multiplying the mass assignment to uncertainty by the mass assignment to any other proposition leads to a contribution to that proposition (e.g., m1()m2(a2) = m(a2)). • Multiplying uncertainty by uncertainty leads to a new assignment to uncertainty (e.g, m1()m2() = m()). • When inconsistency occurs between knowledge sources, assign a measure of inconsistency denoted k to their products (e.g., m1(a1)m2(a1) = k).

35. SOURCE 1 Compute all Credibility Intervals SOURCE 2 Compute all Credibility Intervals 1. Compute Credibility Intervals SOURCE 2 2. Map the Mass of Belief Distribution A B C D K = measure of all mass associated with conflicting reports K = (.2 x .2)+(.4 x .2)+(.3 x .2) = 0.18 SOURCE 1 3. Compute Composite Beliefs • For each proposition, sum all of the masses that support the proposition and divide by 1-K: • SAM-Y TT = A + B + C + D • 1 - K • = 0.49 Compute the credibility intervals for pooled evidence Apply Decision Rule 1-K = 1 - 0.18 = 0.82 Composite Uncertainty:Computing Belief Distributions for PooledEvidence

36. Pooled Evidence

37. Mi/Oj SENSOR #1 Observables  Classifier  Declaration Compute or enumerate mass distribution for given declaration Combine/Fuse Mass Distributions via Dempster’s Rules of Combination M(Oj) = F(mi(Oj)) ETC. SENSOR #2 ETC. Fused Identity Declaration Decision Logic • Select best combined • evidential interval • Fused probability mass for each • object hypothesis, Oj • General level of uncertainty •  • leading to •  • Combined evidential intervals ETC. SENSOR #n ETC. • Transformation from observation • space to mass distributions • mi(Oj) Summary of Dempster-Shafer Fusion for Identity

38. Dempster-Shafer Inference • The good news • Allows incorporation of a priori information about hypotheses and propositions • Allows utilization of subjective evidence • Allows assignment of general level of uncertainty • Allows iterative update • The bad news • Requires specification of “priors” • Not “intuitive” • May result in “weak” decisions • More computationally demanding than Bayes • Can become complex for dependent evidence • May produce “idiot” D-S result

39. Weight for sensor, S1 SENSOR #1 Observables  Classifier  Declaration D1 Voting Combination Formula Decision Logic SENSOR #2 ETC. Weight for sensor S2 D2 • Select highest vote (majority, plurality, etc) Fused Identity Declaration SENSOR #n ETC. Dn • Fused decision via weighted voting formula Weight for sensor SN • Uncertainty in • declaration as • expressed in a • weight or confidence for each sensor • Transform from • observation space to declaration Summary of Voting for Identity Estimation V(Oj) = wiδi(Oj)

40. Some Decision Fusion Techniques

41. Topic 11 Assignments Preview the on-line topic 11 materials Read chapter 7 of Hall and McMullen (2004) Writing assignment 8: One page description of how the level 2 process applies to your selected application Discussion 5: Discuss the challenges of designing and implementing a data fusion system both in general and for your selected application.

42. Data Fusion Tip of the Week Young researchers in automated reasoning and artificial intelligence wax enthusiastically about the power of computers and their potential to automate human-like reasoning processes (saying in effect “aren’t computers wonderful!”) Later in their careers these same researchers admit that it is a very difficult problem and believe they could make significant progress with increased computer speed and memory Still later, these researchers realize the complexities of the problem and praise human reasoning (saying in effect, “aren’t humans wonderful!”)