Information value: the value of evidence

1. Placeholder slide To be used at the beginning and end of presentations or during breaks. This slide remains on screen while the audience is settling and the speaker is being introduced. Placeholder slide To be used at the beginning and end of presentations or during breaks. This slide remains on screen while the audience is settling and the speaker is being introduced.

2. Information value:the value of evidence Dr David J Marsay, C.Math FIMA A presentation to 19 ISMOR 29.08.2002 welcomewelcome

3. 3 Contents 1 Introduction 2 Examples 3 Theory 4 World-view 5 Implications 6 Conclusions Contents slidesContents slides

4. Introduction Section 1 Section divider slide Section divider slide

5. 5 Introduction Information is the life-blood of military C4ISR. Any time we prefer one set of information to another we implicitly �value� it. We think we could do better: lessons identified. studies. Specifically needed to support UK MOD�s ARP 14 �Battlefield picture compilation�.

6. 6 Introduction We use P(A|B) to denote �the probability of A given B� P(A) is used to denote the [prior] probability. For hypotheses {H} and evidence E: Shannon�s �entropy� is calculated from the �final� probabilities, {P(H|E)}. Jack Good�s �weight of evidence� is calculated from the likelihoods, {P(E|H)}. According to Bayes� rule, the probabilities can be calculated from the likelihoods and �the priors�.

7. Examples Section 2

8. 8 Examples Control: Suppose that a source sends accurate data to a deterministic machine. Shannon�s concept does not apply. Nor does the notion of �priors�. The value of the data can be determined by valuing the function of the machine - no fancy method needed. The likelihoods make sense. They are 0 or 1.

9. 9 Examples Command - �soft� aspects: For an information artefact (e.g., an INTSUM) to represent the same information implies that all recipients had the same priors. Thus everyone receives everything in the same order. Is this realistic? Alternatively, one could define some privileged �central� viewpoint for which the information is defined. Does this fit doctrine? Is it helpful?

10. 10 Examples Command - �soft� aspects: The likelihoods {P(E|H)} are a rating of the source of E. They are thus relatively �objective�, �knowable� and �shareable�. Likelihoods relate to current practice (reliability, accuracy).

11. 11 Examples Compilation: The work being reported on has looked at the relatively �hard� problem of compilation, particularly �Battlefield picture compilation� under ARP 14. Weights of evidence can be used. (See accompanying paper.) When is this reliable?

12. Theory Section 3

13. 13 Theory Jack Good�s evidence: Likelihoods are often straightforward.E.g., P(�Heads�|�Fair Coin�) = 0.5 by definition. Lab and field testing traditionally establish, in effect, likelihoods. Surprise = -log(likelihood). Weight of evidence (woe) is surprise, normalised by the prior expected surprise for the same evidence. (So that only �relevant detail� counts.)

14. 14 Theory Evidence is more fundamental than Shannon�s information Shannon�s entropy is expected surprise. The more useful cross-entropy is likely surprise. Woe supports alternative decision methods, such as sequential testing, hypothesis testing.

15. 15 Some questionable assumptions Shannon assumes that systems of interest are �Markov�. Shannon noted that �state-determined systems� are �Markov� with probability 1. But Smuts (e.g.) noted that evolution drives dynamical systems to adopt synergistic �emergent� structures. These had a priori probability 0. So for social systems, international relations, military conflict ... we cannot rely on Shannon�s �information�.

16. 16 Some questionable assumptions But can likelihoods be used? If we abandon Markov models, how are we to judge if a given algebra of likelihoods is valid? We need a �world meta-model� to replace Markov.

17. World-view Section 4

18. 18 SMUTS(synthetic modelling of uncertain temporal systems)

19. 19 Alternative ideas I postulate a model in which systems of interest to the military are Markov in space-time �zones�, with more interesting transitions at their boundaries. Thus Markov locally, but not globally. In essence emergence only happens when an over-adaptation is exploited. (E.g. Ashby, Piaget.) Thus, as long as we can learn at least as quickly, we should be able to recognise these situations too.

20. 20 Supporting evidence Applications to, and experiences of: warfare economics international relations. (My subjective view)

21. 21 Reuters data for the Balkans, the 90s

22. Implications Section 5

23. 23 Implications for �information� Technical differences: The difference between the expected weight of evidence (woe) and Shannon�s entropy is not a constant. Systems of interest tend to have �long range� sources of uncertainty, in addition to the �local� entropy. We need to allow for this and �expect the unexpected� to achieve robustness.

24. 24 Implications for �information� Some cases where Shannon might not be appropriate Poor �local� information. The �situation� cannot necessarily be recognised. The �target� is adaptable (particularly if adapting against us).

25. 25 Implications for �information� Typical symptoms that Shannon is inadequate: Mistakes often reflect a need to validate assumptions. Ossification, atrophy and vulnerability (Ashby / Piaget)

26. 26 Implications for �information� Notes: We can�t expect to have considered all possible hypotheses in advance. However, we do know when the truth is �something else� because the weights of evidence are poor for the assumed hypotheses. Thus we can detect deception and �fixation� (a form of self-deception).

27. Conclusions Section 6

28. 28 Conclusions The common concepts of information assume that systems are globally �simple�. Our systems of interest are not simple, but may be piece-wise �simple�. Jack Good�s �weight of evidence� can be used to �bridge� �islands of simplicity�. Using �weight of evidence� gives significant �added value� to using just Shannon information.

29. Placeholder slide To be used at the beginning and end of presentations or during breaks. This slide remains on screen while the audience is settling and the speaker is being introduced. Placeholder slide To be used at the beginning and end of presentations or during breaks. This slide remains on screen while the audience is settling and the speaker is being introduced.