Toward mission-specific service utility estimation using analytic stochastic process models

1. Toward mission-specific service utility estimation using analytic stochastic process models Dave Thornley International Technology Alliance http://usukita.org Imperial College London

2. Quality, Utility, Value • Quality of Information (QoI) used as a focus for comprehension, generality and communication • What does it mean? • What else could have told me this? • What guarantees can we provide? • Supports choices during action • Utility of information (UoI) or another service supports design choices • Will this system support our achievement of goals, and how well? • Will it still work when we’ve finished with it • Can we sell it or its information products? • Given utility estimates for some purpose, we can assess the value that should/will be ascribed (VoI) • Will be ascribed “This piece of information makes my life easier.” • Should be ascribed “No it doesn’t”

3. Mission Abstraction, Requirements and Structure • PLANs provide structure and projections • PHYsics includes sensor models, traffic generators and environmental modulators • INTelligence includes receipt of signals, fusion, storage, hypothesis and dissemination • Situational Awareness maps knowledge to awareness and understanding (more next slide) • Decision Maker is a representation of the human in the loop • ACTion maps decisions to physical outcomes via effectiveness measures

4. Abstract stochastic perspective • In a given deployment, predictable outcomes are influenced by sensing service design choices • A sensing package that results in better outcomes for the same plan is providing higher quality of information amortized over the mission, and is of higher utility specifically to that mission • Consider the outcomes as a locus of possibilities, which may be a combination of discrete and continuous variables (target location and assessment, sensor energy remaining and functional integrity). • A stochastic model associates probabilities with states as a function of time. If we ensure that there is a state defined for each outcome we care about, we can quantifies the contributions of alternative services in characteristics that can be meaningfully compared

5. Information driven model • Detection cues tracking • Tracking enables focus of various types through intelligence gathering • Alternative competing hypotheses are evaluated using intelligence product arrivals and retrieval/ refactoring to achieve focus and situational awareness • Decisions drive action or instruct sources • Action creates feedback

6. Keithley’s Knowledge Matrix The matrix was originally developed to assess the value of information fusion algorithms to C4ISTAR missions to justify the cost of their development. ISR requirements are specified in terms of a canonical set of questions. The questions need to be supported by details of the required QoI for the mission to succeed. The ISR question is answered at the level of the commander’s information requirements not the data level. BUT insufficiently flexible to allow a detailed consideration for matching resources to dynamic mission requirements.

7. Timed stochastic outcome modeling for utility comparison

8. Methods • Performance Analysis Process Algebra • Compositional timed stochastic modeling • Abstract to information product delivery and operational modes • Can be massaged into a range of solution tools • Native model is a continuous time discrete state Markov chain • Equilibrium solutions • Measurement of consumption and exposure • Transient solutions • Response time predictions • Evolution of accuracy achievable

9. INCIDER • DSTL human factors team • Our example scenario is lifted directly and simplified somewhat from one of their presentations: www.dodccrp.org/events/2006_CCRTS/html/presentations/025.pdf Also see: Dean, D., Vincent, A., Mistry, B., Hossain, A., Spaans, M. and Petiet, P., “Representing a Combat ID Analysis Tool within an Agent Based Constructive Simulation”, The International C2 Journal, Vol 2, No 2

10. Isolated decision making scenario

11. Scenario components • Policy includes orders and tests • Signals include EO interpretation, TID comms, Scout vision and HQ picture • Evidence raises, lowers or sets confidence in Red and Blue hypotheses • Model that can generate Red and Blue traffic, and the SA maintenance and decision making sequences for each has 1597 states, with a 5 phase Erlang FAT transition process FAT <signals> ( (Sensors <evidence> SA) <policy> DM )

12. MARS Federated Analytic Traffic • Entities are modeled as states that combine, in our example • Location – space subdivide according to invariants in the response of the mission • Class, affiliation &c. (mood?) – just Red/Blue here • Multiple sensing modalities must be modeled and correlated, so traffic centralized, and formed of components, each representing an entity or group of entities

13. FAT traffic progress

14. MARS Situational Awareness • Confidence in each hypothesis Red, Blue • Example has zero, low, medium, high • In general, these demarcations will be selected according to regions on the real line that do not change the outcome of fusion • Predicates calculated on these states • Comparison of values (less/greater/equal)

15. Evolution of SA and decisions

16. Decision QoI

17. Decision making utility

19. Mission abstraction • Priors on encounters and conditions enable definition of a traffic and environment generator • Intelligence services formulated and composed • Situational awareness maintained by an abstraction of the fusion functions to map intelligence products to SA upgrades • Decisions taken by recognizing SA patterns • Actions pursued leading to feedback to the mission physics

20. Abstracting space • QoI emission characteristics constrain asset selections and operational modes • Regions of validity of service output can be defined • Optimization requires amortization over mission projections

21. Amortizing costs • States can be defined for a composite sensing service in which measures of interest conform to appropriate invariants • A specific combination of assets is active • Battery energy consumption is approximately constant • Personnel are at definable risk

22. Conclusions=(question,T).Conclusions; • We have shapes for UoI comparison • We can make those shapes from timed stochastic process models • Those models can also estimate QoI • The models are a link between QoI and UoI • VoI that is subjective because of a situational awareness horizon may be found by, for example, marginalizing the dependency structures found in the equilibrium and transient solutions to the models • When we manage to work in human factors, we’ll have a handle on heuristically subjective VoI • I hope this was an absorbing talk