Kai Virtanen, Janne Karelahti, Tuomas Raivio, and Raimo P. Hämäläinen Systems Analysis Laboratory

1. A Multistage Influence Diagram Game for Maneuvering Decisions in Air Combat Kai Virtanen, Janne Karelahti, Tuomas Raivio, and Raimo P. Hämäläinen Systems Analysis Laboratory Helsinki University of Technology

2. t=Dt t=Dt ¼ t=0 t=0 Maneuvering decisions in one-on-one air combat ¼ Outcome depends on all the maneuvers of both players Þ Dynamic game problem Objective Find the best maneuvering sequences with respect to the overall goals of a pilot! - Preference model - Uncertainties - Behavior of the adversary - Dynamic decision environment

3. Influence diagram (Howard, Matheson 1984) • Directed acyclic graphs • Describes the major factors of a decision problem • Offers several possibilities for quantitative analysis Time precedence Informational arc Alternatives available to DM Decision Random variables Conditional arc Chance Probabilistic or functional dependence Deterministic variables Conditional arc Deterministic A utility function Conditional arc Utility

4. Influence diagram (continued) • State of the world is described by attributes • States are associated with • Utility • Probability • Utility is a commensurable measure for goodness of attributes • Results include probability distributions over utility • Decisions based on utility distributions • Information gathering and updating using Bayesian reasoning

5. Decision theoretical maneuvering models • Single stage influence diagram (Virtanen et al. 1999): • Short-sighted decision making • Multistage influence diagram (Virtanen et al. 2004): • Long-sighted decision making • Preference optimal flight path against a given trajectory • Single stage influence diagram game (Virtanen et al. 2003): • Short-sighted decision making • Components representing the behavior of the adversary New multistage influence diagram game model: • Long-sighted decision making • Components representing the behavior of the adversary • Solution with a moving horizon control approach

6. Influence diagram for a single maneuvering decision Adversary's Present State Adversary's Maneuver Adversary’s State Measurement Combat State Present Measurement Present Combat State Situation Evaluation Present State State Maneuver Present Threat Situation Assessment Threat Situation Assessment

7. Multistage influence diagram air combat game White Black • Goals of the players: • 1. Avoid being captured by the adversary • 2. Capture the adversary • Four possible outcomes • Evolution of the players’ states described by a set of differential equations, a point mass model • Evolution of the probabilities described by Bayes’ theorem

8. Situation Evaluation at t-1 Situation Evaluation at t Cumulative expected utility Situation Evaluation at t+1 Situation Evaluation at t-2 Situation Evaluation Graphical representation of the game Black’s viewpoint Combat state White's viewpoint stage t-1 stage t

9. Sketch of geometry Threat situation assessment • Infers the threat situation from the viewpoint of a single player • Discrete random variable, four outcomes: • Neutral • Advantage • Disadvantage • Mutual disadvantage • Probabilities are updated with Bayes’ theorem: Pposterior( outcome | combat state) ∞ Pprior( outcome ) X Plikelihood( combat state | outcome ) • Each outcome leads to a specific goal described with a utility function

10. Terminate? Moving horizon control approach Players’ states at stage t Truncated influence diagram game lasting stages t, t+Dt,…, t+KDt Dynamic programming KDt = length of the planning horizon Game optimal control sequences over stages t, t+Dt, …, t+KDt t:=t+Dt • Resulting game optimal controls • the cumulative expected • utility is maximized • approximative feedback • Nash equilibrium Players’ states at stage t+Dt Implement the controls of stage t

11. Numerical example • Black initially pursuing White • White’s aircraft more agile • White wins • Look-ahead strategies: • one-step, solid lines, payoffs: White/Black = 1.21 • two-step, dashed lines, payoffs: White/Black = 1.25 altitude, km White Black y-range, km x-range, km

12. Threat probability distributions Black White Probability Probability time, sec. time, sec.

13. Effects of the likelihood functions • Threat probability rate of change defined by the likelihood functions • Steep likelihood functions: • Evolution of threat probabilities is sensitive to certain changes in combat state => Outcomes are distinguished sharply Black, flat likelihoods White, steep likelihoods

14. Conclusions • The multistage influence diagram game: • Models preferences under uncertainty and multiple competing objectives in one-on-one air combat • Takes into account • Rational behavior of the adversary • Dynamics of flight and decision making • The moving horizon control approach: • Game optimal control sequences w.r.t. the preference model of the players • Utilization: • Air combat simulators, a good computer guided aircraft • Contributions to the existing air combat game formulations: • New way to treat uncertainties in air combat modeling • Roles of the players are varied dynamically

15. References • Howard, R.A., and Matheson, J.E., “Influence Diagrams,” The Principles and Applications of Decision Analysis, Vol. 2, edited by R.A. Howard and J.E. Matheson, Strategic Decision Group, Palo Alto, CA, 1984. • Virtanen, K., Raivio, T., and Hämäläinen, R.P., “Decision Theoretical Approach to Pilot Simulation,” Journal of Aircraft, Vol. 36, No. 4, 1999. • Virtanen, K., Raivio, T., and Hämäläinen, R.P., “Influence Diagram Modeling of Decision Making in a Dynamic Game Setting,” Proceedings of the 1st Bayesian Modeling Applications Workshop of the 19th Conference on Uncertainty in Artificial Intelligence, 2003. • Virtanen, K., Raivio, T., and Hämäläinen, R.P., “Modeling Pilot's Sequential Maneuvering Decisions by a Multistage Influence Diagram,” Journal of Guidance, Control, and Dynamics, Vol. 27, No. 4, 2004. • Kai’s dissertation available at www.sal.hut.fi/Personnel/Homepages/KaiV/thesis/