Wright State University Biomedical, Industrial & Human Factors Eng. Search Theory Optimization: Agent Models and th

1. Wright State UniversityBiomedical, Industrial & Human Factors Eng. Search Theory Optimization:Agent Models and the Bay of Biscay Raymond Hill Research sponsored in part by:

2. Purpose • Update with project results of DMSO/AFRL sponsored research conducted via • AFIT Operational Sciences Department • WSU BIE Department • Talk some about agent models • Cover the optimization results • Cover the game theory results • Cover the search theory results • Future directions (I hope)

3. Quick Background on Project • Lots of interest in agent models • Project Albert work • Brawler modeling work • Next Generation Mission Model • Other agent model work as well • Adaptive interface agents • Intelligent software agents • Internet agents • Challenge is how to bring agent models into the higher level models?

4. Why A Higher Level Modeling? • Need to better capture command and control effects • Need to capture “intangibles” • Need to model learning based on battlefield information • Need better representation of actual information use versus perfect use • Agents and agent models hold promise but bring along many issues

5. Agent Modeling Challenges • Output analysis • Particularly with more complex models and models that are not necessarily replicable • Accurate human behavior modeling • In particular, command behavior modeling • Level of fidelity in model • Beyond that of bouncing dots found in the complex adaptive systems work • Interaction of agents and legacy modeling approaches • Brawler extensions into theater and campaign level modeling

6. Agent Modeling Challenges (cont). • Human interaction with the models • The visual impact of interactions among the agents • “What if” analyses when human behavior is being modeled • Quantifiable aspects of generally qualitative types of output • Verification and Validation of the behaviors embedded within the model

7. The Project • Needed a “use case” for agent models • Dr McCue’s book great example of operational analysis • Bay of Biscay scenario amenable to agent modeling • Lots of information available • Formed an ideal basis for subsequent research

8. Efforts Completed • Capt Joe Price (masters thesis) • Game theory focus • Capt Ron “Greg” Carl (masters thesis) • Search theory focus • Entering PhD this fall at Purdue • Subhashini Ganapathy • Simulation optimization study • PhD Candidate • Maj Lance Champagne, Ph.D. • Dissertation completed March 2004

9. Snapshot of AFIT Model

10. Methodology - Game Portion • Allied search strategies • When to search? Day versus night? • German U-boat surfacing strategies • When to surface? Day versus night? • Two-person zero-sum game • Players: Allied search aircraft and German U-boats • Met rationality assumption • Non-perfect information • Neither side knows the exact strategy the other uses • Objective is number of U-boat detections • Allied goal: maximize • German goal: minimize • Zero-sum game

11. Game Formulation • Allies: two pure search strategies • Only day and only night • Germans: two pure surfacing strategies • Only day and only night • Next step to include mixed strategies • Let parameter range from 0 to 1 as strategy • More interesting than simple pure strategy • Still more interesting with adaptation • Simple adaptation algorithm • Agents allowed to adapt strategy each month

12. Results – No Adaptation • Response Surface Methodology model • Adjusted R2 = 0.947 Equilibrium Point, 0.7, 0.54

13. Adaptation Experiment • Both sides can adapt strategies (simple model) • Three design points chosen: • Adaptation occurs every month • Investigate results • 20 replications; 12-month warm-up; 12 months of statistics collection (April 1943 – February 1944)

14. Adaptation Convergence

15. Adaptation Convergence

16. Methodology - Search Portion • Design data compiled according to hierarchy • Historical fact • Published studies • Data derived from raw numbers • Good judgment • MOE is number of U-boat sightings • U-boat density constant between replications • Aircraft flight hours same between replications • Therefore, sightings = search efficiency • Two cases; search regions don’t overlap, do overlap

17. Non-overlapping Search Regions 200 NM2 350 NM2

18. Overlapping Search Regions 100 NM2 100 NM2

19. Aircraft Search Patterns Square Search Sector Search Barrier Search Creeping Line Search Parallel Line Search

20. Non-overlapping Search Regions Means Comparison—All Pairs (20 Iterations) (Similar Letters Indicate Statistical Equivalence)

21. Non-overlapping Search Regions Means Comparison—All Pairs (30 Iterations) (Similar Letters Indicate Statistical Equivalence)

22. Overlapping Search Regions Means Comparison—All Pairs (30 Iterations) (Similar Letters Indicate Statistical Equivalence)

23. Simulation Optimization Portion • Simulation-based optimization is the process of finding the best input variable from all possibilities without explicitly examining each possibility • Often involves the use of some search heuristic “wrapped” around the simulation • The simulation becomes the evaluation function • The heuristic sends potential solutions to the evaluation function • The returned evaluations are then used to update the search and eventually return a high-quality, possibly optimal solution

24. Graphic of Sim.-Based Opt. Optimization Module (for the examples: Max Targets Found s.t. non-linear and stochastic variables) Applies Generalized Reduced Gradient Method (search for alternatives along curves of the feasible region) Simulation Module (emulates the system being studied by representing the entities & behavior)

25. The Entities, States and Events • Bay of Biscay • U-Boats traveling under the sea • U-Boats traveling on the surface • Aircraft in search of U-Boats • Aircraft attacking U- Boats • Sunk U-Boats • Starting Point of Aircraft • Refuel • Reloading of Ammunitions • Take-off point • Starting Point of U-Boats • Refuel • Repair U-boats in Atlantic

26. Allied Base Allied Aircrafts Submerged U-boats WSU Simulation Interface

27. Objective Function Used • Improve the efficiency of search, in terms of number of U-Boats sunk • Nsunk= f (altitude, range, speed, flying effort) Where Nsunk is the number of U-Boats sunk. • Nsunk expressed as a function of : • Operational sweep rate • Cost of flying (maintainability, serviceability) • Speed of the aircraft

28. Constraints Employed • Number of allied aircraft available for the mission • Limited maintenance resources and service available to support sortie generation • Aircraft characteristics • Fuel • Schedule maintenance interval • Maximum speed • Range of detection • Altitude • Munitions

29. Results of Search Effort

30. VV&A Portion • Verification • Did you build the model right? • Have you accurately translated the conceptual model? • Debugging is part of verification • Validation • Did you build the right model? • Is the model an accurate representative of system • A function of study objectives • Not a lot has been done on object-oriented and agent-based models

31. So What? • The Bay of Biscay models were built to represent the historical results faithfully so that “what if” analyses could proceed • Comparing simulation results to actual results is not a new task, rather it is a pretty fundamental approach to validation • However, in the case of historical combat data are limited • There are is no such thing as a “do over” • Consider one Scenario 1, October 1942-March 1943, and use as measure, sightings

32. Scenario 1 – Sightings • The historical data on the U-Boat sightings is available for the period being modeled • There is one observation per month modeled Ref: Brian’s book

33. Scenario 1 – Simulated Sightings

34. Simple Statistical Comparison • The confidence interval from the simulation captures the data point from history

35. Aggregate the Monthly Data • The overall confidence level is near 0!

36. New Test for Validation • Efron (1979) first proposed the concept of re-sampling • Well accepted technique • Particular use in this test is somewhat different • Note use of bootstrap to create multiple samples for subsequent comparative uses • Then employ the sign test, a non-parametric test, as a means to compare the real and the simulated data

37. Historical-Based Bootstrap • We now compare these values to the simulation values previously provided

38. Bootstrap Results

39. Future Applications • Generalized architecture promotes re-use • Coast Guard Deep-water efforts • Air Force UAV search in rugged terrain or urban environments • Human-in-the-loop issues permeate • Search and rescue using UAVs • Reconnaissance using UAVs • Combat missions using UCAVs • Much more that can be done in VV&A

40. Future and Ongoing Efforts • Wish to extend the game theory aspects • Would like to do more with the search theory • Examining policy adaptation in multi-cultural, adversarial scenarios • Examining employment of agent-based modeling methods for use in planning and re-planning • Examining the use of distillations as a means to providing real-time decision support to planners

41. Questions?