Simulation and Evolution Work Well Together

1. Simulation and Evolution Work Well Together Lawrence “David” Davis VP of Product Research

2. Terminology What is simulation? What are evolutionary algorithms? Some Case studies: Investigating contacts Target allocation Army/NASA cockpit procedures Interpreting data Conclusions Topics

3. Simulation Monte Carlo Simulation Evolutionary Algorithm Genetic Algorithm Heuristic Evaluation Procedure Terminology

4. Simulation involves reproducing events at the level of detail we care about Can be done at a fine level of detail (agent-based modeling) Can be done at a higher level Often has unexpected outcomes Should include what we care about and finesse the rest About Simulation

5. Simulation: doesn’t change Our strategy: Adapts

6. Simulations represent interactions that we can’t capture in other ways Highly detailed effects “Cascading” effects Probabilistic effects Statistical reports are possible Why Evaluate with Simulations?

7. Genetic algorithms—evolution simulated on a computer We “evolve” the solutions to hard problems instead of figuring them out Good for problems where mathematical techniques can’t be applied Good when we need a reasonable answer fairly quickly Really good when used to find rule sets or strategies that do well under simulation What are Evolutionary Algorithms?

8. We have unidentified contacts in the ocean We have different types of assets available (towed sensor arrays, underwater vehicles, rulees, etc) When we have a contact, we want to allocate assets to investigate it Success means determining what the source of the contact was, and continuing to monitor it if it interests us Case Study: Investigating Contacts

9. The area the contact could be in increases in size with time Different assets may work well together or may hamper each other (underwater vehicles can hinder surface listening devices) We need to be able to investigate other contacts if they occur, so we might not want to allocate all our assets to any one contact Investigating Contacts:Features of the Problem

10. We can model the arrival of contacts probabilistically When contacts occur, they modify the probabilities of other contacts When we learn about the contacts, this modifies our view of the probabilities Some contacts don’t represent interesting things Some contacts are extremely interesting Features of the Simulation

11. Find real contact sources at a high rate of success Investigate multiple contacts with a high rate of success Minimize cost of operations, and/or number of assets used We want to…

12. The simulator generates events with probabilities based on our experience It includes algorithms for computing success rates at finding event sources It includes algorithms for changing the size of the search area with time The simulator measurements of success are sensitive to weather, day/night, season, asset combinations, type of source, etc. What the Simulator is Like

13. A driver that steps us forward in time An event interpreter that creates events based on the input probabilities Objects of various types that can interact: assets, sources, weather events, and equipment A statistics gatherer that tracks success rates and other data that interests us What’s in the Simulator

14. There is currently an unidentified contact (a submarine) at location 1 Assets are allocated to investigate the contact, using the current allocation and search rules The simulator knows the course of the submarine The simulator increases the probability of other contacts related to this source along its course If a source of this type generally travels alone, the probability of other contacts of its type is reduced for some time Example of a Simulator Event

15. There is currently an unidentified contact (a fishing boat) at location 2 Assets are allocated to investigate the contact, using the current allocation and search rules The simulator knows the course of the fishing boat The probability of other contacts related to the boat along its course is increased The probability of identifying the type of source through radio and more detailed monitoring is computed Another Example of a Simulator Event

16. “If no other contacts are live and this contact is within 200 miles of base, send the slow but sensitive assets to investigate” “Don’t send towed arrays and underwater assets to investigate the same contact” “If there are three contacts in a straight line, concentrate search in the area on the projection of that line” Example Rules for Asset Allocation

17. Start with randomly-generated rule sets, or rule sets that represent human heuristics Evolve better and better rule sets Simulate months or years of activity to evaluate a rule set Use the desired features of the problem to decide which are the good rule sets and which are the bad ones Make more rule sets, but let the good ones proliferate more than the bad ones Mutate and cross-breed rule sets How to Get Good Rule Sets

18. The system, evolving rule sets that function well in the context of the simulator, produces a rule set that works well for the kinds of contacts we are simulating Sometimes these rule sets can have unexpected features Mathematical techniques aren’t well suited to find solutions in the context of simulations Evolutionary algorithms are very well suited for finding good procedures under evaluation by simulators The rule sets Get Better

19. Suppose you have a force faced with a group of approaching unfriendly objects How should you allocate fire in order to achieve your goals? Early decisions influence later ones Important targets should receive more attention Some interactions between weapon types are important: visual interference Distance effects matter, as does target change time, etc Case Study: Target Allocation

20. Important targets have a high probability of being eliminated Low probability of elimination of our force members Minimize duration of interaction Minimize expenditure of ammunition Minimize loss of crew How to Evaluate a Target Allocation Strategy

21. This problem can be handled just like investigating contacts, except that the contacts are all considered at the same time A simulation of the interaction is a good way to evaluate a blend of weaponry and a targeting strategy An evolutionary algorithm can be used to find good target allocation rule sets Target Allocation is Similar to Contact Investigation

22. Targets are aircraft Targets are boats Targets are mixed types Targets are far away and of unknown types We are moving We have time constraints Different Rule Sets for Different Types of Engagements

23. (rule for one type of gun) Target the incoming object with the highest combination of importance and residual hit probability (low visibility) Switch targets when probability of kill of the current target is greater than 96% Target the guns with the highest probability of kills first Example Rules for Target Allocation

24. Evolve a high-performance rule set by putting each candidate through a very large number of simulated engagements of the expected types, weighted by probability Evolve rule sets for different types of engagements by starting a different evolutionary process for each type, and creating rule sets that function well for that type of engagement Evolve different rule sets depending on the objectives: high survivability, high kill rate, deterrence, interdiction, etc. Evolve Good Rule Sets

25. A3I project (Army-NASA Aircrew Aircraft Integration) Also called MIDAS Simulated the effects of required procedures on cockpit crews (commercial aircraft and Apache helicopter crews) For commercial crews, simulated cockpit information systems and their effect in normal and emergency situations For helicopter crews, simulated effectiveness of mission procedures Case Study: NASA in-cockpit Procedures Studies

26. There is a truck convoy ahead Two helicopters are assigned to locate it and deliver a missile strike Pop-up and jinking procedures are used to do reconnaissance and evasion of ground-to-air missiles One pilot locates the target for the other Radio procedures, cognitive procedures, and situational awareness are modeled Simulation is critical in assessing the impact of different equipment and mission strategies Example of a Simulator Event

27. Measure pilot effectiveness through hundreds of thousands of mission simulations to find the best strategies Evolve cockpit displays to find those that give the highest levels of performance across hundreds of thousands of mission simulations System used with minor modifications for police emergency call stations and astronaut repair missions Evolution can be Used to Find Good Strategies and Displays

28. We get LOFARgrams from listening apparatus Some contacts may be whales or fishing boats Some may be large metallic fish Human experts can interpret the signals with high accuracy Humans tend to be best in the region and conditions where they were trained—Pacific, no storms, no whales in background, etc Case Study:Interpreting Data

29. Produce an expert system that can do what the humans do Big difficulty: identifying visual patterns that the humans see easily (“lines” in the data) Expert system techniques didn’t produce good results at line-tracing Development team used a genetic algorithm The Task

30. Hundreds of LOFARgrams were marked by humans so that the interesting lines were identified The genetic algorithm evolved rule sets for interpreting the data A rule was evaluated based on how well it matched the human analysis Over time, the system learned to do this as well as humans By changing the training cases, the system could learn to do this in different locations, conditions, and types of background noise How the Algorithm Worked

31. Simulation Monte Carlo Simulation Evolutionary Algorithm Genetic Algorithm Heuristic Evaluation Procedure Terminology

32. A Useful Extension Simulation: Strategies adapt Our strategy: Fixed

33. Simulations can be more accurate and informative than high-level or mathematical models of an event Probabilistic simulations show us what can happen under a wide variety of conditions Many interesting problems can be solved very well if we simulate, evaluate, and evolve Conclusions