Evolving Agents in a Hostile Environment

1. Evolving Agents in a Hostile Environment Alex J. Berry

2. Overview • Motivation • Background • The Approach • Map Generation • Evolutionary Algorithm • Experiments

3. Training First Responders • VEnOM Labs is developing a suite to train First Responders • Is the training effective? • How can we make the training more effective?

5. Goal • To develop a system to allow for friendly and hostile AI agents in the training environment. • To develop a system to create better agents for training First Responders.

6. Simulation of Adaptive Agents in a Hostile Environment[HW95] • Thomas Haynes • Simple Agents • Mines and Energy • Experiments • Single Agent, Static and Random Environment • Multiple Agent, Static and Random Environment

7. The Complicator • Aliases: Dr. T, Dr. Tauritz • Input: 2+2=4 • Output:

8. The Approach • Randomly Generated Grid Environment • Three Types of Agents: • First Responders • Terrorists • Victims • Genetic Programming to Evolve the Agents

9. Randomly Generated Maps • Any Dimension • Percentage walls • Bit Array to Hold the Data

10. Demo

11. Victims Move Randomly Remember Things Forget Things Survive Terrorists Kill Victims Kill First Responders Lay Traps Disguise Themselves Not Get Caught First Responders Help Victims Find and Disarm Traps Survive Catch Terrorists What’s an Agent to do?

12. Evolutionary Algorithm • Two Agents to Evolve • First Responder • Terrorist • Two Competing Evolving Populations • Genetic Programming for the Evolutionary Implementation

13. Terminals Current Grid Location (C) Surrounding Grid Locations (S) Rand (R) Non-Terminals If-Then-Else Threat And, Or, Not Victim, First Responder, Terrorist, Trap Valid Move Actions Save Kill Move Place Trap Remove Trap Not (Action) to invert an Action What An Individual Looks Like

14. Sample Individuals Move

15. First Responder Victims Helped Terrorists Caught Traps Found Traps Removed Survival Amount of the Map explored Terrorist Kills using Traps Kills on Contact Effective Disguises Amount of Time Survived Genetic Programming Evaluation

16. Experiments • Static Environment Evolution • Random Environment Evolution • Varying Ratios of First Responders, Victims, and Terrorists • Evolving one Population at a Time

17. Summary • Looking at agents operating in a hostile environment. • First Responders, Terrorists, and Victims • Evolving first responders and terrorist using genetic programming techniques.

18. Future Work and Questions • Other Evolutionary Approaches • LCS • GP-LCS Hybrid • Integration into a 3D environment • Playable Human Mode • Representations of Real Buildings • Test effectiveness of adding this to Affective Intensity Experiment • Integration of other types of Traps and sensors to detect those traps