Constructing Complex NPC Behavior via Multi-Objective Neuroevolution

1. Constructing Complex NPC Behavior via Multi-Objective Neuroevolution Jacob Schrum – schrum2@cs.utexas.edu Risto Miikkulainen – risto@cs.utexas.edu University of Texas at Austin Department of Computer Sciences

2. Introduction • Goal: discover NPC behavior automatically • Benefits • Save production time/effort • Learn counterintuitive behaviors • Find weaknesses in static scripts • Tailor behavior to human players • Challenges • Games are complex • Multi-modal behavior is required • Success often depends on multiple objectives • RL & Evolution popular approaches • Current methods do not explicitly consider multiple objectives!

3. Multi-Objective Optimization • Exceptional performance in any objective is important • Imagine game with two objectives: • Damage Dealt • Health Remaining • A dominates B iff A is strictly better in one objective and at least as good in others • Population of points not dominated are best: Pareto Front High health but did not deal much damage Tradeoff between objectives Dealt lot of damage, but lost lots of health

4. NSGA-II • Evolution: natural approach for finding optimal population • Non-Dominated Sorting Genetic Algorithm II* • Population P with size N; Evaluate P • Use mutation to get P´ size N; Evaluate P´ • Calculate non-dominated fronts of {P È P´} size 2N • New population size N from highest fronts of {P È P´} *K. Deb et al. 2000

5. Neuroevolution • Genetic Algorithms + Artificial Neural Networks • NNs good at generating behavior • GA creates new nets, evaluates them • Three types of mutations (no crossover used) Perturb Weight Add Connection Add Node

6. Battle Domain 15 NPCs attack player Player Bat Infinite Plane

7. NPC Fitness Objectives • NPCs have several goals • Inflict damage as a group → Attack Assist Bonus • Avoid damage individually → Damage Received • Stay alive individually → Time Alive • Objectives are contradictory and distinct NPCs take damage from bat Player is knocked back by NPC

8. Experiment • Multi-objective vs. Single-objective • Single-objective uses weighted sum of z-scores • Evolve each method against static player strategies • Spinning strategy • Alternating strategy • Chasing strategy • NPCs must beat previous player strategy to progress

9. Results Differences for Alternating and Chasing significant with p < .05

10. Evolved NPC Behaviors • Single-Objective • Cannot find balance between attack and defense • Behavior is ineffectual • Multi-Objective • Baiting Behavior • One agent acts as bait • Others attack player on blind side • Charging Behavior • Try to dodge bat while rushing in • Maximizes damage dealt

11. Discussion • Benefits • Multi-modal behavior • Exploits dynamics of environment • Individuals can have different behaviors • Future Work • Evolve against humans • Learn many behaviors in advance • More complex games: Unreal Tournament

12. Conclusion • Multi-objective encourages multi-modal behavior • Multi-objective better than single-objective • Deals with contradictory objectives • Leads to several different but effective solutions • Lead to multi-modal behavior in future games

13. Questions? See movies at: http://nn.cs.utexas.edu/?multiNPCs E-mail: schrum2@cs.utexas.edu