Evolving Reactive NPCs for the Real-Time Simulation Game

1. Evolving Reactive NPCs for the Real-Time Simulation Game IEEE Symposium on Computational Intelligence and Games

2. Outline • Motivation • Objective • Introduction • The game: Build & Build • Basic behavior model • Co-evolutionary behavior generation • Experiment and Results • Discussion • Conclusion • Personal Opinion

3. Motivation • AI in computer games has been highlighted in recent, but manual works for designing the AI cost a great deal.

4. Objective • Designing NPCs’ behaviors without relying on human expertise.

5. Basic behavior model • Two different grid scales are used for the input of the neural network such as 5×5 and 11×11. The game: Build & Build random action probability: 0.2 five neural networks are used to decide whether the associating action executes or not.

6. Co-evolutionary behavior generation • We use the genetic algorithm to generate behavior systems that are accommodated to several environments.

7. Experiment and Results • 5×5 obtains lower winning averages for complex environment, while it performs better when the environment is rather simple.

8. Introduction • It is challengeable for many researchers to apply AI to control characters. (AI produce more complex and realistic games.) • Finite state machines and rule-based systems are the most popular techniques in designing the movement of characters. • While neural networks, Bayesian network, and artificial life are recently adopted for flexible behaviors. • Evolution generates useful strategies automatically. • This paper proposes a reactive behavior system composed of neural networks is presented, and the system is optimized byco-evolution.

9. Rule based approach • AI of many computer games is designed with rules based techniques such as finite state machines (FSMs) or fuzzy logic. • FSMs have a weak point of its stiffness; however, the movement of a character is apt to be unrealistic. • there is a trend towards fuzzy state machine (FuSM).

10. Adaptation and learning: NNs, EAs, and Artificial life • The adaptation and learning in games will be one of the most major issues making games more interesting and realistic. • Neural network, and evolutionary algorithms (e.g. genetic algorithm) are promising artificial intelligence techniques for learning in computer games. • NN - is badly trained • GE - required too many computations and were too slow to produce useful results.

11. Co-evolution • By simultaneously evolving two or more species with coupled fitness. • Superior strategies for an environment have been discovered by co-evolutionary approaches.

12. Reactive behavior • Reactive model performs effectively since it considers the current situation only. • Neural networks and behavior-based approaches are recently used for the reactive behavior of NPCs keeping the reality of behaviors.

13. The game: Build & Build • ‘Build & Build’ developed in this research is a real-time strategic simulation game, in which two nations expand their own territory. • Each nation has soldiers who individually build towns and fight against the enemies, while a town continually produces soldiers for a given period.

14. The game: Build & Build

15. Designing the game environment • The game starts two competitive units in a restricted land with an initial fund. • The units are able to take some actions at the normal land but not at the rock land. • A unit can build a town when the nation has enough money, while towns produce units using some money.

16. Designing the game environment(cont.)

17. Designing NPCs • NPC can move by 4 directions as well as build towns, attack units or towns, and merge with other NPCs. • The attack actions are automatically executed when an opponent locates beside the NPC.

18. Designing NPCs (cont.)

19. Designing NPCs (cont.)

20. Basic behavior model(cont.) • Two different grid scales are used for the input of the neural network such as 5×5 and 11×11.

21. Basic behavior model(cont.) • In order to actively seek a dynamic situation, the model selects a random action with a probability (in this paper, a = 0.2) in advance. five neural networks are used to decide whether the associating action executes or not.

22. Co-evolutionary behavior generation • We use the genetic algorithm to generate behavior systems that are accommodated to several environments. • Two pair-wise competition patterns are adopted to effectively calculate the fitness of an individual.

23. Co-evolutionary behavior generation (cont.) • The fitness of an individual is measured by the scores against randomly selected M opponents.

24. Experiment and Results • Four different battle maps => demonstrate the proposed method in generating strategies adaptive to each environment.

25. Experiment and Results (cont.) • The case with 11×11 shows more diverse behaviors than that with 5×5, since it observes information on a more large area. • 5×5 obtains lower winning averages for complex environment, while it performs better when the environment is rather simple.

26. Experiment and Results (cont.) The 11×11 shows the better performance than the 5×5, since it considers more various input conditions so as to generate diverse actions. Fig. 8. Winning rate between 5×5 behavior and 11×11 behavior at each generation on map type 3.

27. Experiment and Results (cont.) • For the plain map, 5×5 behavior system shows a simple strategy that tries to build a town as much as possible. Building a town leads to generate many NPCs so as to slowly encroach on the battle map as showns in Fig. 9.

28. Discussion • The reactive system shows good performance on simple environments like the plain map, but it does not work well for complex environments. • Also, the amount of input information is important for the reactive system when the environment is not simple.

29. Conclusion • A reactive behavior system was presented for the flexible and reactive behavior of the NPC. • Co-evolutionary approaches have shown the potentialities of the automatic generation of excellent strategies corresponding to a specific environment.

30. Personal Opinion • Strength • Designing NPCs’ behaviors without relying on human expertise. • Weakness • the limitation of direction • Application • real-time strategic simulation game