Artificial Intelligence in Game Development: Non-Player Character (NPC) Behavior Simulation

1. Yingcai Xiao • Yingcai Xiao Artificial Intelligence in Game Development

2. Non-player character (NPC) / non-person character / non-playable character: any game character that is not controlled by a player.AI in games controls NPCs to simulate intelligence behavior. Artificial Intelligence in Game Development

3. First AI games created in 1951.NIM and CheckerPac-Man (1980) added personalities to enemies. https://www.youtube.com/watch?v=x4tqMv0CvTQGarry Kasparov defeated by IBM's Deep Blue computer in 1997https://www.youtube.com/watch?v=_Y9hriLylIo Examples

4. InteractionInteraction with Natural LanguageJeopardyhttps://www.youtube.com/watch?v=WFR3lOm_xhEAPIhttp://www.ibm.com/smarterplanet/us/en/ibmwatson/?lnk=buwaCloudhttp://www.ibm.com/smarterplanet/us/en/ibmwatson/watson-cloud.html IBM Watson

5. Examples • Halo: Intelligent aliens will retreat after their leader killed. (Jobs) • Sims: smart objects that can go hungry, bored, tired, ... • TechnoSphere: an online digital environment where users from around the globe could create creatures and release them into. • Black & White: belief-desire-intention, god overseeing little people.

6. AI Algorithms

7. AI Algorithms • LOS (line of sight): avoid of being shot. • Graph Theory and Routing: maze games. • BF/DF Searching Algorithms: board games. • FSM (Finite State Machine): state of mined games.

8. AI Algorithms • Genetic Algorithms: evolutionary computing based on adaptation and survival. • http://www.biomanbio.com/GamesandLabs/Genegames/genetics.html • http://www.biomanbio.com/GamesandLabs/Cellgames/sCellExplorerAnimalCell.html • (Arrow keys to move, “s” to shoot an object for explanation.)

9. AI Algorithms • Decision Trees: hierarchical graph to make decisions based on conditions. • Fuzzy Logic: make decisions based on vague information.

10. John von Neumann (The Martians) Computer Architecture https://en.wikipedia.org/wiki/Von_Neumann_architecture Game Theory https://en.wikipedia.org/wiki/Von_Neumann_architecture

11. AI Algorithms • Cellular Automata: a grid of cells with each cell value being updated constantly by its neighbor’s values. • Game of Life • Stephen Wolfram • Wolfram Alpha

12. AlphaGo • AlphaGo: a game engine that plays the Go game. • Go: aChinese board game of strategies. 19x19, 361!, 10761 (total number of fundamental particles in the observable universe:1085). • The first computer program to beat a professional Go player (4:1, March 8-15, 2016).

13. AlphaGo Zero • AlphaGo: learns from past human games. • AlphaGo Zero: learns by itself. • AlphaGo Zero won AlphaGo 100:0

14. Flocking

15. Flocking • Flocking: crowd behaviors. • Starling murmuration: https://www.youtube.com/watch?v=eakKfY5aHmY • Reynolds, Craig W. (1987). "Flocks, herds and schools: A distributed behavioral model.". ACM SIGGRAPH Computer Graphics 21 (4). pp. 25–34. • Batman Returns (1992): flocking bats and The Lion King (1994): wildebeest stampede.

16. Flocking • Craig Reynolds: http://www.red3d.com/cwr/ • Reynolds, Craig W. (1987). "Flocks, herds and schools: A distributed behavioral model.". ACM SIGGRAPH Computer Graphics 21 (4). pp. 25–34. • Authored the Open Steer library • Won 1998 Academy Scientific and Technical Award for pioneering contributions to 3D computer animation for movies. • Uses procedural models to simulate complex natural phenomenon.

17. The Flocking Algorithm • The algorithm • Implementation in Unity3D • Building a Unity package for reuse • Building a flocking library in a game engine

18. The Flocking Algorithm • Craig Reynolds’ Boids: • artificial objects follow natural flocking behavior • Unity Implementation: • http://black-square.github.io/BirdFlock/ • JavaScript implementation: http://gpolo.github.io/birdflocking/ • OpenGL implementation: http://www.navgen.com/3d_boids/

19. The Flocking Algorithm • Tutorial: • http://www.vergenet.net/~conrad/boids/pseudocode.html • Unity Code Explanation: • http://wiki.unity3d.com/index.php?title=Flocking

20. The Flocking Algorithm • Craig Reynolds’ Boids: artificial objects flow natural flocking behavior. • 3 Basic Rules: • cohesion - to move toward the center of the mass of the flockmates • separation - to avoid flockmates • alignment – to steer towards the flock direction

21. Implementing Flocking

22. The Flocking Algorithm • Craig Reynolds’ Boids: artificial objects flow natural flocking behavior. • 3 Basic Rules: • cohesion - to move toward the center of the mass of the flockmates • separation - to avoid flockmates • alignment – to steer towards the flock direction • http://www.vergenet.net/~conrad/boids/pseudocode.html

23. init_game(); while(!done) { e = get_event() switch(e) { case “p”: change_game_parameters( ); break; … default: display_game_objects(); compute_next_frame(); } } Event Loop Event Event Mapping & Event Dispatching EDP Game Development Event Handler

24. init_game(); while(!done) { e = get_event() switch(e) { case “p”: change_game_parameters( ); break; … default: compute_next_frame(); display_game_objects(); } } Event Loop Event Event Mapping & Event Dispatching EDP Game Development Event Handler

25. init_game(); create_boids(); while(!done) { e = get_event() switch(e) { case “p”: change_game_parameters( ); break; … default: compute_CoM_AV(); compute_boid_velocity(); display_boids(); } } Event Loop Event Event Mapping & Event Dispatching EDP Game Development Event Handler

26. Flocking Code • C# code at http://wiki.unity3d.com/index.php?title=Flocking • Create the flock at the BiodController::Start() • //all boids • Update animation parameters in BiodController::Update () //all boids • Control the animation in BoidFlocking::BoidSteering () //each boid

27. Flocking Code • Create the flock at the Start() • Flock Size • Boids Array • Position • Direction

28. Flocking Code • Create the flock at the Start() • Bounding box • Global and local transformation • The Controller

29. Flocking Code • (2) Update animation parameters in Update () • Center of Mass • Average Direction • The Controller

30. Flocking Code • (3) Control the animation in BoidFlocking::BoidSteering () • This is the script for each boid. • Separation is taking care of by rigidbody. • Cohesion is to follow the CoM. • Alignment is to follow the Average Direction

31. PA3 Coding / Flocking http://www.cs.uakron.edu/~xiao/game/igd3.htm The Theory / Rules: cohesion - to move toward the center of the mass of the flockmates separation - to avoid flockmates alignment – to steer towards the flock direction

32. PA3 Coding / Flocking The Example: Holistic Game Development with UnityBy: Penny de Byl http://proquest.safaribooksonline.com/book/programming/game-programming/9780240819334 Section 5.6: a flock of seagulls flying against wind in small groups. http://proquest.safaribooksonline.com/book/programming/game-programming/9780240819334/chapter-5-character-mechanics/ch5_6_006_9780240819341_web_ch05_html

33. Yingcai Xiao Sorting and Searching in Game Development

34. Sorting & Searching • S&S needed to develop AI games to find optimal strategies, paths, in the game logic. • The main challenge is the huge size of the search tree. The number of branches in the search tree of a 19x19 Go board game is larger than the number of atoms in the known universe. • The key of success is to reduce the search space but not the optimal solutions.

35. Sorting • Sorting the search space can make searches faster. • Sorting can be used to prioritize the search space. • Weights can be added to reflect heuristic information.

36. Basic Sorting Algorithms • Sorting Algorithm Animations • http://www.sorting-algorithms.com/ • https://www.cs.usfca.edu/~galles/visualization/ComparisonSort.html • https://www.youtube.com/watch?v=aXXWXz5rF64 • Wiki listing • https://en.wikipedia.org/wiki/Sorting_algorithm • eights can be added to reflect heuristic information.

37. Searching • Basic searching • Statistical searching • Searching for globally optimal solutions

38. Basic Searching Algorithms Code Listing https://users.dcc.uchile.cl/~rbaeza/handbook/search_a.html Comparisons http://bigocheatsheet.com/ http://research.cs.queensu.ca/home/cisc121/2006s/webnotes/search.html Example: How many steps are needed to find the one over-weighted golf ball among the eight of them using just a scale.

39. Statistical Searching Algorithms https://en.wikipedia.org/wiki/Monte_Carlo_tree_search “The focus of Monte Carlo tree search (MCTS) is on the analysis of the most promising moves, expanding the search tree based on random sampling of the search space. The application of Monte Carlo tree search in games is based on many playouts. In each playout, the game is played-out to the very end by selecting moves at random. The final game result of each playout is then used to weight the nodes in the game tree so that better nodes are more likely to be chosen in future playouts” https://en.wikipedia.org/wiki/Search_tree https://en.wikipedia.org/wiki/Monte_Carlo_method https://en.wikipedia.org/wiki/Monte_Carlo_method#History https://en.wikipedia.org/wiki/Monte_Carlo_Casino

40. MCTS • Four Steps: • Selection • Expansion • Simulation • Backpropagation

41. MCTS • Selection • Start from root R and select successive child nodes down to a leaf node L. The section below says more about a way of choosing child nodes that lets the game tree expand towards most promising moves, which is the essence of Monte Carlo tree search.

42. MCTS https://commons.wikimedia.org/wiki/File:MCTS_(English).svg#/media/File:MCTS_(English).svg

43. MCTS Expansion Unless L ends the game with a win/loss for either player, either create one or more child nodes or choose from them node C.

44. MCTS https://commons.wikimedia.org/wiki/File:MCTS_(English).svg#/media/File:MCTS_(English).svg

45. MCTS Simulation Play a random playout from node C.

46. MCTS https://commons.wikimedia.org/wiki/File:MCTS_(English).svg#/media/File:MCTS_(English).svg

47. MCTS Backpropagation Use the result of the playout to update information in the nodes on the path from C to R.”

48. MCTS https://commons.wikimedia.org/wiki/File:MCTS_(English).svg#/media/File:MCTS_(English).svg

49. AlphaGo Algorithms • Based on tree searches and neural networks that can learn. • Policy Network: only consider a few promising positions (limit the breadth of the search tree.) • Value Network: only consider a few steps deep (limit the depth of the search tree).

50. Globally Optimal Solutions “Simulated annealing (SA) is a probabilistic technique for approximating the global optimum of a given function. ” “Specifically, it is a metaheuristic to approximate global optimization in a large search space” “for problems where finding the precise global optimum is less important than finding an acceptable local optimum in a fixed amount of time, simulated annealing may be preferable to alternatives such as brute-force search or gradient descent.” https://en.wikipedia.org/wiki/Simulated_annealing