Squad Tactics

1. Squad Tactics An overview of current and potential techniques for squad based tactics in games. Bryan Auslander bla204@lehigh.edu

2. Introduction • I am originally from New Jersey. • Fourth year CSB student. • Planning to be here a fifth year obtain a masters in computer science. • Spent the last 2 semesters working on a case based reasoning system (RetaliCase) for Unreal Tournament bots. (More on this later.)

3. Agenda • Introduction to squad tactics. • Brief overview of waypoints and squad tactics derived directly from these. • Decentralized Team AI based on emergent behavior. • Planned squad tactics through centralized squad leader/system. • RetaliCase development and overview. Halo 3 examples from developers and in game footage will be used throughout presentation.

4. Squad Tactics Introduction • Deviation between true AI systems and the illusion of intelligence is most evident. • Difficulty to program. • CPU time. • Unpredictable behavior. • Chris Butcher and Jamie Griesemer (designers on Halo) comment: • “If your looking for tips about how to make the enemies in your game intelligent we don’t have any for you we don’t know how to do that, but it sounds really hard.”

5. Squad Tactics in Sports Games • The Madden NFL is a widely praised learning squad AI. • 11 players on each team to coordinate. • Each player has different attributes. • Coordinate blocking to allow running plays. • Learns how to predict and counter players play choices. • Can choose a wide variety of offensive plays based on situation. Top: Madden 07 Bottom: Madden 08 Video: http://www.youtube.com/v/qNoxHk53uNw&rel=1

6. Madden AI to good? • Accused of cheating: • Game more difficult than previous ones. • AI seems to make miraculous comebacks. • Other ways Madden makes AI seem better. • Higher difficulties increase AI player attributes while decreasing player attributes.

7. First Person Shooters • Large area of focus for squad tactics. • Military applications. • Used to simulate Large dynamic and uncertain environment. • Limited information. • Imperfect communication. • Popular gaming category. • Halo • Rainbow 6 • Bio Shock.

8. Brief Intro to Waypoints. • Waypoints are commonly used in many games. • This is going to be covered extensively in another class • NPCs use them to navigate the environment. • Points are nodes in a node graph which is a representation of all paths NPC can travel. • Greatly simplifies AI movement. • Use path finding like A* to get around. • Nodes can be used for other applications besides just path finding. • Store precalculated visibility information. • Tactical value can be derived. Supreme Commander

9. Rainbow Six 3 Mission Planning

10. Squad Tactics Through Waypoints. • For entire node graph store visibility and connectivity as a bit string. • Used to find safe nodes. • Good attack positions. • Multiple team mates sharing information increases knowledge of good and bad nodes. • Coordinate Flanking.

11. Squad Tactics Through Waypoints: Pinch Points • In FPS need to anticipate opponent. • Pinch Points are places to launch an ambush. • Single exit from a room. • End of hallway. • Coordinated effort in a multiple exit room

12. Squad Tactics Through Waypoints: Issues • Node graphs can become large so may need to only store part of it at a time. • Processing issues. • Memory issues • More complex. • Requires proper node locations. • Less of a problem with experience. • No good automatic node placement algorithms available yet. • Sensory calculations can’t be done on the fly. “If you want to run the AI logic every frame then you will bring down the whole frame rate. Raycasts account for 60% of the AI computation budget.”

13. Decentralized Squad Tactics • No squad Leader. • Each squad member can have its own unique AI. • Team members communicate with each other. • Squad tactics are an emergent behavior. Decentralized Halo video.

14. Decentralized Squad Tactics features • Easy extension to individual AI. • Can handle a wide variety of situations. • Can effectively use each team members capabilities. • Easy to combine with scripted action’s.

15. Decentralized squad AI individual behavior.

16. Simple Example: Fire and Maneuver • Squad members move and announce movements and actions to squad. • When enemy found decides whether to engage threat based on: • Number of squad members engaging. • Number of squad members moving. • Engaging an enemy too long. • Results in Fire and Maneuver behavior: • Squad members are able engage multiple enemies effectively. • Don’t stand still to long. • Can switch targets easily. • Allows for individual member specialties to be used. • Can easily handle squad member deaths. Halo vs swarm

17. Example Two: Squad assaulting hostile positoin. • Each squad member chooses next position. • Based off individual preferences • Intentions of other team members • Can make heavy use of waypoints as described before for faster planning.

18. Ineffective Example: Ambushing • Wait till enemy arrives in kill zone. • Engage them then pull back to rally point. • If discovered return fire and head back to rally zone.

19. Ambush Problems • Difficult for squad to decide on action since no leader. Would have to put squad to sleep till enemy in kill zone. • Squad can not react if the enemy decides to pull back before reaching kill zone. • Squad would never create L shaped ambush on its own.

20. Decentralized Conclusions • Advantages: • Does not require major addition to individual AI. • Can make team members act to take advantage of their specialties. • Easy to Combine scripting to create custom scenarios. • Disadvantages: • Can not easily obtain unanimous decisions. • Movement can become difficult when avoiding each other paths. • Can be memory intensive for each member to maintain its own state. • Halo: [Maintaining full state]too complicated – O(n2) for n=25. instead actors only track characters that are important to them, n around 3 for friends, 5 for enemies

21. Squad Tactics:Centralized Planned Maneuvers "There is nothing inevitable about military victory, even for forces of apparently overwhelming strength. The Greeks at Marathon, Alexander against the Persian Empire, the success of the colonists against the British in the American Revolution, Napoleon over the Austrians in Italy... all offer dramatic evidence to the contrary. In the absence of inspired military leadership... the more powerful side wears down the weaker." - Bevin Alexander • As is true in the military squad AI can not assess its situation and decide the best course of action without a leader and predefined courses of action. • Adding a centralized planning element does not replace the individual AI, but extends it.

22. Comparison of Decentralized vs. Centralized

23. Command Styles • Authoritarian • Squad members always obey and execute commands. • Yields tightly coordinate maneuvers. • Allows squad member sacrifices for overall squad. • Ineffective when squad member view conflicts with squad level. • Ex: Sniper visible to squad member, but conflicts with orders • Coaching • Squad AI assigns tasks to members. • Squad members tries to execute it and reports when it can’t so it can be reassigned. • Yields looser coordination due to members being easily distracted. • By mixing command styles advantages from both can be obtained.

24. Command Style Examples • Authoritarian • When command is issued unit must perform. • Coaching • Command is issued then up to NPCs to decide how to proceed.

25. How to represent situations • Situations can be expressed in abstract features. • Features can be used to choose best maneuver. • Situations features can be represented through influence map as above.

26. Choosing a Maneuver: Fuzzy Logic • Choosing a Maneuver • AI keeps track of permissible maneuvers. • Evaluates each ones fitness. • Chooses maneuver with highest fitness. • Fitness can be represented with fuzzy rules that express situations the maneuver is applicable in. • Situations can have multiple rules to represent different situations • Closely resemble our own rules of thumb. • Halo: Behaviors are made obvious:“Initially nobody noticed so we had to keep adding clues to make it more obvious. By the time we shipped we had made it so every single grunt runs away every single time an Elite is killed.”

27. Bounding Overwatch Pullback Manuever • Pull back while maintaining cover fire. • Members in line provide suppressive fire. • Lead member runs to back. • Repeats until they have reached destination.

28. Pullback Maneuver Details • Path used to pullback requires finding a path that • Has short travel times • Provides concealment from enemies • Gives sufficient space to leapfrog teammates. • Searching for this path is CPU intensive so squad level decision so save CPU time. • A maneuver is performed in several steps • Preparation. • Execution • Finalization

29. Pullback Sequence Chart

30. Centralized Squad Conclusions • Advantages • Good for synchronization and coordination • Resembles human tactic and rules of thumbs. • Can include create many premade plans. • Disadvantages • Can not easily adopt to small variations. • Ignoring a sniper because it goes against plans. • Can sometimes ignore squad member strengths. • Requires a leader to analyze the situation • Requires lots of communication and processing to keep state current. • Halo: Hide weaknesses with design. • Design levels to showcase AI. • Design behaviors to cover up weaknesses • Ex: AI gets more accurate the longer a player is out of cover.

31. Halo 3 Videos • Thank you to Jim Blaney and Tabin Ahmad for assisting me in creating these clips. • Clip 1 • Clip 2 • Clip 3 • Halo AI problems. CONSIDER MOVING CLIPS T APPROPRIATE PLACES IN THE PRESENTATION

32. RetaliCase • Case based reasoning system applied to a reinforcement learning algorithm to control a team of unreal tournament bots. • Retaliate originally created by Megan Vasta • Was presented in class before • Bots would learn strategies during game based of current state and score (Reinforcement Learning). • System proved very effective, capable of defeating the previous research project “HTN bots” in only two games.

33. 60 60 RETALIATE HTNbots 50 50 40 40 30 30 Score 20 Score 20 10 10 0 0 -10 Time -10 Time Retaliate Findings. • Retaliate’sbiggest weakness is maintaining memory between different opponents. • Retaliate must unlearn a case to learn a new one. • Can not identify previous opponents. • Can take a while to learn a strategy. RETALIATE HTNbots

34. RetaliCase Conception • Retaliate could be improved to respond to changing opponents. • Could store previous knowledge (qtables) for future use. • Could be more competitive against new opponents by finding a similar situation. • Case-Based reasoning is ideally suited for this task. • Store plan information. • Can retrieve data based on in game situation. • Could be built right on top of Retaliate.

35. RetaliCase Tactical View • Uses 18 calculated features about current state to save and retrieve cases • Location of Bots compared to domination locations. • Average time a domination point is owned by Retaliate or Enemy • Team score and enemy score differences. • These features are stored in the case based system for each case. • A case about the current situation is also maintained and continuously updated.

36. Simple RetaliCase Chart Case Maintains Retaliate Evaluates Case RetaliCase Loads case into retaliate Yes (winning) Save Case No No (do nothing) Store Case Load Case Yes (Losing) Save Case Retreive and Load Case Retrieves Case Case List

37. RetaliCase Storage • Cases are saved during every successful game. • At the start of every game cases are read into memory. • Each case takes up very little memory and comparisons are fast. • On larger projects however may need a database to speed up case retrieval and minimize memory usage

38. RetaliCase Results • Results so far • Loading good cases has shown increases in performance. • Retalicase seems to be performing better then Retaliate in start of game. • Discovered the need to make sure cases are mature and have had a chance to have been modified before saving/loading. • Poor cases were being saved. • Bad case loading loops were occuring. • Hindered learning.

39. RetaliCase Demonstration • Possible demonstration.

40. Summary • Squad tactics are starting to move toward true AI. • Squad tactics are applicable to variety of genres and applications. • Makes heavy use of other technologies • Waypoints • Scripting Languages • Increasing research and development going into organizing and controlling squads. • Centralized squads • Decentralized squads. • True learning AI systems being created and tested all the time (ex. RetaliCase).