RECAP CSE 348 AI Game Programming

1. RECAPCSE 348 AI Game Programming Héctor Muñoz-Avila

2. C A B C A B C B A B A C AI research “AI” as game practitioners implemented it B A C B A B C C B A B A C A C A A Our goal was to understand the connections and the misconceptions from both sides B C B C C B A A B C Course Goal projects

3. Patrol • Preconditions: No Monster • Effects: patrolled • Fight • Preconditions: Monster in sight • Effects: No Monster Soldier Attack E,~D Planning Operators E Rifleman Machine Gunner Officer ~E Chase S,~E,~D S D Wander ~E,~S,~D E ~S D American German American German American German E S Spawn D ~E British Soviet British Soviet British Soviet D Controlling the AI Opponent: FSMs • FSM: States, Events and Actions • Stack Based FSM’s • Polymorphic FSM • Multi-tier FSM FSM: Monster In Sight Robocode Patrol Fight No Monster A resulting plan: Monster in sight No Monster patrolled Fight Patrol

4. UT task: Domination Strategy: secure most locations UT action: move Bot1 to location B Controlling the AI Opponent: HFSMs Attack Wander ~E Chase Pick-up Powerup E ~S S Spawn Start Turn Right D ~E Go-through Door

5. Controlling AI Opponent: Scripting • Autonomous agents calculate their action based on… (Nick Haynes) Wargus (Jon Martin) Desires Sensory Input Proximity to items of interest 1 1 1 Space reservation: quasi-coordination 1 1

6. E E T P E E Controlling AI Opponent: Team Unreal tournament (Eric Lease) • (Dayne Mickelson) • Team sports • Identify high-level decisions • Multi-layered approach • Line of sight (player, npcs) • GOAP: • Agent can dynamically find alternate solutions to problems • Dead Reckoning • Predicting future state • For games: Newton physics • Estimate future trajectory: Kinematics

7. team controlled team controlled b y computer by human player A Combat B Learning: Adaptive Behavior • Dynamic scripting: Reinforcement learning • But sometimes the problem resides in the scripts not the ordering • Use evolutionary computation to improve scripts (Megan Vasta) Neural networks • Evolve a population (each member is a candidate solution) …

8. Learning: Adaptive Behavior (2) Allegiance (Jeff Storey) • User model • Flexibility beyond predefined difficulty levels • When/what to update Friendly Enemy Defense -1.0 Weak Strong Medium 0.4 -0.3 0.1 • Induced from a collection of data • Based on information gain formulas • Assume discrete values (Brigette Swan)

9. Learning: Adaptive Behavior (3) • Pattern recognition • Symbols • Optimization: balancing units in an RTS game • 2. Curse of dimensionalit • Analysis of Machine learning Usage • 1. Cheap to recognize what to learn from? • 2. Cheap to store the knowledge? • 3. Cheap to use the knowledge? • 4. Does game benefit from learning? (Chris Kramer)

10. Spatial Analysis • Random map generator: • Location of players • Map is generated step-wise by adding clumps • Terrain analysis: • Concepts: borders, corridors • Selection of new colonies • Spatial Analysis: Transport units in RTS games: (Russell Kuchar) (Jay Shipper )

11. Spatial Analysis (2) • Wall generation • Graph representation: • (tiles, connections) • Greedy algorithm Hierarchy in RTS games (Rami Khouri)

12. Path-Finding (1) A*: minimize f(n) = g(n) + h(n) • Grid • Graphs • Meshes (Dan Bader) Rep. simplicity versus optimality - Can be used to compute AI • (Tom Gianos) • Navigation set hierarchy String pulling • Interface tables • Reduction memory • Increase performance

13. Path-Finding (2) (Owen Cummings) (Tom Schaible) • Path Look-Up tables • Several times faster than A* • But memory consumption is high • Solution: Area-based Look-up tables • Notion of portals • Very fast • Throwing a grenade is not so simple! • Add information to nodes • Add behavior info in edges Flying Edge Rappelling Edge Flying Edge Door Edge Vault Edge Jump Edge Hunting players in a convincing manner

14. Path-Finding (3) • (Adam Balgach) • Racing vehicle control • Multi-layer system • Each layer defines behavior • Optimal racing line • Use of Newton physics • (Don DeLorenzo) • Avoiding obstacles • Should be smooth • Crucial in dynamic worlds Ra a Da Va Obstacle Sidestep Repulsion • Intelligent Steering • Use error correction: • current error + history error + rate error

15. Game theory Declarative Knowledge Spectimax kind of search Initial state Goals A C B A B C • HTN approach for declarer play • Use HTN planning to generate a game tree in which each move corresponds to a different strategy, not a different card • Reduces average game-tree size to about 26,000 leaf nodes • Compute expectimax and expectimin • Evaluation functions • Pruning search space poker

16. Game Design • (Peter Shankar) • “Meaningful play” • Outcome is discernable and integrated • Elements for meaningful play: • Semiotics • Systems • Interactivity • Choice Cultural System Experiential System Formal System • Sid Mier says: • “personal touch” is needed

17. Hall of Fame • Winners Project 1: • Tournament: Adam Balgach, Tom Gianos. Bot: Yankees • Innovation: Tom Shaible, Don Delorenzo. For: "meta-level" FSM design of code. • Winners Project 2: • Tournament: Adam Balgach, Tom Gianos. Team: Yankees (continuing champions!) • Innovation: Swan, Brigette L, Vasta, Megan E., and Khouri, Rami H. For: a number of interesting ideas: predicting next place for firing, distributing battlefield, training examples. • Winners Project 3: • Project # 3 was no tournament. • Winners Project 4: • Tournament: Adam Balgach, Tom Gianos. Team: Yankees (unbeatable!) • Winners Project 5: • Tournament: Tom Schaible, Don Delorenzo. Team: DDTS (new champions!) • Winners Project 6: • Tournament:. Owen Cummings, Dayne Mickelson Team: Tony Wonder (new champions!) • Innovation: Tom Shaible, Don Delorenzo. For: decision trees and reinforcement learning

18. Acknowledgements • Jon Martin and Eric Lease • All of you: • Presentations were very good • Projects were worked well (despite difficulties) • Changes: • 4 projects: robocode, UT, MadRTS, poker • UT: 2 bots only • Poker: use downloadable version

19. The End…