Announcements

1. Announcements • Next week doesn’t exist! • Including both Saturdays • Regular retries for Final Idea+Platformer 4 will be due on Tuesday, April 1 • Try to catch up on everything by the end of the break so you can focus your efforts on the final project when you get back • zdavis and dpercy are here! Yay :D

2. Lecture 8 Advanced Game Design I’m afraid I can’t do that! Presentations+Playtesting C++ Tip of the Week AI in Games

3. Overview • A smattering of game AI techniques • Goal-based planning • Influence maps • Neural networks • Learning bots in Quake • Modeling randomness

4. Classical game AI • Finite state machine • Hard-coded behaviors, specify all possible combinations of actions • A-star path planning • On top of a waypoint graph navigation mesh • Also local steering forces, object avoidance, etc. • These techniques are widely used today • ...But are not the focus of today's lecture

5. Alternative: Goal-driven AI • Hierarchical set of goals • e.g. "buy sword" => "obtain gold" + "go to smithy" • Plan & perform actions to achieve goals • Planner finds a sequence of actions that will achieve a goal • Sequence is not hard-coded, discovered on the fly • May also use A-star path planning

6. Planning challenges • Combinatorial explosion of world states • Need to simplify representation of world • Only model what is relevant to a specific agent • Efficiently searching through possible actions • Can model as graph search problem • Biasing towards certain actions • Weights on graph

7. Case study: F.E.A.R • 2005 horror FPS • Ranked #2 Most influential AI Game

8. F.E.A.R. AI • Uses GOAP (Goal-Oriented Action Planning) • Consists of goals and actions • Goal: certain state of the world we want to reach • Action: set of preconditions and effects • Solved using graph search • Nodes are world states and edges are actions • Search from current state toward goal • Edges are directional (directed graph) • Analogy to chess AI: nodes are board states and edges are moves

9. GOAP example • World state: • boolenemyHasGun • boolplayerAlive • Only models states relevant to agent Current state enemyHasGun = false playerAlive = false enemyHasGun = false playerAlive = true Drop gun Pick up gun Drop gun Pick up gun Shoot player enemyHasGun = true playerAlive = false enemyHasGun = true playerAlive = true Goal state

10. GOAP • Different from FSMs • FSM: Spend all time in nodes (actions), transitions are instant • GOAP: Spend all time in transitions (actions), nodes are instant • Advantages over FSMs • Don't need to encode all possible transitions from each action to each other action • Simpler to specify and easier to scale • Just specify preconditions and effects for each action

11. GOAP in F.E.A.R. • Graph is dynamic • Edges are actions with preconditions and effects • Edges come and go based on game state • Procedural preconditions • Taking an action may be currently impossible • Example: Escape through door requires door unlocked • Procedural effects • Effects take time to execute and may fail • Example: Firing a projectile is blocked by something

12. GOAP in F.E.A.R. • Don't want all agents to act the same • Vary edge weights based on individual preference (aggressive vs careful individual) • Need variety in available actions (run, crouch, dodge, roll, slide) • Hard for player to understand AI's behavior • Add audio cues for player • "Send in reinforcements!"

13. Influence maps • Goal: decide which team controls an area • Used as an input to higher level decision algorithms

14. Influence maps • Details • Numeric value that varies throughout the world • Positive for team A, negative for team B • Magnitude indicates strength of influence • Can also encode other information: safety, congestion, etc... • Advantages • Entire influence map not updated immediately, will "remember" recent history • Helps reason about strategy in complex worlds (open areas, choke points) • Commonly used in RTS games

15. Influence maps • Algorithm • Influence map starts off at zero • Sources of influence are marked • Entities (friends and foes) • Events like grenade explosions, taking damage • Influence is propagated (diffused) to neighbors • Parameters • Momentum (m): How fast do new values overwrite old values? • Decay (d): How fast do values decrease from the source?

16. Influence maps • Influence maps can be combined with graphs

17. Genetic algorithms in games • Hardly ever used • Developers want fine control over behavior, often just enumerate all the cases • Difficult to test • Genetic algorithms take thousands of iterations to converge • Creating faster genetic algorithms is an active area of research

18. Case study: Learning bots in Quake • Remco Bonse et. al. in 2004 • Use the Q-learning algorithm to train a neural network for an AI bot in Quake III • One-on-one game against preprogrammed AI • Replaced combat movement subsystem with NN • Reward is given for avoiding damage • State: • Distance and angle of opponent • Distance and angle of nearest projectile • Actions: • All 18 combinations of WASD + jump

19. Case study: Learning bots in Quake • Q-learning algorithm: • Agent is in some state s ∈ S • Agent can take some action a ∈ A • State/action quality function: Q : S × A → ℝ • Agent gets reward for each state change • Make a correction to Q based on new information • Learning rate: 0 = learn nothing, 1 = ignore past • Discount factor: 0 = short-term, 1 = long-term

20. Case study: Learning bots in Quake • Best parameters were to play to 100,000 frags on a network of 15 neurons (10 hours training time) • Learned bot wins twice as often against a preprogrammed bot • Human players didn't notice a difference • Limited interaction time in a FPS

21. Randomness and AI • One motivation for AI is variability • Adding randomness can help • Easy way of varying AI behaviors • Need to be careful about how users perceive randomness

22. Randomness and human perception • Which is more random? 50% or 50%

23. Randomness and human perception • Humans are illogical • Conditioned to see patterns • Patterns are more memorable • Gambler's Fallacy • We think outcomes depend on past events for independent events with fixed probabilities • Not just gamblers, everyone does this • "His luck will run out sooner or later"

24. Law of Large Numbers • Average approaches the expected value as the number of trials increases Unfair in the short term

25. Law of Small Numbers? • Humans expect random binary choices will alternate up to 40% more than is mathematically reasonable • Example: If a player loses a game involving randomness 4 times in a row, is that a good experience? • Make randomness easier to understand • Random choices without replacement • Keep track of past choices and prevent them from being chosen again too soon

26. Lecture 8 AI in Games Now its own section! Presentations+Playtesting C++ Tip of the Week Advanced Game Design

27. Advanced Game Design • Hi, I’m Zach • Graduated last year, now studying game design at USC’s Interactive Media and Games division • This section covers a smattering of advanced game design topics • Huizinga’s magic circle, formal elements of games, game feel, juice • Game design covered in 195n is helpful but not required

28. Huizinga’s Magic Circle • In games, we frequently do things we would never do in real life • Be rude/insulting in conversations • Betray/lie to our friends • Destroy cities • What makes us willing to do this in a game but not in the real world?

29. Huizinga’s Magic Circle • Johan Huizinga described games as taking place in a “magic circle” • Temporary world in which the rules of the game apply instead of the rules of the ordinary world • Bounds the experience • Helps players feel safe to experiment • Players know the game will end, or they can leave/quit at any time • Helps players suspend disbelief

30. Formal Elements of Games • Players • Objective • Procedures • Rules • Resources • Conflict • Boundaries • Outcome

31. Players • Number of players • Roles of players • Sometimes defined by rules (e.g. RPG classes) • Sometimes emergent player behavior (Bartle’s taxonomy) • Player interaction pattern • Can have multiple • Invitation to play • Akin to dimming lights/opening curtains • Verbal in physical games • Many different ways in digital games • “Press Start” • Picking up a plastic guitar • Helps players enter magic circle

32. Objective • The main thing the player(s) strive for • What they want/need to accomplish in the system • Achieving the objective frequently ends the game • Common objective types: • Capture, chase, race, alignment, rescue/escape, forbidden act, construction, exploration, solution, outwit

33. Procedures • Actions the player can take to achieve the objective • Who does what where, when, and why? • Distinct from rules • Specifically focused on player actions • In many digital games, end up being equivalent to the controls • Four common categories • Starting action, progression of action, special actions, resolving actions

34. Rules • Define game objects and allowable actions • Can be explicit or implicit • Should be clear to the player either way • Common rule types • Defining objects/concepts • Restricting actions • Determining effects

35. Resources • Just like resources in the real world • Assets that can be used to accomplish goals • Value determined by utility and scarcity • Can be both in-game (e.g. health) and out (e.g. attention, fatigue) • Common resources: • Lives, units, health, money, limited actions, powerups, special terrain, time

36. Conflict • The part of the game that prevents players from accomplishing their goals directly • Usually the part of the game that makes it “interesting” • Common sources: • Obstacles • Opponents • Dilemmas

37. Boundaries • Separate the game from everything not part of the game • Physical • Edges of arena/playing field/level • Conceptual • Social agreements • Varying levels of rigidity • FPS levels vs tag or ARGs

38. Outcome • Generally determined by nature of the objective and player interaction pattern • Games with score almost always use score as their outcome • PvP usually results in one winner and one loser • Must be uncertain for players to remain interested

39. Game Feel • Which of the five senses do digital games engage? • Obviously sight/sound and not smell/taste • Touch is kinda weird • In his book Game Feel, Steve Swink argues touch can be engaged through what he calls “game feel” ?

40. Components of Game Feel • Real-time control • Cycle of interpreting output and • Spatial simulation • Simulated physical interactions in virtual space • Perceived actively by the player • Polish • Particles, sound, animation • Artificially enhances interaction without changing underlying system

41. Juice It or Lose It http://www.youtube.com/watch?v=Fy0aCDmgnxg

42. Juice • Other ways to make your game juicy • Avoid discrete visual changes whenever possible • Make pickups pop upwards and fade out • Fade/slide between menus and game • Actually have a game shell (main menu, etc.) • Not really juice, but definitely makes a game feel more “real” • Caveat: Juice will not make a bad game good • Another way to put it: “You can’t polish a turd” • For good examples, try Teggle or Dots on iOS

43. Bonus: Plot without Conflict • http://stilleatingoranges.tumblr.com/post/25153960313/the-significance-of-plot-without-conflict • If not necessary for stories, not necessary for games?

44. Further resources/references • Books • Game Design Workshop by Tracy Fullerton • Great balance of theory and practice of game design • Game Feel by Steve Swink • Articles • How to prototype a game in under 7 days • Actually just really good advice about making games in general

45. Lecture 8 AI in Games Now its own section! Presentations+Playtesting Advanced Game Design C++ Tip of the Week

46. C++ tip of the week • Multiple inheritance • C++ classes can have multiple superclasses • Can be used to emulate Java interfaces // Declare interfaces using pure virtual functions structReader { virtualchar read() = 0; }; structWriter { virtualvoid write(char) = 0; }; // File * can be dynamic_castto Reader * and Writer * structFile : Reader, Writer { char read() { ... } void write(char) { ... } };

47. C++ tip of the week • More powerful than Java interfaces • All superclasses can have state structTimestamp { constlongcreationTime; Timestamp() : creationTime(time(NULL)) {} }; structUniqueID { constintid; staticintnextID; UniqueID() : id(nextID++) {} }; structEntity : Timestamp, UniqueID {};

48. C++ tip of the week • The diamond problem structBase { intnum; }; structDerived1 : Base {}; structDerived2 : Base {}; structJoin : Derived1, Derived2 {}; Joinjoin; cout << join.num; // error, join has two copies of num cout << join.Derived1::num; cout << join.Derived2::num; Base Base Derived2 Derived1 Join

49. C++ tip of the week • Can be mitigated with virtual inheritance structBase { intnum; }; structDerived1 : virtualBase {}; structDerived2 : virtualBase {}; structJoin : Derived1, Derived2 {}; Joinjoin; cout << join.num; // works, only one copy of Base and num Base Derived2 Derived1 Join

50. C++ tip of the week • Tricky case • Multiple implementations of foo in Join (base1 != base2) • Using virtual inheritance here would cause an error structBase { virtualvoid foo() = 0; }; structDerived1 : Base { void foo() { cout << 1; } }; structDerived2 : Base { void foo() { cout << 2; } }; structJoin : Derived1, Derived2 {}; Joinjoin; Base *base = &join; // error, ambiguous conversion Base *base1 = static_cast<Derived1 *>(&join); Base *base2 = static_cast<Derived2 *>(&join);