1 / 42

Game Playing and Game Programming (Chapter 6)

Game Playing and Game Programming (Chapter 6). Outline. Strategies The minimax algorithm alpha-beta pruning Expectiminimax Applications of AI in game programming. Game vs Search Problems. “Unpredictable” opponent”  solution is a strategy specifying a move for every opponent play

laddie
Download Presentation

Game Playing and Game Programming (Chapter 6)

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Game Playing and Game Programming(Chapter 6) BUSI 0088 Handout 5 - Game Playing and Game Programming

  2. Outline • Strategies • The minimax algorithm • alpha-beta pruning • Expectiminimax • Applications of AI in game programming BUSI 0088 Handout 5 - Game Playing and Game Programming

  3. Game vs Search Problems • “Unpredictable” opponent”  solution is a strategy specifying a move for every opponent play • Because of time limits, it is unlikely to find all possibilities. Must approximate based on algorithms. BUSI 0088 Handout 5 - Game Playing and Game Programming

  4. Types of Games • Single player vs two players vs multiple players • Zero-sum vs non-zero-sum • Perfect information vs imperfect information • Deterministic vs chance BUSI 0088 Handout 5 - Game Playing and Game Programming

  5. Some Examples of 2-player Games deterministic chance perfect information imperfect information BUSI 0088 Handout 5 - Game Playing and Game Programming

  6. Game Trees • Considering games with 2 players called MAX and MIN. MAX moves first, and then they take turns moving until the game is over. • A game can be formally defined as a search problem with the following components: • Initial state: the board position and identifies the player to move. • Successor function: a list of (move, state) pairs • Terminal test: which determines when the game is over • Utility function: Gives a numeric value for the terminal states. For example, in chess, the outcome is win, loss, or draw, which can be given a value 1, -1, or 0. BUSI 0088 Handout 5 - Game Playing and Game Programming

  7. Game Trees • The initial state and the legal moves for each side define the game tree for the game. • Consider the game of tic-tac-toe. From the initial state, MAX has nine possible moves. • MAX and MIN take turns until we reach leaf nodes corresponding to terminal states — one player has three in a row or all the squares are filled. BUSI 0088 Handout 5 - Game Playing and Game Programming

  8. Game Tree for Tic-Tac-Toe *The utility values are for MAX BUSI 0088 Handout 5 - Game Playing and Game Programming

  9. Game Tree for a Trivial 2-ply Game • MAX moves first and can choose the path a1, a2, or a3. MIN then makes the second move. The utility values are the values for MAX. BUSI 0088 Handout 5 - Game Playing and Game Programming

  10. Optimal Strategy • What is the optimal strategy? • Roughly speaking, an optimal strategy leads to the best outcome assuming one is playing against the best opponent. BUSI 0088 Handout 5 - Game Playing and Game Programming

  11. Minimax • Given a game tree, the optimal strategy can be determined by examining the minimax value of each node • Consider zero-sum games: MAX’s heaven is MIN’s hell. • The minimax value of a node is the utility (for MAX) of being in the corresponding state, assuming that both players play optimally from there to the end of the game. • Obviously, the minimax value of a terminal state is just its utility. At other nodes, MAX will prefer to move to a state of maximum value, whereas MIN prefers a state of minimum value. BUSI 0088 Handout 5 - Game Playing and Game Programming

  12. Minimax • MINIMAX-VALUE(n) = UTILITY(n) if n is a terminal node maxsSuccessors(n) MINIMAX-VALUE(s) if n is a MAX node minsSuccessors(n) MINIMAX-VALUE(s) if n is a MIN node BUSI 0088 Handout 5 - Game Playing and Game Programming

  13. Minimax • Example with the 2-ply game • The minimax value at B = min(3, 12, 8) = 3 • The minimax value at C = min(2, 4, 6) = 2 • The minimax value at D = min(14, 5, 2) = 2 • The minimax value at A = max(3, 2, 2) = 3

  14. The Minimax Algorithm • Uses a simple recursive computation of the minimax values of each successor state by performing a complete depth-first search of the game tree. • Chooses the move with the highest minimax value. BUSI 0088 Handout 5 - Game Playing and Game Programming

  15. The Minimax Algorithm • If the maximum depth of the tree is m and there are b legal moves at each node, the time complexity of the minimax algorithm is O(bm) • For real games, the time cost is impractical, e.g., in chess, b 35 and m  100 • Suppose the computer has 100 seconds to “think”, and it can explore 10,000 nodes per second  106 nodes per move • bm = 106,b = 35  m = 4 • 4-ply  human novice • 8-ply  typical PC, human master • 12-ply  Deep Blue, Kasparov BUSI 0088 Handout 5 - Game Playing and Game Programming

  16. Alpha-Beta Pruning • To compute the correct minimax decision without looking at every node in the game tree. • Prunes away branches that cannot possibly influence the final decision. • Always returns the same move as the standard minimax. BUSI 0088 Handout 5 - Game Playing and Game Programming

  17. Alpha-beta Pruning BUSI 0088 Handout 5 - Game Playing and Game Programming

  18. Alpha-beta Pruning BUSI 0088 Handout 5 - Game Playing and Game Programming

  19. Alpha-beta Pruning BUSI 0088 Handout 5 - Game Playing and Game Programming

  20. Alpha-beta Pruning BUSI 0088 Handout 5 - Game Playing and Game Programming

  21. Alpha-beta Pruning BUSI 0088 Handout 5 - Game Playing and Game Programming

  22. Alpha-beta Pruning • Good ordering of moves can improve effectiveness of pruning. • With “perfect ordering”, time complexity = O(bm/2) • Double depth of search • Can easily reach depth 8 and play good chess BUSI 0088 Handout 5 - Game Playing and Game Programming

  23. Alpha-beta Pruning • Why is it called Alpha-Beta Pruning (- pruning)? •  = the value of the best choice we have found so far for MAX •  = the value of the best choice we have found so far for MIN • Alpha-beta search updates the values of  and  as it goes along and prunes the remaining branches at a node as soon as the value of the current node is known to be worse than the current  or  value for MAX or MIN, respectively. BUSI 0088 Handout 5 - Game Playing and Game Programming

  24. Alpha-beta Pruning • In general, if m is better than n for the player, we will never get to n in play. BUSI 0088 Handout 5 - Game Playing and Game Programming

  25. Nondeterministic Games • In many games there are chances introduced by dice, card-shuffling, etc. • A simple example with coin-flipping: BUSI 0088 Handout 5 - Game Playing and Game Programming

  26. Expectiminimax • Expectiminimax gives optimal play. • Just like minimax, except that we must also handle chance nodes. • EXPECTIMINIMAX-VALUE(n) = UTILITY(n) if n is a terminal node maxsSuccessors(n) EXPECTIMINIMAX-VALUE(s) if n is a MAX node minsSuccessors(n) EXPECTIMINIMAX-VALUE(s) if n is a MIN node sumsSuccessors(n) P(s)  EXPECTIMINIMAX-VALUE(s) if n is a chance node • P(s) = the probability that s will be chosen based on the chance node BUSI 0088 Handout 5 - Game Playing and Game Programming

  27. State-of-the-Art • Checkers • In 1952, Arthur Samuel of IBM developed a checkers program that learned its own evaluation function by playing itself thousands of times. Starting as a novice, it improved very well after a few days and defeated Samuel. 10 years later it defeated Robert Nealy, a human champion. Samuel’s computer had 10,000 words of main memory, magnetic tape for long-term storage, and a 0.001 MHz processor. • In 1994, the human world champion was defeated by a computer defining perfect play for all possible positions involving 8 or fewer pieces on board (around 443 billion positions). BUSI 0088 Handout 5 - Game Playing and Game Programming

  28. State-of-the-Art • Chess: In 1997, Deep Blue, developed at IBM, defeated Kasparov in a six-game exhibition match. Deep Blue searches 200 million positions per seconds, uses very sophisticated evaluation, and undisclosed methods for extending some lines of search up to 40 ply. • Othello: Defeated human world champion by six games to none. It is generally acknowledged that humans are no match for computers at Othello. • Go: The board is 19x19, making it too daunting for regular search methods (b > 300). There are some better programs these days. BUSI 0088 Handout 5 - Game Playing and Game Programming

  29. Other Things to Consider • We have only considered two-player zero-sum game. Game theory can be applied in non-zero-sum games and multiplayer games. Collaboration (teaming up) may be needed in multiplayer games. • Risk analysis: how much risk should the computer take? • High level reasoning or planning can eliminate the search space significantly. BUSI 0088 Handout 5 - Game Playing and Game Programming

  30. Other AI Applications in Game Programming • Finite State machines • Behaviors of computer characters • Prediction and Learning • Artificial Life BUSI 0088 Handout 5 - Game Playing and Game Programming

  31. State Machines • Finite State Machines (FSMs) are the most commonly used techniques in AI game programming. • FSM is a concise, nonlinear description of how an object can change its state over time. BUSI 0088 Handout 5 - Game Playing and Game Programming

  32. Finite State Machines • Example of a simple FSM for a computer-programmed opponent (CPO) Enemy in sight Walk around Fight Enemy dead Food in sight Finish eating Low energy Cornered Flee Eat Food Food in sight BUSI 0088 Handout 5 - Game Playing and Game Programming

  33. Finite State Machines • Finite State Machines can be improved by using randomness and probabilities  Fuzzy State Machines BUSI 0088 Handout 5 - Game Playing and Game Programming

  34. Behavior • Pathfinding is often needed in game programming. E.g., asking a CPO (computer-programmed opponent) or NPC (non-player character) to go to a specific position on the map. • Usually based on A* search • Need to avoid obstacles • Need to make sure the entities are not trapped in loop • Need to control the entities to move together (flocking) BUSI 0088 Handout 5 - Game Playing and Game Programming

  35. Behavior • The characters need to be aware of: • The player (the most important!) • Environment (don’t walk through a wall) • Team management (ensure that the members are working as a team) BUSI 0088 Handout 5 - Game Playing and Game Programming

  36. Behavior • Behavior can be goal-directed (using the agent concept). • For example, in Age of Empires II, the following rule was used in the AI Script by the CPO of the game: (defrule (goal resource-needed WOOD) (current-age == dark-age) (civilian-population >= 10) (not (strategic-number sn-wood-gatherer-percentage = 40) ) => (set-strategic-number sn-wood-gatherer-percentage 40) (set-strategic-number sn-food-gatherer-percentage 60) ) BUSI 0088 Handout 5 - Game Playing and Game Programming

  37. Behavior • Sometimes the computer can’t be too tough; otherwise the player will never win. • Artificial stupidity is often introduced, usually in the easy mode of the game. • Missing the first attack • Bad aiming skills • Warning the player before attacking • Not attacking at the same time BUSI 0088 Handout 5 - Game Playing and Game Programming

  38. Prediction and Learning • Needs to predict what the users will do next (e.g., punching in a fighting game, passing or shooting the ball in soccer, running in a certain direction). • In the past, many computer games cheat by having more resources (e.g., Age of Empires) or allowing illegal moves (e.g., Street Fighter). This is less appreciated by players now and more sophisticated AI techniques are used in games. BUSI 0088 Handout 5 - Game Playing and Game Programming

  39. Prediction and Learning • For example, in a street fighting game, the player’s actions can be accumulated into a move history. It is possible to record the player’s movement for statistical prediction. If the following statistics are obtained, then we can better predict the player’s moves. Player sequence Occurrences Frequency Low Kick, Low Punch, Uppercut 10 50% Low Kick, Low Punch, Low Punch 7 35% Low Kick, Low Punch, Low Kick 3 15% • Virtua Fighter 4 uses a similar technique. BUSI 0088 Handout 5 - Game Playing and Game Programming

  40. Prediction and Learning • In recent years, there are more AI techniques (like neural networks, genetic algorithms, reinforcement learning) being used for learning in game programming. • Some examples • Creatures (GA and NN) • Cloak, Dagger, and DNA (GA, can be freely downloaded at: http://www.gameai.com/src/cddna.zip) • Black & White (NN) • Colin McRae Rally 2.0 (NN) • World Championship Snooker 2003 (NN) • Fields of Battle (simulated annealing and NN) BUSI 0088 Handout 5 - Game Playing and Game Programming

  41. Artificial Life • Making artificial lives in computers. • Examples: • Princess Maker • The Sims • Petz (the Dogz and Catz screensavers). • Creatures • Theme Hospital • Each A-life has his own goals, behaviors, personalities, emotions, and even a family and career. BUSI 0088 Handout 5 - Game Playing and Game Programming

  42. Artificial Life • In The Sims, even the objects have their own behaviors. For example, a smart microwave oven knows what it can accomplish (cook food) and how it should be used (open door, place food inside, close door, …). • Agents can use the objects with which they were never programmed to interact. BUSI 0088 Handout 5 - Game Playing and Game Programming

More Related