1 / 85

Cognitive Game Theory

Cognitive Game Theory. Alpha-Beta minimax search Inductive Adversary Modeling Evolutionary Chess. Jennifer Novosad, Justin Fox and Jeremie Pouly. Motivation. Good benchmark Similar to military or financial domains Computer can beat humans Fun $. Reasoning Techniques for Games.

onan
Download Presentation

Cognitive Game Theory

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Cognitive Game Theory Alpha-Beta minimax search Inductive Adversary Modeling Evolutionary Chess Jennifer Novosad, Justin Fox and Jeremie Pouly

  2. Motivation • Good benchmark • Similar to military or financial domains • Computer can beat humans • Fun • $

  3. Reasoning Techniques for Games

  4. Cognitive Game Theory • Alpha/Beta Search – Jeremie • Adversary Modeling – Jennifer • Evolutionary Algorithms – Justin

  5. Cognitive Game Theory • Alpha/Beta Search • Minimax search • Evaluation function • Alpha-Beta cutoffs • Other improvements • Demo • Adversary Modeling • Evolutionary Algorithms

  6. MAX Initial Board Situation New Board Situations MIN : : : Win Loss Draw Loss MAX 1 -1 0 -1 Final Board Situations - End Games Adversarial search • Two-person games: Players = Max & Min • Max wants to win • Min wants Max to loose

  7. Minimax search • Basic Assumption • Strategy: • MAX wants to maximise its payoff • MIN is trying to prevent this. • MiniMax procedure maximises MAX’s moves and minimises MIN’s moves.

  8. MAX a 1 MIN c b 1 -1 TerminalStates e f g d 1 1 -1 0 An example Best value for MAX is 1

  9. Minimax recursive procedure • If terminal state then return payoff • Else ifMAX node then use MINIMAX on the children and return the maximum of the results. • Otherwise (MIN node), use MINIMAX on the children and return the minimum of the results. Function MINIMAX (called at each node):

  10. Problems • Time complexity: O(bm) b branching factor and m depth of the terminal states (Chess, b=35, m=100  3510010154 nodes to visit) • Not possible to search the full game tree Cutoff the tree at a certain depth • But payoffs defined only at terminal states

  11. Cognitive Game Theory • Alpha/Beta Search • Minimax search • Evaluation function • Alpha-Beta cutoffs • Other improvements • Demo • Adversary Modeling • Evolutionary Algorithms

  12. Heuristic evaluation function • Estimate the chance of winning from board configuration. • Important qualities: • Must agree with terminal states • Must be fast to compute • Should be accurate enough • Chess or checkers: Value of all white pieces – Value of all black pieces

  13. Heuristic evaluation function Val = (4*1) – (4*1+1*2) = -2 Val ???

  14. Our evaluation function • Normal checker = 100000 • 4 parameters (long): • King value • Bonus central square for kings • Bonus move forward for checkers • Bonus for order of the moves (*depth/2)

  15. Our evaluation function • Normal checker = 100000 • 4 parameters (long): • King value • Bonus central square for kings • Bonus move forward for checkers • Bonus for order of the moves (*depth/2) + 3*Bonus + 2*Bonus + 1*Bonus No Bonus

  16. Cognitive Game Theory • Alpha/Beta Search • Minimax search • Evaluation function • Alpha-Beta cutoffs • Other improvements • Demo • Adversary Modeling • Evolutionary Algorithms

  17. Alpha-Beta pruning • Search deeper in the same amount of time • Basic idea: prune away branches that cannot possibly influence the final decision • Similar to the Branch-and-Bound search (two searches in parallel: MAX and MIN)

  18. General case MAX m MIN : MAX MIN n If m is better than n for MAX then n will never get into play because m will always be chosen in preference.

  19. Review of Branch-and-Bound root Var A A1 A2 A3 1 0 4 Var B B1 B2 B3 B1 B2 B3 ≥ 4 3 12 8 4 4 6 Best assignment: [A1,B1], value = 3

  20. Alpha-Beta procedure • Search game tree keeping track of: • Alpha: Highest value seen so far on maximizing level • Beta: Lowest value seen so far on minimizing level • Pruning: • MAX node: prune parent if node evaluation smaller than Alpha • MIN node: prune parent if node evaluation greater than Beta

  21. Branch-and-Bound analogy • MIN: minimize board valuation  minimize constraints in Branch-and-Bound • MAX: maximize board valuation  inverse of Branch-and-Bound (but same idea) • Prune parent instead of current node(stop expanding siblings)

  22. 2 Example MIN 3 Min Beta not define Beta = 3 Beta = 3 2 3 ≥4 Max 3 1 -5 4 1 0 2 • Beta: Lowest value seen so far on minimizing level

  23. 10 Example MAX 3 Max Alpha not define Alpha = 3 Alpha = 10 10  2 3 Min 2 3 11 5 14 24 10 Alpha: Highest value seen so far on maximizing level

  24. Beta cutoffs MaxValue (Node,a,b) If CutOff-Test(Node) then return Eval(Node) For each Child of Node do a = Max(a, MinValue(Child,a,b)) if a ≥ b then return b Return a

  25. Alpha cutoffs MinValue (Node,a,b) If CutOff-Test(Node) then return Eval(Node) For each Child of Node do b = Min(b, MinValue(Child,a,b)) if b ≤ a then return a Return b

  26. Alpha-Beta gains • Effectiveness depends on nodes ordering • Worse case: no gain (no pruning)  O(bd) • Best case (best first search)  O(bd/2) i.e. allows to double the depth of the search! • Expected complexity: O(b3d/4)

  27. Cognitive Game Theory • Alpha/Beta Search • Minimax search • Evaluation function • Alpha-Beta cutoffs • Other improvements • Demo • Adversary Modeling • Evolutionary Algorithms

  28. Other improvements • Nodes ordering (heuristic) • Quiescent search (variable depth & stable board) • Transposition tables (reconnect nodes in search tree)

  29. Advanced algorithm MaxValue (Node,a,b) If board already exist in transposition tables then if new path is longer return value in the table Save board in transposition table If CutOff-Test(Node) then if quiescent board then return Eval(Node) Find all the children and order them (best first) For each Child of Node (in order) do a:=Max(a,MinValue(Child,a,b)) if a>=b then return b Return a

  30. Statistics: opening

  31. Statistics: jumps available

  32. Statistics: conclusions Gain of more than 99.9% both in time and number of nodes

  33. Cognitive Game Theory • Alpha/Beta Search • Minimax search • Evaluation function • Alpha-Beta cutoffs • Other improvements • Demo • Adversary Modeling • Evolutionary Algorithms

  34. Cognitive Game Theory • Alpha/Beta Search • Adversary Modeling • Psychological Background • Structure of IAM • Getting Chunks • Applying Chunks • Results/Application to  min-max • Flexibility in Other Domains • Evolutionary Algorithms

  35. Inductive Adversary Modeler • Incorporate Model of Opponent into αβ • Currently, Assumes Opponent Plays Optimally • Reduce Computation • Make αβ More Extendable to other domains

  36. Cognitive Game Theory • Alpha/Beta Search • Adversary Modeling • Psychological Background • Structure of IAM • Getting Chunks • Applying Chunks • Results/Application to  min-max • Flexibility in Other Domains • Evolutionary Algorithms

  37. Modeling a Human Opponent *From a study by Chase and Simon

  38. Storing Data -- Chunks • Recall Studies, Masters vs. Beginners • Frequently Used Pattern • Contains Previous Points (Proximity, Similarity, Continuation, Symmetry) • Used to Encapsulate Information

  39. Modeling a Human Opponent 3 Assumptions • Humans Acquire Chunks • Winning Increases Chunk Use (Reinforcement Theory) • People Tend to Reduce Complexity via Familiar Chunks

  40. Cognitive Game Theory • Alpha/Beta Search • Adversary Modeling • Psychological Background • Structure of IAM • Getting Chunks • Valid Chunks • Acquiring Chunks • Applying Chunks • Results/Application to  min-max • Flexibility in Other Domains • Evolutionary Algorithms

  41. Structure of IAM Text Chunks Prior Adversary Games Text Processor Visual Chunks Internal Chess Model Visual Chunk Collector Noise Filter Prediction Current Board Partial Chunk Finder Move Predictor Heuristic Move Selection

  42. Valid Visual Chunks • Proximity - 4x4 grid, adjacent vertically or horizontally • Similarity - same color (exception – pawn structure) • Continuation - pieces defending each other included • Symmetry – symmetrical chunks stored as one (reduces stored chunks by about 60%)

  43. Visual Chunk Collector • Internal Board Model – Matrix of Values, X • After Adversary Move, Search for Valid Chunks • Convolution on Adversary Pieces • Store Values in 8x8 Matrix, Y • If Neighbor in Pattern, Convolve Recursively • 8 16 • X 32 • 1 128 64 • General • 8 16 • X 32 • 0 128 0 • Pawn • 0 8 0 • X 32 • 0 128 0 • Rook, Knight

  44. Convolution Example • 0 8 0 • X 32 • 0 128 0 • Rook, Knight X: 0 0 0 0 0 0 Y: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

  45. Convolution Example • 0 8 0 • X 32 • 0 128 0 • Rook, Knight X: 0 0 0 0 0 0 Y: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

  46. Convolution Example • 0 8 0 • X 32 • 0 128 0 • Rook, Knight X: 0 0 0 0 0 0 Y: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

  47. Convolution Example 0 8 0 2 X 32 0 128 0 Rook, Knight X: 0 0 0 0 0 0 Y: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

  48. Convolution Example • 0 8 0 • X 32 • 0 128 0 • Rook, Knight X: 0 0 0 0 0 0 Y: 0 0 0 128 0 0 0 0 0 128 0 0 0 0 0 0 0 0

  49. Convolution Example • 8 16 • X 32 • 0 128 0 • Pawn X: 0 0 0 0 0 0 Y: 0 0 0 128 0 0 0 0 0 128 0 0 0 0 0 0 0 0

More Related