Myopic and non-myopic agent optimization in game theory, economics, biology and artificial intelligence

1. Myopic and non-myopic agent optimization in game theory, economics, biology and artificial intelligence Kae Nemoto Quantum Information Science National Institute of Informatics, Japan Email: nemoto@nii.ac.jp Michael J Gagen Institute of Molecular Bioscience University of Queensland Email: m.gagen@imb.uq.edu.au

2. Overview: Functional Optimization in Strategic Economics (and AI)  Formalized by von Neumann and Morgenstern,Theory of Games and Economic Behavior (1944) Mathematics / Physics (minimize action)

3. Overview: Functional Optimization in Strategic Economics (and AI)  Formalized by von Neumann and Morgenstern,Theory of Games and Economic Behavior (1944) Strategic Economics (maximize expected payoff) Functionals: Fully general Not necessarily continuous Not necessarily differentiable Nb: Implicit Assumption of Continuity !!

4. Overview: Functional Optimization in Strategic Economics (and AI) Strategic Economics (maximize expected payoff) von Neumann’s “myopic” assumption Evidence: von Neumann & Nash used fixed point theorems in probability simplex equivalent to a convex subset of a real vector space von Neumann and Morgenstern, Theory of Games and Economic Behavior (1944) J. F. Nash, Equilibrium points in n-person games. PNAS, 36(1):48–49 (1950)

5. Overview: Functional Optimization in Strategic Economics (and AI) Non-myopic Optimization No communications between players Correlations  Constraints and forbidden regions

6. “Myopic” Economics (= Physics) Non-myopic Optimization ∞ correlations &  ∞ different trees constraint sets Myopic  One Constraint = One Tree Overview: Functional Optimization in Strategic Economics (and AI) X Myopic  “The” Game Tree lists All play options

7. Myopic = Missing Information! Correlation = Information What Information? Nemoto: “It is not what they are doing, its what they are thinking!” • Chess: • “Chunking” or pattern recognition in human chess play • Experts: • Performance in speed chess doesn’t degrade much • Rapidly direct attention to good moves • Assess less than 100 board positions per move • Eye movements fixate only on important positions • Re-produce game positions after brief exposure better than novices, but random positions only as well as novices Learning Strategy = Learning information to help win game

8. Optimization and Correlations are Non-Commuting! • Complex Systems Theory • Emergence of Complexity via correlated signals  higher order structure

9. Prokaryotic gene Eukaryotic gene Hidden layer networking mRNA mRNA & eRNA functions protein protein Mattick: RNA signals in molecular networks Optimization and Correlations are Non-Commuting! • Life Sciences (Evolutionary Optimization) • Selfish Gene Theory • Mayr: Incompatibility between biology and physicsRosen: “Correlated” Components in biology, rather than “uncorrelated” partsMattick: Biology informs information science 6 Gbit DNA program more complex than any human program, implicating RNA as correlating signals allowing multi-tasking and developmental control of complex organisms.

10. Optimization and Correlations are Non-Commuting! • Economics • Selfish independent agents: “homo economicus” • Challenges: Japanese Development Economics, Toyota “Just-In-Time” Production System

11. o i Optimization and Correlations are Non-Commuting! 1 Player Evolving / Learning Machines (neural and molecular networks) endogenously exploit correlations to alter own decision tree, dynamics and optima o = F(i) = F(t,d) = Ft (d)  {F(x,y,z), … ,F(x,x,z),…} Functional Programming, Dataflow computation, re-write architectures, …

12. Iterated Prisoner’s Dilemma Iterated Ultimatum Game Chain Store Paradox (Incumbent never fights new market entrants) Discrepancies: Myopic Agent Optimization and Observation Heuristic statistics

13. ? ?  Myopic Agent Optimization Strategic Form Normal Form Px Py Sum-Over-Histories or Path Integral formulation von Neumann and Morgenstern (1944): All possible information = All possible move combinations for all histories and all futures

14. Optimization Probability of each path Payoff from each stage for each path Sum over all stages Sum over all paths to nth stage Myopic Agent Optimization

15. x1 y1 1-p p 0 ≤ p ≤ 1/2 • Myopic agents ( probability distributions) • uncorrelated • no additional constraints Backwards Induction & Minimax Myopic Agent Optimization

16. Fully general, notationally emphasized by: Optimization Sum over all correlation strategies Payoff for each path Conditioned path probability Sum over all paths given strategy Probability of each strategy Constraint set of each strategy Non-Myopic Agent Optimization

17. Non-Myopic Agent Optimization in the Iterated Prisoner’s Dilemma In 1950 Melvin Dresher and Merrill Flood devised a game later called the Prisoner’s Dilemma Two prisoners are in separate cells and must decide to cooperate or defect Cooperation Defect CKR: Common Knowledge of Rationality Payoff Matrix

18. max  Non-Myopic Agent Optimization in the Iterated Prisoner’s Dilemma Myopic agent assumption

19. =1 > 0  PN-1,x,HN-2(1) = 1 = 0 > 0  PNx,HN-1(1) = 1 Non-Myopic Agent Optimization in the Iterated Prisoner’s Dilemma Myopic agents: N max constraints Simultaneous solution  Backwards Induction  myopic agents always defect

20. 2 max constraints Non-Myopic Agent Optimization in the Iterated Prisoner’s Dilemma Correlated Constraints: (deriving Tit For Tat) < 0  P1x(1) = 0, so Pxcooperates < 0  P1y(1) = 0, so Pycooperates

21. Non-Myopic Agent Optimization in the Iterated Prisoner’s Dilemma Families of correlation constraints: k, j index Change of notation: “dot N” = N, “dot dot N” = 2N, “dot dot N-2” = 2N-2, etc Optimize via game theory techniques Many constrained equilibria involving cooperation Cooperation is rational in IPD

22. Further Reading and Contacts Kae Nemoto Email: nemoto@nii.ac.jp URL: http://www.qis.ex.nii.ac.jp/knemoto.html Michael J Gagen Email: m.gagen@imb.uq.edu.au URL: http://research.imb.uq.edu.au/~m.gagen/ See: Cooperative equilibria in the finite iterated prisoner's dilemma, K. Nemoto and M. J. Gagen, EconPapers:wpawuwpga/0404001 (http://econpapers.hhs.se/paper/wpawuwpga/0404001.htm)