On the Role of Interactive Epistemology in Multiagent Planning

1. On the Role of Interactive Epistemology in Multiagent Planning Artificial Intelligence and Pattern Recognition (AIPR), 2007 Author Prashant Doshi Speaker Sergei Fogelson Dept. of Computer Science and AI Center University of Georgia

2. Decision-theoretic Planning Single agent setting Physical State (Loc, Orient,...) state transition • State is perfectly observable • Act to optimize expected reward Markov decision processes (MDPs)

3. Decision-theoretic Planning Single agent setting Beliefs Physical State (Loc, Orient,...) state transition • State is partially observable • Act to optimize expected reward given beliefs Partially observable Markov decision processes (POMDPs)

4. Decision-theoretic Planning Multi-agent setting Beliefs state transition Physical State (Loc, Orient,...) How should agent i act in a multi-agent setting?

5. Planning in Multiagent Settings • Traditional game-theoretic approach • Nash equilibrium • Plans that are best responses to each other • Epistemological commitment • Common knowledge of rationality • Common knowledge of joint prior over agent types • Limitations • Formalization of common knowledge is self-referential • Epistemological commitment difficult to satisfy • N.E. is non-unique • Multiple equilibria may exist • N.E. is incomplete • Does not prescribe outside of equilibria

6. Illustration: Centipede Game Each agent may defect or cooperate A A B B cooperate (18,18)‏ defect (-1,10) (9,9) (8,19) (0,0) • Common knowledge of rationality => A will defect (backward induction) • Mutual knowledge of rationality where depth = length of the game • is also sufficient • A is uncertain about B's rationality => A will cooperate

7. Interactive Epistemology Beliefs Physical State (Loc, Orient,...) Beliefs • Agent i's belief is a distribution over the physical state and models of • other agent j • Model of j = (Beliefj, Frame) • Agent j's belief is a distribution over the state and models of i

8. Recursive State Space • Interactive state space: ISi= S x j • Agent i's belief: • Recursive description • Limit to finite nesting for computability (Sufficient statistic) Analogous to hierarchical beliefs systems in game theory

9. Interactive POMDPs (I-POMDPs) (Gmytrasiewicz&Doshi, JAIR05) • I-POMDPs: A framework for decision-theoretic planning in multiagent settings • Generalizes POMDPs to multiagent settings • -- set of interactive states of agent i • -- set of joint moves of all agents • -- state transition function • -- set of observations of agent i • -- observation function • -- preferences of agent i

10. I-POMDPs • Belief update for agent i in I-POMDP • Use the other agent's model to predict its action(s) • Anticipate the other agent's observations and how it updates its beliefs • Use own observations to correct the beliefs • Recursively updates the beliefs at each nesting level down to level 0 • Plan computation • Analogous to POMDPs given the new belief update • Previously this approach has been called • Decision-theoretic approach to game theory(Kadane&Larkey82) • Subjective rationality(Kalai&Lehrer93)

11. Discussion • In multiagent settings assumptions about behaviors of agents assume significance • Common assumption • Common knowledge of rationality and prior beliefs • Necessary epistemic condition for Nash equilibrium • Beliefs are private • Common knowledge of beliefs is infeasible • Beliefs over beliefs => nested belief systems • Infinitely nested belief systems are not computable

12. Discussion • Finitely nested beliefs systems – Computable but suboptimal • I-POMDPs – Planning in multiagent settings cognizant of nested beliefs • Contributes to a growing focus on dynamic interactive epistemology • Future Work • Address the computational complexity of I-POMDPs

13. Thank you