1 / 15

Fifth International Conference on Autonomous Agents and Multi-agent Systems (AAMAS-06)

Fifth International Conference on Autonomous Agents and Multi-agent Systems (AAMAS-06) Exact Solutions of Interactive POMDPs Using Behavioral Equivalence Speaker Prashant Doshi University of Georgia Authors B. Rathnasabapathy, Prashant Doshi, and Piotr Gmytrasiewicz. Overview.

ngowen
Download Presentation

Fifth International Conference on Autonomous Agents and Multi-agent Systems (AAMAS-06)

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Fifth International Conference on Autonomous Agents and Multi-agent Systems (AAMAS-06) Exact Solutions of Interactive POMDPs Using Behavioral Equivalence Speaker Prashant Doshi University of Georgia Authors B. Rathnasabapathy, Prashant Doshi, and Piotr Gmytrasiewicz

  2. Overview • I-POMDP – Framework for sequential decision making for an agent in a multi-agent setting • Takes the perspective of an individual in an interaction • Problem • Cardinality of the interactive state space → infinite • Other agent's models (incl. beliefs) are part of an agent's state space (interactive epistemology) • An algorithm for solving I-POMDPs exactly • Aggregate behaviorally equivalent models of other agents

  3. Background – Properties of POMDPs and I-POMDPs • Finitely nested • Beliefs are nested up to a finite strategic level l • Level 0 models are POMDPs • Value function of POMDP and finitely nested I-POMDP is piecewise linear and convex (PWLC) • Agents’ behaviors in POMDP and finitely nested I-POMDP can be represented using policy trees

  4. Interactive POMDPs • Definition • Interactive state space • S: set of physical states : set of intentional models : set of subintentional models • Intentional models contain the other agent’s beliefs

  5. Example: Single-Agent Tiger Problem -100 +10 ? -1

  6. P3 P1 P2 Behaviorally Equivalent Models Equivalence Classes of Beliefs

  7. Equivalence Classes of Interactive States • Definition • Combination of a physical state and an equivalence class of models

  8. Lossless Aggregation • In a finitely nested I-POMDP, a probability distribution over , provides a sufficient statistic for the past history of i’s observations • Transformation of the interactive state space into behavioral equivalence classes is value-preserving • Optimal policy of the transformed finitely nested I-POMDP remains unchanged

  9. Solving I-POMDPs Exactly Procedure Solve-IPOMDP ( AGENTi, Belief Nesting L ) : Returns Policy If L = 0 Then Return { Policy : = Solve-POMDP ( AGENTi ) } Else For all AGENTj < > AGENTi Policyj : = Solve-IPOMDP( AGENTj , L-1) End Mj := Behavioral-Equivalence-Models(Policyj ) ECISi : = S x { xj Mj } Policy : = Modified-GIP(ECISi , Ai , Ti , Ωi , Oi , Ri ) Return Policy End

  10. Multi-Agent Persistent-Tiger Problem -100 +10 {Growl Left, Growl Right} X {Creak Right, Creak Left, Silence}

  11. Beliefs on ECIS Agent j’s policy

  12. Agent i’s policy in the presence of another agent j Policy becomes diverse as i’s ability of observing j’s actions improves

  13. Conclusions • A method that enables exact solution of finitely nested interactive POMDPs • Aggregate agent models into behavioral equivalence classes • Discretization is lossless • Interesting behaviors emerge in the multi-agent Tiger problem

  14. Thank You and Please Stop by my Poster Questions

More Related