1 / 27

Graphical Models

Graphical Models. Michael Kearns Michael L. Littman Satinder Signh. Presenter: Shay Cohen. So far we have seen …. Players payoffs and the games are represented in tabular form n agents with 2 actions: n matrices of exponential size:

lawrencejackson
Download Presentation

Graphical Models

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Graphical Models Michael Kearns Michael L. Littman Satinder Signh Presenter: Shay Cohen

  2. So far we have seen… • Players payoffs and the games are represented in tabular form • n agents with 2 actions: n matrices of exponential size: • Needed: More compact representations and algorithms for manipulating them

  3. Graphical models (not formal) • n-player game is given by undirected graph with n vertices and n matrices • Payoff is determined only by the neighbors • “local games” composing “global game”

  4. Examples • Games with geographical aspects involved (salespersons) • Topology of computer networks with a limited set of neighbors • … and so on

  5. Reminder… • n-player two-action game: n matrices of size • specifies the payoff for pure strategy x • Nash-Equilibrium: (for all i and for all p’) -Nash-Equilibrium:

  6. Graphical Games • Graphical game: (G,M) • G is undirected graph on n vertices • M is a set of n matrices representing the payoff of player i with its neighbors • Size of is when

  7. Algorithm TreeNash • Works in two passes: the downstream pass and the upstream pass • Downstream: passes indicator tables (with witnesses) from the leafs to the root • Upstream: selects witnesses from root to the leafs (see the attached appendix)

  8. TreeNash – more details • Downstream: A parent U will send to a child V a binary-valued table T(v,u) s.t.: T(v,u)=1 there is NE for in which U=u (v,u – mixed strategies) • Upstream: A child V will be V=v s.t. for all its parents :

  9. Downstream in general • W – child, V – current node, U – parents (b.r. – best response)

  10. U V W Z How? - Downstream T(w,v)=1 v b.r. to w T(w,u)=1 u b.r. to w • T(z,w)=1 for some (u,v): • T(w,u)=1, T(w,v)=1 • W=w b.r. to U=u,V=v,Z=z • T(z)=1for some w: • T(z,w)=1 • Z=z b.r. to W=w (b.r. – best response)

  11. U V W Z How? – Upstream Choose U=u, V=v s.t. T(w,u)=1 and T(w,u)=1 Choose Z=z, W=w s.t. T(z,w)=1

  12. TreeNash • Theorem: TreeNash computes a Nash equilibrium for the tree game (G,M) • Non-deterministic choices: select all of them, and all NE will be found • But the tables are continuous… How do we compute them?

  13. Approximate TreeNash • Tables will be of finite size: • All computations of best responses are computations of -best responses in the grid • Each table has entries, therefore running time is (k parents)

  14. Approximate TreeNash (2) • Lemma: Let p be a NE for (G,M)and let q be the nearest (in metric) mixed strategy on the . Then provided q is a -NE for (G,M)

  15. Approximate TreeNash (3) • Theorem: For any >0, let Then ApproximateTreeNash computes an -NE for the tree game (G,M).

  16. Exact TreeNash • Tables will be made of finite unions of rectangles • Each table T(v,u) will be represented by a v-list: For each interval there is a subset of [0,1] of disjoint intervals: where T(v,u)=1

  17. Exact TreeNash (2) • Assume share a common v-list (by merging) • Downstream: How do we find T(w,v) using them, and keep such representation of rectangles?

  18. Exact TreeNash (3) • Fix a v-interval and set of intervals appropriate to the v-interval for each parent: • T(w,v)=1 is of the form WxI - why? • What would be the region Wfor which some v in the interval is b.r. to u,w?

  19. Exact TreeNash (4) • Denote expected payoff of V • Lemma: If then W is either empty, a continuous interval in [0,1] or union of two intervals.

  20. Exact TreeNash (5) • Can be shown that the leafs can be represented using at most 3 rectangles • Therefore, the representation can be kept and is exponential in the number of vertices • Witnesses can be found easily, because representation is finite

  21. ExactTreeNash • Theorem: ExactTreeNash computes a Nash equilibrium for the tree game (G,M). The algorithm runs in exponential time in the number of vertices of G

  22. Polynomial algorithm • Use downstream pass and upstream pass as well • Pass breakpoints policies (W child of V): Interpretation (“b.p. for V”):

  23. How? - Downstream • Denote: - ordered set of breakpoints of V’s parents - Set of values that W can play that allow V to play any strategy, given - Set of values that W can play, and V’s parents play according to V=b, then V=b is a best response -

  24. How? - Downstream • Lemma: is either empty, a single interval or the union of two intervals • Lemma: • Construct the policy for V by covering [0,1] with them – will produce at most set of 2+l breakpoints. • How do we start with the leafs?

  25. How? - Upstream • Add a dummy root with constant payoff and no influence on the real root • Once we select a value for the child, the value for the parents are determined according to the policies

  26. Running time • Sorting and computing new breakpoint policy: (t – number of breakpoints) • Number of breakpoints is bounded by 2n, therefore total running time:

  27. Summary • First framework gave us: 1. Finding approximation for NE in graphical games which are trees in polynomial time 2. Finding NE for trees in exponential time (ALL of the NEs representation) • Second algorithm: finding NE in polynomial time

More Related