Distributed Rational Decision Making

Distributed Rational Decision Making Author: Tuomas W. Sandholm Speakers: Praveen Guddeti (1---5) Tibor Moldovan (6---9) CSE 976, April 15, 2002

Outline • Introduction • Evaluation criteria • Non-cooperative interaction protocols • Voting • Auctions • Bargaining • General equilibrium market mechanisms • Contract nets • Coalition formation • Conclusions

Introduction • Automated negotiation systems with self-interested agents are becoming increasing important. 1. Technology push. 2. Application pull. • Paper deals with protocols designed using a non-cooperative, strategic perspective.

Evaluation Criteria • Social welfare • Pareto efficiency • Individual rationality • Stability • Computational efficiency • Distribution and communication efficiency

Evaluation Criteria1. Social Welfare • It is the sum of all agents’ payoffs or utilities in a given solution. • Requires inter-agent utility comparisons.

Evaluation Criteria2. Pareto Efficiency • A solution x is Pareto efficient if there is no other solution x’ such that • at least one agent is better off in x’ than in x, and • no agent is worse off in x’ than in x. • Does not require inter-agent utility comparisons. • Social welfare maximizing solutions are a subset of Pareto efficient ones.

Evaluation Criteria3. Individual Rationality • Participation in a negotiation is individually rational to an agent only if it is profitable. • A mechanism is individually rational if participation is individually rational for all agents. • Only individually rational mechanisms are viable.

Evaluation Criteria4. Stability • The protocol mechanisms should motivate each agent to behave in the desired manner. • Protocol mechanisms may have dominant strategies. This means that an agent is best off by using a specific strategy no matter what strategies the other agents use. • Nash equilibrium: Each agent chooses a strategy that is a best response to the other agents’ strategies.

Nash Equilibrium:Formal definition • The strategy profile S*A={ S*1, S*2,…, S*|A|} among agents A is in Nash equilibrium if for each agent i , S*i is the agent’s best strategy given that the other agents choose strategies {S*1, S*2,…, S*i-1, S*i+1,… S*|A|}. • In some games no Nash equilibrium exists. • Some games have multiple Nash equilibrium.

Nash Equilibrium:Comments • Even if Nash equilibrium exists and is unique, there are limitations regarding what the Nash equilibrium guarantees. • In sequential games it only guarantees stability in the beginning of the game. • Subgame perfect Nash equilibrium. • Nash equilibrium is often too weak because subgroups of agents can deviate in a coordinated manner. • Sometimes efficiency and stability goals conflict.

Evaluation Criteria5. Computational Efficiency • The protocol mechanisms when used by agents should need as little computation as possible. • Trade off between: • the cost of the computation needed for the protocol mechanisms and • the solution quality.

Evaluation Criteria6. Distribution and Communication Efficiency • Distributed protocols should be preferred in order to avoid a single point of failure and a performance bottleneck – among other reasons. • Minimize the amount of communication required to get to a desired global solution. • These two goals can conflict.

Non-cooperative interaction protocols • Voting • Auctions • Bargaining • General equilibrium market mechanisms • Contract nets • Coalition formation

Non-cooperative Interaction Protocols1. Voting • All agents give input to a mechanism. • Outcome chosen by the mechanism is solution for all agents. • Outcome is enforced. • Voters. • Truthful voters. • Strategic (Insincere) voters.

VotersTruthful Voters (1) • Each agent i  A has an asymmetric and transitive strict preference relations  i on O. • Social choice rule. • input the agents’ preference relations ( 1,…, |A|). • output the social preferences denoted by a relation *.

Truthful Voters (2)Properties of a social choice rule • * should exist for all possible inputs. • * should be defined for every pair o, o’ O. • * should be asymmetric and transitive over O. • The outcome should be Pareto efficient. • The scheme should be independent of irrelevant alternatives. • No agent should be a dictator.

Truthful Voters (3) • Arrow’s Impossibility Theorem: No social rule satisfies all of these six conditions. • Relax the first property. • Relax the third property. • Plurality protocol. • Binary protocol. • Borda protocol.

Truthful Voters (4)Plurality Protocol • Majority voting protocol. • All alternatives are compared simultaneously. • The one having the highest number of votes wins. • Irrelevant alternative can split the majority.

Truthful Voters (5)Binary Protocol (1) • Pair wise voting with the winner staying to challenge remaining alternatives. • Irrelevant alternatives can change outcomes. • Agenda i.e. order of the pairings can change the outcomes.

Truthful Voters (6)Binary Protocol (2) 35 % of agents have preferences cdba 33 % of agents have preferences acdb 32 % of agents have preferences bacd

Truthful Voters (7)Borda Protocol (1) • Assign an alternative |O| points whenever it is the highest in some agent’s preference list, |O| -1 when it is second and so on. • Sum the counts of all alternatives. • Alternative with highest count is the winner. • Irrelevant alternatives lead to paradoxical results.

Borda Protocol (2) Agent Preferences 1 a  b  c  d 2 b  c  d  a 3 c  d  a  b 4 a  b  c  d 5 b  c  d  a 6 c  d  a  b 7 a  b  c  d Borda count c wins with 20, b has 19, a has 18, d loses with 13 Borda count a wins with 15, b has 14,loses with 13 with d removed

VotersStrategic (Insincere) Voters (1) • Revelation principle: Suppose some protocol implements social choice function f(.) in Nash (or dominant strategy) equilibrium, then f(.) is implementable in Nash (or dominant strategy) equilibrium via a single-step protocol where the agents reveal their types truthfully.

VotersStrategic (Insincere) Voters (2) • Gibbard-Satterthwaite impossibility theorem: Let each agent’s type i, consist of a preference order i on O. Let there be no restrictions on i, i.e. each agent may rank the outcomes O in any order. Let |O|  3. Now, if the social choice function f(.) is truthfully implementable in a dominant strategy equilibrium, then f(.) is dictatorial, i.e. there is some agent i who gets (one of) his most preferred outcomes chosen no matter what types the others reveal.

VotersStrategic (Insincere) Voters (3) • Circumventing the GSIT: • Restricted preferences. • Groves-Clarke Tax Mechanism. • Groves-Clarke Tax Mechanism: • o = (g ,1,… |A|). • i is the amount agent i receives. • g encodes the other features of the outcome.

Strategic (Insincere) Voters (4)Groves-Clarke Tax Mechanism(1) • Quasilinear preferences: ui(o) = vi(g) + i. • Net benefit: vi(g) = vi gross(g) – P / |A|. • Every agent iA reveals his valuation vi(g) for every possible g. • The social choice is g* =arg maxg i vi(g). • Every agent is levied a tax: tax i =  ji vj (g*) -  ji vj (arg maxg  ki vk (g)).

Strategic (Insincere) Voters (5)Groves-Clarke Tax Mechanism(2) • Size of an agent’s tax is exactly how much his vote lowers the other’s utility. • Quasilinearity: • No agent should care how others divide payoffs among themselves. • An agent’s valuation vi gross(g) should not depend on the amount of money that the agent will have.

VotersStrategic (Insincere) Voters (6) • If each agent has quasilinear preferences, then each agent’s dominant strategy is to reveal his true preferences. • Agents need not waste effort in counter speculating each others’ preference declarations. • Participation is individually rational.

VotersStrategic (Insincere) Voters (7) • Problems of Groves-Clarke Tax Mechanism: • Does not maintain budget balance. • Not coalition proof. • Intractable. • Other ways to circumvent the GSIT: • Choosing a dictator randomly. • Make the computation of an untruthful revelation prohibitively costly.

Non-cooperative Interaction Protocols2.Auctions • Unlike voting where the outcome binds all agents, in auctions the outcome is usually a deal between two agents. • In voting the protocol designer wants to enhance the social good, while in auctions, the auctioneer wants to maximize his own profit. • Classical setting. • Contracting setting.

Auctions (2) • Auction settings. • Auctions protocols. • Efficiency of the resulting allocation. • Revenue equivalence and non-equivalence. • Bidder collusion. • Lying auctioneer. • Bidders lying in non-private-value auctions. • Undesirable private information revelation. • Roles of computation in auctions.

1. Auction settings • Three qualitatively different auctions depending on how an agent’s value of the item is formed: • Private value. • Common value. • Correlated value.

2. Auctions protocols • English (first-price open-cry) auction. • First-price sealed-bid auction. • Dutch (descending) auction. • Vickrey (second-price sealed-bid) auction. • Allocation of computation resources in OS, allocation of bandwidth in computer networks, computationally control building heating. • Has not been widely adopted in auctions among humans.

3. Efficiency of the resulting allocation. • In isolated private value or common value auctions, each one of the four auction protocols allocates the auctioned item Pareto efficiently to the bidder who values it the most. • All four protocols are Pareto efficient in the allocation. • The dominant strategies (Vickrey and English) are more efficient.

4. Revenue equivalence and non-equivalence. • Revenue equivalence: All of the four auction protocols produce the same expected revenue to the auctioneer in private value auctions where the values are independently distributed and bidders are risk-neutral. • Among risk averse bidders, the Dutch and the first-price sealed-bid protocols give higher expected revenue to the auctioneer. • A risk averse auctioneer achieves higher expected utility via the Vickrey or English protocols.

Revenue equivalence and non-equivalence. (2) • In non-private value auctions, both the English and Vickrey protocols produce greater expected revenue to the auctioneer than the first-price sealed-bid auction or Dutch auction. • In non-private value auctions with at least three bidders, the English auction leads to higher revenue than the Vickrey auction.

5. Bidder collusion. • The English auction and the Vickrey auction actually self-enforce some of the most likely collusion agreements. • First-price sealed-bid and the Dutch auctions are preferred for deterring collusion. • For collusion to take place in Vickrey, first-price sealed-bid or Dutch auctions the bidders have to identify each other before placing the bids.

6. Lying Auctioneer. • In Vickrey auction the auctioneer may lie about the value of the second highest bidder. • In the English auction the auctioneer can use shills that bid in the auction in order to make the real bidders increase their valuations of the item. • The auctioneer may bid himself to guarantee that the item will not be sold below a certain price.

7. Bidders lying in non-private-value auctions. • Winner’s curse: If an agent bids its valuations and wins the auction, it will know that its valuation was too high because the other agents bid less. • Agents should bid less than their valuations. • This is the best strategy in Vickrey auctions. • Vickrey fails to induce truthful bidding in most auction settings.

8. Undesirable private information revelation. • In Vickrey auctions the agents often bid truthfully. This leads to the bidders revealing their true valuations. • This information is sensitive and the bidders would prefer not to reveal it. • Another reason why the Vickrey auction protocol is not widely used among humans.

9. Roles of computation in auctions. • Two issues arise from computation in auctions: • Computationally complex look ahead that arises when auctioning interrelated items one at a time. • Implications of costly local marginal cost (valuation) computation or information gathering in a single-shot auction.

Interrelated auctions. • Look ahead: • Without look ahead the allocation may be inefficient. • With look ahead the agents will not bid their true per-item cost. • Computation cost may be prohibitively great. • Allow agents to backtrack from commitments by paying penalties.

Single-shot auctions • Incentive to counter speculate: In a single-shot private value Vickrey auction with uncertainty about an agent’s own valuations, a risk neutral agent’s best action can depend on the other agents. It follows that is is worth counter speculating.

Non-cooperative Interaction Protocols3. Bargaining • Real world settings usually consist of a finite number of competing agents, so neither monopoly,nor monopsony nor perfect competition assumptions strictly apply. • Bargaining theory fits in this gap. • Bargaining theory: • Axiomatic • Strategic

Bargaining Theory Axiomatic • Does not use the idea of equilibrium. • Desirable properties for a solution, called axioms of the bargaining solution, are postulated. • Then the solution that satisfies these axioms are sought. • Nash bargaining solution.

Axiomatic Bargaining Theory Nash Bargaining Solution (1) • Nash analyzed a 2-agent setting where the agents have to decide on an outcome o  O, and the fallback outcome ofallback occurs if no agreement is reached. • There is a utility function ui: O  R for each agent i  [1,2]. • It is assumed that the set of feasible utility vectors { (u1 (o), u2 (o)) | o  O} is convex.

Axiomatic Bargaining TheoryNash Bargaining Solution (2) • Axioms for the Nash bargaining solution u* = (u1(o*), u2(o*)) are: 1. Invariance. 2. Anonymity (symmetry). 3. Independence of irrelevant alternatives. 4. Pareto efficiency.

Axiomatic Bargaining TheoryNash Bargaining Solution (3) • The unique solution that satisfies these four axioms is: o* = arg maxo[u1(o)–u1(ofallback)][u2(o)–u2 (ofallback)] • Other bargaining solutions also exist.

Bargaining TheoryStrategic (1) • Bargaining situation is modeled as a game. • Solution is based on an analysis of which of the players’ strategies are in equilibrium. • Solution is not unique. • Explains the behavior of rational utility maximizing agents better than axiomatic approaches. • Usually analyses sequential bargaining.

Bargaining TheoryStrategic (2) • Finite number of offers with no time discount. • Finite number of offers with time discount. • Infinite number of offers with no time discount. • Infinite number of offers with time discount.

Distributed Rational Decision Making