Reaching Agreements: Voting

1. Reaching Agreements: Voting

2. Voting Truthful voters vote for the candidate they think is best. Why would you vote for something you didn’t want? (run off election – want to pick competition) (more than two canddiates, figure your candidate doesn’t have a chance) We vote in awarding scholarships, teacher of the year, person to hire. • Rank feasible social outcomes based on agents' individual ranking of those outcomes • A - set of n agents • O - set of m feasible outcomes • Each agent i has a preference relation >i : O x O, asymmetric and transitive 2

3. Social choice rule (good for society) • Input: the agent preference relations (>1, …, >n) • Output: elements of O sorted according the input - gives the social preference relation <* of the agent group • In other words – creates ordering for the group 3

4. Desirable properties of the social choice rule: • A social preference ordering >* should exist for all possible inputs (Note, I am using >* to mean “is preferred to.) • >* should be defined for every pair (o, o')O • >* should be asymmetric and transitive over O • The outcomes should be Pareto efficient: if i A, o >i o' then o >* o‘ (not misorder if all agree) • The scheme should be independent of irrelevant alternatives (if all agree on relative ranking of two, should retain ranking in social choice): • No agent should be a dictator in the sense that o >i o' implies o >* o' for all preferences of the other agents 4

5. Arrow's impossibility theorem • No social choice rule satisfies all of the six conditions • Must relax desired attributes • May not require >* to always be defined • We may not require that >* is asymmetic and transitive Use plurality protocol: all votes are cast simultaneously and highest vote count wins. • Introducing an irrelevant alternative may split the majority causing the old majority and the new irrelevant to drop out of favor (The Ross Perot effect). • A binary protocol involves voting pairwise – single elimination The order of the pairing can totally change the results

6. One voter ranks c > d > b > a One voter ranks a > c > d > b One voter ranks b > a > c > d Notice, just rotates preferences. winner (c, (winner (a, winner(b,d)))=a winner (d, (winner (b, winner(c,a)))=d winner (d, (winner (c, winner(a,b)))=c winner (b, (winner (d, winner(c,a)))=b surprisingly, order of pairing yields different winner!

7. Borda protocol (used if binary protocol is too slow) = assigns an alternative |O| points for the highest preference, |O|-1 points for the second, and so on • The counts are summed across the voters and the alternative with the highest count becomes the social choice • Winner turns loser and loser turns winner if the lowest ranked alternative is removed (does this surprise you?) 7

8. Borda Paradox – remove loser, winner changes(notice, c is always ahead of removed item) • a > b > c • b > c >a • c > a > b • a > b > c • b > c > a • c > a >b • a <b <c a=15,b=14, c=13 • a > b > c >d • b > c > d >a • c > d > a > b • a > b > c > d • b > c > d> a • c >d > a >b • a <b <c < d a=18, b=19, c=20, d=13 When loser is removed, next loser becomes winner!

9. Strategic (insincere) voters • Suppose your choice will likely come in second place. If you rank the first choice of rest of group very low, you may lower that choice enough so yours is first. • True story. Dean’s selection. Each committee member told they had 5 points to award and could spread out any way among the candidates. The recipient of the most points wins. I put all my points on one candidate. Most split their points. I swung the vote! What was my gamble? • Want to get the results as if truthful voting were done.

10. Typical Competition Mechanisms • Auction:allocate goods or tasks to agents through market. Need a richer technique for reaching agreements • Negotiation:reach agreements through interaction. • Argumentation:resolve confliction through debates.

11. Reaching Agreements: Voting

12. Negotiation • May involve: • Exchange of information • Relaxation of initial goals • Mutual concession

13. Mechanisms, Protocols, Strategies • Negotiation is governed by a mechanism or a protocol: • defines the ”rules of encounter” between the agents • the public rules by which the agents will come to agreements. • The deals that can be made • The sequence of offers and counter-offers that can be made • Given a particular protocol, how can a particular strategy be designed that individual agents can use?

14. Negotiation Mechanism Negotiation is the process of reaching agreements on matters of common interest. It usually proceeds in a series of rounds, with every agent making a proposal at every round. • Issues in negotiation process: • Negotiation Space: Allpossible deals that agents can make, i.e., the set of candidate deals. • Negotiation Protocol: – A rule that determines the process of a negotiation: how and when a proposal can be made, when a deal has been struck, when the negotiation should be terminated, and so. • Negotiation Strategy: When and what proposals should be made.

15. Protocol • Means kinds of deals that can be made • Means sequence of offers and counter-offers • Protocol is like rules of chess game, whereas strategy is way in which player decides which move to make • Do we even understand what is up for grabs? We may ask for a raise without considering bigger office, different appointment, 4-day work week, etc.

16. Negotiation Protocol • Who begins • Take turns • Build off previous offers • Give feed back (or not). • Tell what utility is (or not) • Obligations – requirements for later • Privacy • Allowed proposals you can make as a result of negotiation history

17. Proposal Counter Proposal Agenti concedes Agenti Agentj Negotiation Process 1 • Negotiation usually proceeds in a series of rounds, with every agent making a proposal at every round. • Communication during negotiation:

18. Point of Acceptance/ aggreement Proposals by Aj Proposals by Ai Negotiation Process 2 • Another way of looking at the negotiation process is (can talk about 50/50 or 90/10 depending on who ”moves” the farthest):

19. Jointly Improving Direction method Iterate over • Mediator helps players criticize a tentative agreement (could be status quo) • Generates a compromise direction (where each of the k issues is a direction in k-space) • Mediator helps players to find a jointly preferred outcome along the compromise direction, and then proposes a new tentative agreement.

20. Example: list of things to be done. Assigned to individuals already • Who does what • What is order to do • What is paid for tasks

21. Goals of Negotiation (in many cases) • Efficiency – not waste utility. Pareto Opt • Stability – no agent have incentive to deviate from agreed-upon strategy (as in one-shot negotiation). • Simplicity – low computational demands on agents • Distribution – interaction rules not require a central decision maker • Symmetry – (in some cases) may not want agents to play different roles.

22. Example: • Planes need to be assigned landing time. • Rule could be that airplanes with less fuel land first. Any disadvantage?

23. Slotted Blocks world • Like blocks world, only a fixed number of slots on table. • Forces need to coordinate • Ex: Need to share car. Has side effects • Ex: Schedule classes/professors – no side effect.

24. Various Domains Worth Oriented Domain State Oriented Domain Task Oriented Domain

25. Typical Negotiation Problems Task-Oriented Domains(TOD): an agent's activity can be defined in terms of a set of tasks that it has to achieve. The target of a negotiation is to minimize the cost of completing the tasks. State Oriented Domains(SOD): each agent is concerned with moving the world from an initial state into one of a set of goal states. The target of a negotiation is to achieve a common goal. Main attribute: actions have side effects (positive/negative). TOD is a subset of SOD. Most classical AI domains are instances of SOD. Main attribute of SOD – actions have side effects. Agents can unintentionally achieve one another’s goals. Negative interactions can also occur. Worth Oriented Domains(WOD): agents assign a worth to each potential state, which captures its desirability for the agent. The target of a negotiation is to maximize mutual worth (rather than worth to individual). Superset of SOD. Rates the acceptability of final states. Allows agents to compromise on their goals.

26. The simplest plan to achieve On(White,Gray) has the side effect of achieving Clear(black)

27. Single issue negotiation • Like money • Symmetric (If roles were reversed, I would benefit the same way you would) • If one task requires less travel, both would benefit equally by having less travel • utility for a task is experienced the same way by whomever is assigned to that task. • Non-symmetric – we would benefit differently if roles were reversed • if you delivered the picnic table, you could just throw it in the back of your van. If I delivered it, I would have to rent a U-haul to transport it (as my car is small).

28. Multiple Issue negotiation • Could be hundreds of issues (cost, delivery date, size, quality) • Some may be inter-related (as size goes down, cost goes down, quality goes up?) • Not clear what a true concession is (larger may be cheaper, but harder to store or spoils before can be used) • May not even be clear what is up for negotiation (I didn’t realize not having any test was an option) (on the job…Ask for stock options, bigger office, work from home.)

29. How many agents are involved? • One to one • One to many (auction is an example of one seller and many buyers) • Many to many (could be divided into buyers and sellers, or all could be identical in role) • n(n-1)/2 number of pairs

30. Negotiation Domains:Task-oriented • ”Domains in which an agent’s activity can be defined in terms of a set of tasks that it has to achieve”, (Rosenschein & Zlotkin, 1994) • An agent can carry out the tasks without interference (or help) from other agents – such as ”who will deliver the mail” • All resources are available to the agent • Tasks redistributed for the benefit of all agents

31. Task-oriented Domain: Definition • How can an agent evaluate the utility of a specific deal? • Utility represents how much an agent has to gain from the deal. (it is always based on change from original allocation) • Since an agent can achieve the goal on its own, it can compare the cost of achieving the goal on its own to the cost of its part of the deal. • If utility<0, it is worse off than performing tasks on its own. • Conflict deal: (stay with status quo) if agents fail to reach an agreement: • where no agent agrees to execute tasks other than its own. • utlity = 0

32. Formalization of TOD A Task Oriented Domain(TOD) is a triple <T, Ag, c> where: • T is a finite set of all possible tasks; • Ag={A1, A2,…, An} is a list of participant agents; • c:(T)R+defines cost of executing each subset of tasks. • Assumptions on cost function: • c() = 0. • The cost of a subset of tasks does not depend on who carries out them. (Idealized situation) • Cost function is monotonic, which means that more tasks, more cost. (It can’t cost less to take on more tasks.) • T1  T2 implies c(T1)  c(T2)

33. Redistribution of Tasks Given a TOD <T, {A1,A2}, c>, T is original assignment, output is D: assignment after the “deal” • An encounter (instance) within the TODis an ordered list (T1, T2) such that for all k, Tk  T. This is an original allocation of tasks that they might want to reallocate. • A pure deal on an encounteris the redistribution of tasks among agents: (D1, D2), such that all tasks are reassigned D1 D2= T1 T2 Specifically, : (D1, D2)=(T1, T2) is called the conflict deal. • For each deal=(D1, D2), the cost of such a deal to agent k is Costk()=c(Dk) (i.e, cost to k of deal  is cost of Dk, k’s part of deal)

34. Examples of TOD • Parcel Delivery: Several couriers have to deliver sets of parcels to different cities. The target of negotiation is to reallocate deliveries so that the cost of travel to each courier is minimal. • Database Queries: Several agents have access to a common database, and each has to carry out a set of queries. The target of negotiation is to arrange queries so as to maximize efficiency of database operations (Join, Projection, Union, Intersection, …) . You are doing a join as part of another operation, so please save the results for me.

35. Possible Deals Consider an encounter from the Parcel Delivery Domain. Suppose we have two agents. Both agents have parcels to deliver to city aand only agent 2 has parcels to deliver to city b. There are nine distinct pure deals in this encounter: • ({a}, {b}) • ({b}, {a}) • ({a,b}, ) • (, {a,b}) • ({a}, {a,b}) ({b}, {a,b}) ({a,b}, {a}) ({a,b}, {b}) ({a,b}, {a,b}) the conflict deal

36. Figure deals knowing union must be {ab} • Choices for first agent: {a} {b} {ab} {} • Second agent must “pick up the slack” • {a} for agent 1 b|ab (for agent 2) • {b} for agent 1a|ab • {ab} for agent 1 a|ab|b|{} • {} for agent 1 ab

37. Utility Function for Agents Given an encounter (T1, T2), the utility function for each agent is just the difference of costs and is defined as follow: Utilityk()=c(Tk)-Costk() = c(Tk)- c(Dk) where • =(D1, D2) is a deal; • c(Tk) is the stand-alone cost to agent k (the cost of achieving its original goal with no help) • Costk() is the cost of its part of the deal. Note that the utility of the conflict deal is always 0.

38. Parcel Delivery Domain (assuming do not have to return home – like Uhaul) Distribution Point Cost function: c()=0 c({a})=1 c({b})=1 c({a,b)}=3 1 1 city a city b 2 Utility for agent 1 (org {a}): Utility1({a}, {b}) = 0 Utility1({b}, {a}) = 0 Utility1({a, b}, ) = -2 Utility1(, {a, b}) = 1 … Utility for agent 2 (org {ab}): Utility2({a}, {b}) = 2 Utility2({b}, {a}) = 2 Utility2({a, b}, ) = 3 Utility2(, {a, b}) = 0 …

39. Dominant Deals • Deal  dominates deal ' if  is better for at least one agent and not worse for the other, i.e., •  is at least as good for every agent as ': k{1,2},Utilityk() Utilityk(') •  is better for some agent than ': k{1,2},Utilityk()>Utilityk(') • Deal  weakly dominates deal ' if at least the first condition holds (deal isn’t worse for anyone). Any reasonable agent would prefer (or go along with)  over ' if  dominates or weakly dominates '.

40. Negotiation Set: Space of Negotiation • A deal  is called individual rational if  weaklydominates the conflict deal. (no worse than what you have already) • A deal  is called Pareto optimalif there does not exist another deal ' that dominates . (best deal for x without disadvantaging y) • The set of all deals that are individual rational and Pareto optimal is called the negotiation set(NS).

41. Utility Function for Agents (example from previous slide) Utility2({a}, {b}) =2 Utility2 ({b}, {a})=2 Utility2 ({a,b}, )=3 Utility2 (, {a,b})=0 Utility2 ({a}, {a,b})=0 Utility2 ({b}, {a,b})=0 Utility2 ({a,b}, {a})=2 Utility2 ({a,b}, {b})=2 Utility2 ({a,b}, {a,b})=0 • Utility1({a}, {b}) =0 • Utility1({b}, {a})=0 • Utility1({a,b}, )=-2 • Utility1(, {a,b})=1 • Utility1({a}, {a,b})=0 • Utility1({b}, {a,b})=0 • Utility1({a,b}, {a})=-2 • Utility1({a,b}, {b})=-2 • Utility1({a,b}, {a,b})=-2

42. Individual Rational for Both(eliminate any choices that are negative for either) • ({a}, {b}) • ({b}, {a}) • ({a,b}, ) • (, {a,b}) • ({a}, {a,b}) • ({b}, {a,b}) • ({a,b}, {a}) • ({a,b}, {b}) • ({a,b}, {a,b}) ({a}, {b}) ({b}, {a}) (, {a,b}) ({a}, {a,b}) ({b}, {a,b}) individual rational

43. Pareto Optimal Deals • ({a}, {b}) • ({b}, {a}) • ({a,b}, ) • (, {a,b}) • ({a}, {a,b}) • ({b}, {a,b}) • ({a,b}, {a}) • ({a,b}, {b}) • ({a,b}, {a,b}) is (-2,3), but nothing beats 3 for agent 2 ({a}, {b}) ({b}, {a}) ({a,b}, ) (, {a,b}) Pareto Optimal Beaten by ({a}{b}) deal

44. Negotiation Set Individual Rational Deals ({a}, {b}) ({b}, {a}) (, {a,b}) ({a}, {a,b}) ({b}, {a,b}) Pareto Optimal Deals ({a}, {b}) ({b}, {a}) ({a,b}, ) (, {a,b}) Negotiation Set ({a}, {b}) ({b}, {a}) (, {a,b})

45. Negotiation Set illustrated • Create a scatter plot of the utility for i over the utility for j • Only those where both is positive are individually rational (for both) (origin is conflict deal) • Which are pareto optimal? Utility for i Utility for j

46. Negotiation Set in Task-oriented Domains Utility for agent i Negotiation set: (pareto optimal+ Individual rational) B A C Utility of conflict Deal for agent i The circle delimits the space of all possible deals E Conflict deal D Utility for agent j Utility of conflict Deal for agent j

47. Negotiation Protocol • P(d) – Product of the two agent utilities from d • product maximizing negotiation protocol One step protocol • Concession protocol • At time t >= 0, A offers d(A,t) and B offers d(B,t), such that • Both deals are from the negotiation set • "i e {A,B} and "t >0, Utilityi(d(i,t)) <= Utilityi(d(i,t-1)) • I propose something less desirable for me • Negotiation ending • Conflict - Utilityi(d(i,t)) = Utilityi(d(i,t-1)) • Agreement, $j !=i e {A,B},Utilityj(d(i,t)) >= Utilityj(d(j,t)) • Only A => agree d(B,t) either agrees with proposal of other • Only B => agree d(A,t) either agrees with proposal of other • Both A,B => agree d(k,t) such that P(d(k))=max{P(d(A)),P(d(B))} • Both A,B and P(d(A))=P(d(B)) => flip a coin (product is the same, but may not be the same for each agent – flip coin to decide which deal to use) Pure deals Mixed deal

48. The Monotonic Concession Protocol – One direction, move towards middle Rules of this protocol are as follows. . . • Negotiation proceeds in rounds. • On round 1, agents simultaneously propose a deal from the negotiation set [individually rational, pareto optimal). Can re-propose same deal. • Agreement is reached if one agent finds that the deal proposed by the other is at least as good or better than its proposal. • If no agreement is reached, then negotiation proceeds to another round of simultaneous proposals. • An agent is not allowed to offer the other agent less (in term of utility ) than it did in the previous round. It can either stand still or make a concession. Assumes we know what the other agent values. • If neither agent makes a concession in some round, then negotiation terminates, with the conflict deal. • Meta data may be present: explanation or critique of deal.

49. Condition to Consent an Agreement If both of the agents finds that the deal proposed by the other is at least as good or better than the proposal it made. Utility1(2) Utility1(1) and Utility2(1) Utility2(2)

50. The Monotonic Concession Protocol • Advantages: • Symmetrically distributed (no agent plays a special role) • Ensures convergence • It will not go on indefinitely • Disadvantages: • Agents can run into conflicts • Inefficient – no quarantee that an agreement will be reached quickly