Chapter 15 Bargaining

1. Chapter 15 Bargaining • Negotiation may involve: • Exchange of information • Relaxation of initial goals • Mutual concession

2. Mechanisms, Protocols, Strategies • Negotiation is governed by a mechanism or a protocol: • defines the ”rules of encounter” between the agents • the public rules by which the agents will come to agreements. • The deals that can be made • The sequence of offers and counter-offers that can be made

3. Negotiation Mechanism Negotiation is the process of reaching agreements on matters of common interest. It usually proceeds in a series of rounds, with every agent making a proposal at every round. • Issues in negotiation process: • Negotiation Space: Allpossible deals that agents can make, i.e., the set of candidate deals. • Negotiation Set: deals that are pareto optimal and individually rational • Negotiation Protocol: – rules to determine the process: how/when a proposal can be made, when a deal has been struck, when the negotiation should be terminated, and so. • Negotiation Strategy: When and what proposals should be made. • Agreement Deal: rule to determine when a deal has been struck.

4. Typical Goals of Negotiation • Efficiency – not waste utility. Pareto Optimal • Stability – no agent have incentive to deviate from dominant strategy • Simplicity – low computational demands on agents • Distribution – no central decision maker • Symmetry (possibly) – may not want agents to play different roles.

5. Negotiation Protocol • Who begins • Take turns • single or multiple issues • Build off previous offers • Give feed back (or not). Tell what utility is (or not) • Obligations – requirements for later • Privacy (not share details of offers with others) • Allowed proposals you can make as a result of negotiation history • process terminates (hopefully)

6. Simple Model The rewards to be gained from negotiation are fixed and divided between the two parties Suppose (x, 1-x) represents the portion of the utility each person gets. If the number of rounds are fixed, a person can propose (, 1-) on the last round. Theoretically, a person would rather have  than nothing. Even in multiple rounds: If you knew that the other agent wouldn’t offer you more than , it would be in your best interest to accept it. (So we have a pair of Nash Equilibrium strategies) 

7. Impatient players – shrinking pie Discount factor of  (a fraction in [0,1]) is applied to the utility. At time t, the value of slice x is tx The larger value of , the more patient the player is. Typically the players will have different values for , say 1 and 2 if player 1 offers player 2, 2, he can do no better than accept it.

8. The following offer could be accepted in the first time step (assuming agent 1 would make the first proposal): [ (1- 2)/(1-12), 2(1-1)/(1-12)] Example: 1 =.8, 2 = .9 yields: (.36, .64) Example: 1 =.8, 2 = .8 yields: (.56, .44)

9. Proposal Counter Proposal Agenti concedes Agenti Agentj Negotiation Process 1 • Negotiation usually proceeds in a series of rounds, with every agent making a proposal at every round. • Communication during negotiation:

10. Point of Acceptance/ aggreement Proposals by Aj Proposals by Ai Negotiation Process 2 • Another way of looking at the negotiation process is (can talk about 50/50 or 90/10 depending on who ”moves” the farthest):

11. Single issue negotiation • Like money • Symmetric (If roles were reversed, I would benefit the same way you would) • If one task requires less time, both would benefit equally by taking less time • utility for a task is experienced the same way by whomever is assigned to that task. • Non-symmetric – we would benefit differently if roles were reversed • negotiate about who picks up an item, but you live closer to the store

12. Multiple Issue negotiation • Could be hundreds of issues (cost, delivery date, size, quality) • Some may be inter-related (as size goes down, cost goes down, quality goes up?) • Not clear what a true concession is (larger may be cheaper, but harder to store or spoils before can be used) • May not even be clear what is up for negotiation (I didn’t realize having a bigger office was an option) (on the job…Ask for stock options, travel compensation, work from home, 4- day work week.)

13. How many agents are involved? • One to one • One to many (auction is an example of one seller and many buyers) • Many to many (could be divided into buyers and sellers, or all could be identical in role – like officemate) • n(n-1)/2 number of pairs

14. Jointly Improving Direction method Iterate over • Mediator helps players criticize a tentative agreement (could be status quo) • Generates a compromise direction (where each of the k issues is a direction in k-space) • Mediator helps players to find a jointly preferred outcome along the compromise direction, and then proposes a new tentative agreement.

15. Various Domains Worth Oriented Domain – maximize value to all State Oriented Domain Get to a state we both agree to Task Oriented Domain

16. Typical Negotiation Problems Task-Oriented Domains(TOD): each agent has set of tasks that it has to achieve. The target of a negotiation is to minimize the cost of completing the tasks by divvying them up differently. State Oriented Domains(SOD): each agent is concerned with moving the world from an initial state into one of a set of goal states. The target of a negotiation is to achieve a common goal. Main attribute: actions have side effects (positive/negative). TOD is a subset of SOD. Agents can unintentionally achieve one another’s goals. Negative interactions can also occur. Utility = worth of goal – cost to achieve it Worth Oriented Domains(WOD): agents assign a worth to each potential state (via a function), which captures its desirability for the agent. The target of a negotiation is to maximize mutual worth (rather than worth to individual). Superset of SOD. Rates the acceptability of final states. Allows agents to compromise on their goals.

17. Negotiation Domains:Task-oriented ”Domains in which an agent’s activity can be defined in terms of a set of tasks that it has to achieve”, (Rosenschein & Zlotkin, 1994) • An agent can carry out the tasks without interference (or help) from other agents – such as ”who will deliver the mail” • Any agent can do any task. • Tasks redistributed for the benefit of all agents

18. Types of deals • Conflict deal: keep the same tasks as had originally • Pure – divide up tasks • Mixed – we divide up the tasks, but we decide probabilistically who should do what • All or Nothing (A/N) - Mixed deal, with added requirement that we only have all or nothing deals (one of the tasks sets is empty)

19. Examples of TOD • Parcel Delivery: Several couriers have to deliver sets of parcels to different cities. The target of negotiation is to reallocate deliveries so that the cost of travel for each courier is minimal. • Database Queries: Several agents have access to a common database, and each has to carry out a set of queries. The target of negotiation is to arrange queries so as to maximize efficiency of database operations (Join, Projection, Union, Intersection, …) . e.g., You are doing a join as part of another operation, so please save the results for me.

20. Consider tasks. 1 delivers to a. 2 delivers to both. Must return home. Can’t find a deal where both win. Try mixed deal. Distribution Point Cost function: c()=0 c({a})=6 c({b})=6 c({a,b)}=8 3 3 city a city b 2 Utility for agent 1 (org {a}): Utility1({a}, {b}) = 0 Utility1({b}, {a}) = 0 Utility1({a, b}, ) = -2 Utility1(, {a, b}) =6 … Utility for agent 2 (org {ab}): Utility2({a}, {b}) = 2 Utility2({b}, {a}) = 2 Utility2({a, b}, ) =8 Utility2(, {a, b}) = 0 …

21. What mixed deals are possible if splitting utility is our goal?

22. Consider deal 3 with probability • ({},{ab}):p means agent 1 does {} with p probability and {ab} with (1-p) probability. • What should p be - to be fair to both (equal utility) • (1-p)(-2) + p6 = utility for agent 1 • (1-p)(8) + p0 = utility for agent 2 • (1-p)(-2) + p6= (1-p)(8) + p0 • -2+2p+6p = 8-8p => p=10/16 • If agent 1 does no deliveries 10/16 of the time, it is fair. Note how a mixed deal allows us to be fair.

23. Incomplete Information • Don’t know tasks of others in TOD. • Solution • Exchange missing information • Penalty for lie • Possible lies • False information • Hiding letters (don’t admit part of your job) • Lie about letters (claim work that isn’t required) • decoy – produce if needed • phantom – can’t produce, caught in lie • Not carry out a commitment (trust: misrepresent or unreliable)

24. Difficult to think about • So many situations • So many kinds of lies • So many kinds of deals • Approach – divide into special cases so we can draw conclusions

25. Subadditive Task Oriented DomainCost of whole is ≤ cost of parts • for finite X,Y in T, c(X U Y) <= c(X) + c(Y)). • Examples of subadditive(c(X U Y) < c(X) + c(Y)).): • Deliver to one, saves distance to other (in a tree arrangement if have to return home) • Example of subadditiveTOD (c(X U Y) = c(X) + c(Y)). • deliver in opposite directions –doing both saves nothing • Not subadditive: doing both actually costs more than the sum of the pieces. Example: One way delivery. Doing both causes backtracking.

26. Tasks that don’t exist • We call producible tasks decoy tasks (no risk of being discovered). Unproducible non-existent tasks are called phantom tasks. • Example decoy: Need to pick something up at store. (Can think of something for them to pick up, but if you are the one assigned, you won’t bother to make the trip.) • Example phantom: Need to deliver load (but recipient won’t accept unwanted item)

27. What if phantom task to furthest point?

28. What if hidden task to b? Agent 1: f and b (hides) Agent 2: e Must return to postoffice

29. Incentive compatible MechanismAre the rules (in terms of allowable deals) we establish sufficient to produce truth telling? • L there exists a beneficial lie in some encounter • T  There exists no beneficial lie. • T/P  Truth is dominant if the penalty for lying is stiff enough. Example indicates a case where lying helps. (Assume we deliver and return to post office.) Can you see it? Who lies? What is lie?

30. Explanation of arrow • If it is never beneficial in a mixed deal encounter to use a phantom lie (with penalties), then it is certainly never beneficial to do so in an all-or-nothing mixed deal encounter (which is just a subset of the mixed deal encounters).

31. Concave (a special case of subadditive): c(YU Z) –c(Y) ≤c(XU Z) –c(X) • •X is subset of Y • The cost that tasks Z add to set of tasks Y cannot be greater than the cost Z add to a subset of Y • Expect it to add more to subset (as is smaller) • At seats – is postmen domain concave ? • Example: Y is in pacman shape, X is nodes in polygon. • adding Z adds 0 to X (as was going that way anyway) but adds 2 to its superset Y (as was going around loop) • Concavity implies sub-additivity • Modularity: c(YU Z) –c(Y) =c(XU Z) –c(X) • Modularity implies concavity y

32. Concave Task Oriented Domain • We have 2 tasks X and Y, where X is a subset of Y • Another set of task Z is introduced • c(YU Z) –c(Y) ≤c(XU Z) –c(X)

33. Explanation of Previous Chart • Arrows indicate reasons we know this fact (diagonal arrows are between domains). For example, What is true of a phantom task, may be true for a decoy task in same domain as a phantom is just a decoy task we don’t have to create. • Similarly, what is true for a mixed deal may be true for an all or nothing deal (in the same domain) as a mixed deal is an all or nothing deal where one choice is empty. The direction of the relationship may depend on truth (never helps) or lie (sometimes helps). • The relationships can also go between domains as sub-additive is a superclass of concave and a super class of modular.

34. Modular TOD • c(X U Y) = c(X) + c(Y) - c(X Y). • X and Y are sets of tasks • Notice modular encourages truth telling, more than others

35. Implied relationship between cells Implied relationship between domains (slanted arrows).L means lying may be beneficialT means telling the truth is always beneficialT/P Truth telling is beneficial if penalty for being caught is great

36. Incentive Compatible Facts (assume we must return to post office) Fact1: in SubadditiveTOD, any Optimal Negotiation Mechanism over all or nothing deals, “hiding” lies are not beneficial • Ex: A1 = {b,c}, A2 = {a,b} • A1hides letter to c, his utility doesn’t increase. • If he tells truth : p=1/2 • Expected util ({abc}{})1/2 = 5 • Lie: p=1/2 (as both still go around). This is obviously bad as you deliver all at the same rate, but never deliver nothing. • Expected util (for 1) ({abc}{})1/2 = ½(0) + ½(2) = 1 (as has to deliver to c) 1 4 4 1

37. Fact 2 Modular, all or nothing, decoyencourages truth telling. Both deliver to e and b. Suppose agent 2 lies about having a delivery to c. Under Truth, p (prob of agent 1 doing everything) would be ½. Utility = 3. Under Lie: If we assign p ({ebc}, ) p agent 1 utility -2*p + 6(1-p) Agent 2 (under lie) 8p+0(1-p) -2*p + 6(1-p)= 8p+0(1-p) -8p+6 = 8p p=6/16 (so 2 is worse off) Obviously, we can’t prove we need to tell the truth from an example.

38. Originally each is assigned five places

39. Fact3:in Modular TOD, any ONM over Mixed deals, “Hide” lies can be beneficial. A1={acdef} A2={abcde} • A1 hides his letter to node a • Under truth, each delivers to three. • Under truth Util({fae}{bcd})1/2 = 4 (save going to two) • Under lie, can we divide as ({efd}{cab})p ? • Try lie again, under ANOTHER division ({ab}{cdef})p • p(4) + (1-p)(0) = p(2) + (1-p)(6) • 4p = -4p + 6 • p = 3/4 • Utility is actually • 3/4(6) + 1/4(0) = 4.5

40. Conclusion • In order to use Negotiation Protocols, it is necessary to know when protocols are appropriate • TOD’s cover an important set of Multi-agent interaction

41. Various Domains Worth Oriented Domain – maximize value to all State Oriented Domain Get to a state we both agree to Task Oriented Domain

42. State Oriented Domain • Goals are acceptable final states (superset of TOD) • Example – Slotted blocks world -blocks cannot go anywhere on table – only in slots (restricted resource) • Have side effects - agent doing one action might hinder or help another agent. Example in blocks world, on (white,gray) has side effect of clear(2). • Negotiation : develop joint plans (what they each do) and schedules for the agents, to help and not hinder other agents • Note how this simple change (slots) makes it so two workers get in each other’s way even if goals are unrelated.

43. Assumptions of SOD • Agents will maximize expected utility (will prefer 51% chance of getting $100 than a sure $50) • Agent cannot commit himself (as part of current negotiation) to behavior in future negotiation. • No explicit utility transfer (no side-payment that can be used to compensate one agent for a disadvantageous agreement) • Inter-agent comparison of utility: common utility units • Symmetric abilities (all can perform tasks, and cost is same regardless of agent performing) • Binding commitments

44. Achievement of Final State • Goal of each agent is represented as a set of states that they would be happy with. • Looking for a state in intersection of goals • Possibilities: • (GREAT) Both can be achieved, at gain to both (e.g. travel to same location and split cost) • (IMPOSSIBLE) Goals may contradict, so no mutually acceptable state (e.g., both need the car) • (NEED ALT) Can find common state, but perhaps it cannot be reached with the primitive operations in the domain (could both travel together, but may need to know how to pickup another) • (NOT WORTH IT) Might be a reachable state which satisfies both, but may be too expensive – unwilling to expend effort (i.e., we could save a bit if we car-pooled, but is too complicated for so little gain).

45. Examples: CooperativeEach is helped by joint plan • Slotted blocks world: initially white block is at 1 and black block at 2. Agent 1 wants black in 1. Agent 2 wants white in 2. (Both goals are compatible.) • Assume pick up is cost 1 and set down is one. • Mutually beneficial – each can pick up at the same time, costing each 2 – Win – as didn’t have to move other block out of the way! • If done by one, cost would be four – so utility to each is 2. 

46. Examples: CompromiseBoth succeed, but worse for both than if other agent was not present. • Slotted blocks world: initially white block is at 1 and black block at 2, two gray blocks at 3. • Agent 1 wants black in slot 1, but not on table. • Agent 2 wants white in slot 2, but not directly on table. • Alone, agent 1 could just pick up black and place on white. Similarly, for agent 2. But would undo others goal. • But together, all blocks must be picked up and put down. Best plan: one agent picks up black, while other agent rearranges (cost 6 for one, 2 for other) • Can both be happy, but unequal roles. 

47. Example: conflict • I want black on white (in slot 1) • You want white on black (in slot 1) • Can’t both win. Could flip a coin to decide who wins. Better than both losing. Weightings on coin needn’t be 50-50. • May make sense to have person with highest worth get his way – as utility is greater. (Would accomplish his goal alone) Efficient but not fair? • What if we could transfer half of the gained utility to the other agent? This is not normally allowed, but could work out well.

48. Negotiation Domains: Worth-oriented – more flexible • ”Domains where agents assign a worth to each potential state (of the environment), which captures its desirability for the agent”, (Rosenschein & Zlotkin, 1994) • agent’s goal is to bring about the state of the environment with highest value • we assume that the collection of agents have available a set of joint plans – a joint plan is executed by several different agents • Note – not ”all or nothing” – but how close you got to goal.

49. Rates the acceptability of final states Allows partially completed goals Negotiation : a joint plan, schedules, and goal relaxation. May reach a state that might be a little worse that the ultimate objective Example – Multi-agent Tile world (like airport shuttle) – isn’t just a specific state, but the value of work accomplished Worth Oriented Domain

50. How can we calculate Utility? • Weighting each attribute • Utility = {Price*60 + quality*15 + support*25} • Rating/ranking each attribute • Price : 1, quality 2, support 3 • Using constraints on an attribute • Price[5,100], quality[0-10], support[1-5] • Try to find the pareto optimum