Modeling Complex Multi-Issue Negotiations Using Utility Graphs

1. Modeling Complex Multi-Issue Negotiations Using Utility Graphs Valentin Robu, Koye Somefun, Han La Poutré CWI, Center for Mathematics and Computer Science, Amsterdam, The Netherlands TFG - MARA, Budapest, September 2005

2. Multi-issue (multi-item) negotiation • Negotiation = method of competitive (or partially cooperative) allocation of goods, resources, tasks between agents • Applications: • E-commerce: Bundling can be an effective method to increase sales (use in recommender systems) • High degree of customization – possible through negotiations • Logistics: mechanism for task allocation • Many deals are negotiated bilaterally or in closed groups of companies (e.g. transportation contracts) • Utility functions are not (or partially) revealed => indirect revelation mechanism • Search with incomplete information TFG - MARA, Budapest, September 2005

3. Utility functions for multi-issue negotiations • Linearly additive: • Linear combination of issue utilities: • Search space is structured -> more accesible to heuristics [Faratin Sierra & Jennings. 2002], [Jonker & Robu 2004], [Coehoorn & Jennings 2004] [Gerding & La Poutre, 2004] • “Auction-type”: XOR of ANDs • K-additive: • Captures local substitutability/complementarity effects between k issues • Finding optimal allocation can become hard even for the 2-additive case • Exiting solutions: assume a trusted mediator, computationally expensive (3000-5000 bids for 50 issues) • [Klein, Faratin, Sayama & Bar-Yam, 2003] [Lin 2004] TFG - MARA, Budapest, September 2005

4. Utility graphs: basic ideas • Inspiration: probabilistic graphical models • Each node = one issue under negotiation (or item in a bundle) • Nodes grouped into clusters of connected nodes • Cost of representation • Exponential in size of the cluster • Linear in the number of clusters • Use in negotiation • Opponent modelling: seller maintains & updates a model of buyer’s preferences TFG - MARA, Budapest, September 2005

5. Utility graphs: an example • Global utility is a sum of utility over clusters, rather than individual issues • Buyer - cluster potentials: u(I1) = $7, u(I2) = $5, u(I3) = $0 u(I4) = $0, u(I1, I2)= - $5, u(I2, I3)=$4, u(I2, I4)=$4 • Seller - all items have cost $2. uBUYER(I1=1, I2=0, I3=1, I4=0) = $7 Gains from Trade = Buyer_utility – Seller_Cost Optimal combination? GT(I1=0, I2=1, I3=1, I4=1)=$13 - 3*$2 = $7 TFG - MARA, Budapest, September 2005

6. Utility graphs: Use in negotiation • Bundles with maximal G.T.  Pareto-optimal bundles [Somefun, Klos & La Poutré 2004] • Seller keeps a model of the utility graph of the buyer and aims for a bundle with maximal GT • After each counter-offer, he updates this model (true graph of the buyer remains hidden) • Seller knows a super-graph of possible buyer utility graphs (qualitative assumption) TFG - MARA, Budapest, September 2005

7. Partitioning a utility graph • Q: How to select the bundle with a maximal GT, with respect to a utility graph learned so far? • A1 (Brute force answer): generate all possible bundles and select the best one. • Complexity for 50 issues: 250 > 1015 bundles • A2: Partition the graph into sub-graphs • Nodes belonging to more than 1 subgraph = cutset nodes • For all possible instantiations of cutset nodes, compute local sub-bundle combination • Merge them, such that a local optimum is achieved TFG - MARA, Budapest, September 2005

8. Partitioning a utility graph (2) • Complexity of exploring all bundles: 2c * (2p + 2q) • Partitions can be found in polynomial time (always for graphs of tree-width 2) TFG - MARA, Budapest, September 2005

9. Learning in utility graphs (1) • Seller has a super-graph for possible inter- dependencies in the buyer population • This graph contains tables for each cluster, with size 2 at the power of size of the cluster • Initial values = proportional to the Hamming distance Values are adjusted as follows: , for the combination induced from buyer’s bid , for all other combinations TFG - MARA, Budapest, September 2005

10. Learning: a simple example • Two complementary issues: I1 and I2 Buyer asks, for several rounds: I1=0, I2=1 This combination gets updated with (1+α), the others with (1-α) • Supposing costs are c(I1)=c(I2)=$3, α=0.2the bundle with maximal GT changes from (1,1) to (0,1) after 2 steps TFG - MARA, Budapest, September 2005

11. Learning in utility graphs (2) • The cluster update factor is clique-specific: • |C| = total number of cliques; α, β = learning parameters • Where the clique Gains from Trade Ratio is defined as ratio of “local” (per clique) vs. total (bundle-wide) GT: • We adjust the model more towards the other’s value for clusters which are less important, and less for the others TFG - MARA, Budapest, September 2005

12. Experimental validation: set-up • Graph with 50 issues, 28 clusters: 3 of size 4, 16 of size 3, 6 of size 2, 3 of size 1 • Costs and strength of interdependencies: drawn from a independent, normal distributions (i.i.d-s): • Means around 1*(Hamming Distance) • Spreads between 0 and 5 • => highly non-linear search space • Results averaged for 100 tests/configuration TFG - MARA, Budapest, September 2005

13. Experimental results TFG - MARA, Budapest, September 2005

14. Negotiation part: Conclusions • It is possible to reach Pareto-efficient outcomes reasonably fast, by exploiting the decomposable structure of utility functions • Consequence: • We can handle complex negotiations even in time constrained domains / with buyer impatience • Assumption: A structure of the super-graph for the population of likely buyers • Solution: collaborative filtering past negotiation data TFG - MARA, Budapest, September 2005

15. Structure of the initial utility graph • Preferences of buyers are in some way clustered • Class (population) of buyers with similar preference structures => largely overlapping utility graphs • Can we estimate which items can be potentially complementary/substitutable by looking at previous buying patterns? • Collaborative filtering asks the same questions ! • Not all relationships hold for all users – only a super-graph of these relationships is required TFG - MARA, Budapest, September 2005

16. Architecture & simulation model view TFG - MARA, Budapest, September 2005

17. Collaborative filtering: Overview • Output recommendations to buyers, based on previous buy instances • User-based: for each user, select a neighbourhood of users with a similar preferences • Item-based: identify relationships between items, based on previous buying patterns • In our case, recommendation step is completely replaced by negotiation => more customization possible TFG - MARA, Budapest, September 2005

18. Step 1: Data preparation 4 Item-item matrixes Negotiation outcomes matrix • 1-1 pairs: Ni,j(1,1) • 1-0 pairs: Ni,j(0,1) • 0-1 pairs: Ni,j(1,0) • 0-0 pairs: Ni,j(0,0) TFG - MARA, Budapest, September 2005

19. Criteria 1: Cosine-based similarity • Measure of distance between the buying vectors for two items i, j • Intuitive, but not so precise • Complementarity effect: • Substitutability effect: TFG - MARA, Budapest, September 2005

20. Criteria 2: Correlation-based similarity • Average buys per item: • Similarity between items i and j: TFG - MARA, Budapest, September 2005

21. Results: Correlation-based similarity TFG - MARA, Budapest, September 2005

22. Conclusions & discussion • Utility graphs efficient way to guide online learning of buyer preferences in electronic negotiations • Learning a starting structure of these graphs – possible through collaborative filtering • By combining the two techniques => relatively short negotiations (around 20 steps/50 issues) • Intuition: we explicitly utilize the clustering effect between utility functions of typical buyers • Personalization techniques used in collaborative filtering can be successfully combined with personalization through agent-mediated negotiation TFG - MARA, Budapest, September 2005

23. Questions • Thank you very much for your attention! • Full paper(s) available from: • homepages.cwi.nl/~robu TFG - MARA, Budapest, September 2005