An experience in using ontologies for automated negotiations

1. An experience in using ontologies for automated negotiations Valentina Tamma Agent ART Group Department of Computer Science University of Liverpool

2. Objective • The main objective of the project is to facilitate automated negotiation in open environments: • Agents engage in negotiation without prior knowledge of the negotiation mechanism used; • Agents are able to exchange knowledge about arbitrary negotiation mechanisms; • Reduce the amount of knowledge hardcoded in the agent; • Ontological representations could be used to support the process of knowledge sharing;

3. Negotiation Ontology Negotiation in open environments Agent Request Protocol Type NegotiationProtocol X + Strategy Send Protocol • Flexibility • Open environments

4. Why a Negotiation Ontology • makes the assumptions on the negotiation process explicit • shared as merge of a number of accepted classification frameworks: Lomuscio and colleagues, 2000 the generic framework for automated negotiation developed at HP Labs (Bartolini and colleagues, 2002) Wurman and colleagues, 2001 London classification at the DEXA 2000 workshop on e-commerce • identified the concepts and the relationships that are shared across most negotiation protocols

5. Negotiation mechanism • In order to model the negotiation mechanisms we have taken the view of negotiation as a process; • The modelling is based on PSL (ISO TC184/SC4), which defines a neutral representation for processes. • PSL has an underlying, formal ontology. All concepts in PSL are formally defined, using the Knowledge Interchange Format (KIF), to eliminate the ambiguity usually encountered when exchanging information among disparate applications; The ontology is modular, we have used only the core that defines high-level, primitive concepts inherent to process specification • The core ontology defines a process as a set of activity types and activity occurrences which happen at timepoints. Relations and functions concerning timepoints (such as before, after) and activities (such as begin-of, end-of) are axiomatically defined; • A simple theory of subactivity is also provided;

8. Making the ontology operative • In order to operationalise the ontology it needs to be translated into an executable language. We chose DAML+OIL because it provides a set of constructs with which to create ontologies and to markup information so that it is machine readable and understandable; • The markup information offers support to semantic web applications; • A number of ontology editors provide automatic translation into DAML+OIL, thus reducing the translation effort; • A number of tools and ontologies (such as the time ontology) are already available to support applications using DAML+OIL;

9. The demonstrator We aimed to show that an ontological representation of the knowledge concerning the rules of encounter could reduce the need to hardcode knowledge in the agent in the agent. The strategy is guided by the ontology Automated negotiation by hardcoding both the protocol and the strategy. Open automated negotiation. Mechanism and strategy determined “on the fly”

10. Implementation issues • The strong axiomatisation of PSL and the need to represent executable rules pose a problem; • The concept in the ontology support only the protocol but not the strategy. Can an agent use a negotiation protocol without any notion of strategy that is, is it enough to explain the rules of encounter? • Need for a language that offers support to rules and some limited reasoning capabilities, DAML+OIL is not sufficient and there is not agreement on how to represent rules on the semantic web (RuleML, etc…)

11. Assumptions In order to prove our aim we have: • We use Jess to represent the ontology because it offers support to rules and it permits some form of reasoning; • Scaled the size of the ontology, considering only the relevant information. This was needed to limit the manual translation effort; • Some generic definitions in the ontology (such as buyer and seller) are hardcoded instead of being sharing through a centralised blackboard; • Other concepts not shared are explicitly expressed by Jess facts and rules; • These facts and rules are used to tune the strategy used in the auction; • The mapping between concepts and axioms in the ontology and facts and rules known to the agents is fuzzy;

14. (deffacts auctioneer-processing-facts (auctioneer-operation-at end-of-round clear) (buyers many) (sellers single) (k-value 0.0) ) (deffunction transaction-price (?k ?askprice ?bidprice) (+ (* ?k ?askprice) (* (- 1 ?k) ?bidprice)) )

15. Strategic Bidding • The problem with is to participate at the negotiation getting the best price • In order to get the best price one need to bid strategically (ie “lie” about your valuation) • Eg In an ascending auction, with private-values, strategic bidding involves refining your value-claim upwards from an initial small offer, in the hope that you will be awarded the resource at the lower price. – Note that incremental bidding is not the only good strategy for an e-Bay style auction, eg “Sniping”

16. What’s the best strategy? • Depends on the rules of the game (ie the negotiation mechanism) • Depends on what strategies are being played by other agents • “Solve” the game: find an equilibrium solution or a dominant strategy • Problem: solving the game for arbitrary games (mechanisms) is uncomputable

17. Learning strategies • • Reinforcement learning has been used to automatically acquire strategies for arbitrary N-player games • • These strategies are not always provably optimal, but the advantage is that we can compute them (by running the learning algorithm) • The problem with this approach is that agents have to go through a learning process before they can negotiate efficiently. • • We can try to use a description of the negotiation mechanism (the ontology) to guide the learning process..

18. Example (deffacts auctioneer-processing-facts (auctioneer-operation-at end-of-round clear) (buyers many) (sellers 1) (k-value 0.0) ) (defrule determine-clearing-price (quote (ask ?ap) (bid ?bp)) (k-value ?k) => (bind ?tp (transaction-price ?k ?ap ?bp)) (assert (transaction-price ?tp)) ) (deffunction transaction-price (?k ?a ?b) (+ (* ?k ?a) (* (-1 ?k) ?b)) )

19. Example ( (buyers many) (sellers single) deffacts auctioneer-processing-facts (auctioneer-operation-at end-of-round clear) (k-value 0.0) ) (deffunction transaction-price (?k ?askprice ?bidprice) (+ (* ?k ?askprice) (* (-1 ?k) ?bidprice)) )

20. Implementing the auctioneer • We extend JASA’s AbstractAuctioneer class, and create a JESSAuctioneer that uses the expert system to reason about what actions to take based on the rules describing the negotiation mechanism. • eg:..

21. Implementing the auctioneer public class JESSAuctioneer extends AbstractAuctioneer { ... public double determineClearingPrice( Shout bid, Shout ask ) { try { jess.reset(); assertString(shout2Jess(J_CLEAR_BID, bid)); assertString(shout2Jess(J_CLEAR_ASK, ask)); assertString(quote2Jess(clearingQuote)); jess.run(); double clearingPrice = jess.fetch(J_TRANSACTION_PRICE).floatValue(jess.getGlobalContext()); return clearingPrice; } .... }

22. Tuning strategies Rules determining best strategies are explicitly coded in the agent knowledge base (defrule tuning-2ndprice (tuning-strategies) (sellers 1) (not (auctioneer-operation-at end-of-round generate-quote)) (k-value 0.0) => (assert (use-strategy "uk.ac.liv.auction.agent.PureSimpleStrategy")) (assert (use-strategy-parameter "delta" 0)) )

23. A strategy heuristic We also code heuristics to improve the learner efficiency and avoid the use of lookup tables (defrule tuning-quotevisible (tuning-strategies) (auctioneer-operation-at end-of-round generate-quote) (sellers many) => (assert (use-strategy "uk.ac.liv.auction.agent.MDPStrategy")) )

24. Lessons learnt • Heavy weight ontologies needed for this application • Lack of appropriate tools for building and translating heavy weight ontologies • Translations had to be done by hand, eventually • We started over ambitiously and had to scale down dramatically

25. Future research directions • Negotiation through a shared ontology: extend the approach to support the whole negotiation, not only the strategy; • Explore the translation of KIF based axioms into a markup language for rules, such as RULE-ML; • Look to alternatives to Process Specification Language, such as negotiation rules, which can be described in terms of preconditions and post conditions; • Determine exactly to what knowledge should be already encoded to the agent and what can be modelled in the ontology; • Explore the possibilities of expressing negotiation protocols in form of problem solving methods:

26. Future research directions • Explore the possibilities of expressing negotiation protocols in form of problem solving methods organised into a library: Domain Factual Knowledge Control Knowledge <conditions on problem state (including goals)> <conditions on domain knowledge> <conditions on data describing the problem instance> <changes to problem solving states (including goal components)> Concepts Attributes Relationships