Agent Reputation and Trust (ART) Testbed

1. Agent Reputation and Trust (ART) Testbed Craig Dillabaugh cdillaba@connect.carleton.ca 100253992 Fullam, K., T. Klos, G. Muller, J. Sabater, A. Schlosser, Z. Topol, K. S. Barber, J. Rosenschein, L. Vercouter, and M. Voss. (2005) "A Specification of the Agent Reputation and Trust (ART) Testbed: Experimentation and Competition for Trust in Agent Societies," The Fourth International Joint Conference on Autonomous Agents and Multiagent Systems (AAMAS-2005), Utrecht, July 25-29, pp. 512-518.

2. Agent Trust • In a Multi-agent System an agent may require resources provided by other agents. • An agent benefits when such interactions are ‘successful’ but such interaction can put the agent at risk. • An agent may minimize risk by accurately predicting outcome of interactions. • Modeling ‘trustworthiness’ of other agents lets the agent make these predictions.

3. You don’t want to delegate important tasks to untrustworthy agents!

4. Accurate Adaptive Quickly converging Multidimensional Efficient Identifying and isolating untrustworthy agents and deciding with whom to interact. Evaluating an interaction’s utility. Trust Model Research Objectives Build trust models that are: Agent’s ability to translate models for:

5. Need for a Trust Testbed • Growth of trust modeling algorithms. • No unified performance benchmarks for comparing technologies. • Need for transparent, recognizable, and objective standards to justify systems and provide baseline for research and evaluation.

6. Agent Reputation and Trust (ART) Testbed Initiative • Goal: to establish a testbed for inter-agent trust and reputation related technologies. • Provides a competitive forum for the comparison of Agent trust technology. • A suite of tools allowing researchers to perform repeatable experiments.

7. Testbed Characteristics • Multipurpose Design • Competition and Experimentation • Modularity • Accessibility • Objective Metrics • Problem Focus The competition involves …

8. Art Appraisal courtesy of: http://meatboxrecords.tripod.com/artz/ground_chuck_group_shot.gif Courtesy of Webmuseum, Paris: http://www.ibiblio.org/wm/paint/auth/monet/waterlilies/

9. Competition Basics • Agents function as painting appraisers with varying levels of expertise for specific eras. • An agent maximizes its own ‘profit’ by giving accurate appraisals. • An agent without expertise for a given era may request advice from other agents more expert in that era.

10. Client Appraisals • Multiple client’s present agent with paintings for appraisal, pay fixed fee. • An agent’s own appraisal is selected from a normal error distribution (by the Simulation) but is weighted by expertise and effort. • Agents get return business based on accuracy of appraisals.

11. Opinion Transactions Provider Provider Requester Provider Weights associated with opinion from each provider. Simulation Engine (Calculates the final appraisal of each appraiser)

12. Reputation Transactions • Are less expensive than opinion transactions. • Requester asks Provider about their opinion on a specific agent with respect to its trustworthiness for a given era. • The Provider can lie!

13. Client Shares & Analysis • Client shares distributed among Agents based on accuracy of appraisals. • Appraisal strategies analyzed according to agent and system-wide metrics. • Agent Perspective – utility to the single agent. • System assess the quality of the Agent’s trust based decisions. • Agent who finished with the most money WINS! • System Perspective – Social Welfare

14. Critique • The good: • Art appraisal system requires strong social component. • System allows Agent’s to model various ‘dimensions’ of trust. • The Not so good: • Is a single competition adequate?

15. References ART Testbed Web Site (2005) http://www.lips.utexas.edu/art-testbed Source Forge Project Page (2005) http://sourceforge.net/project/showfiles.php?group_id=148987 D. Kim, Y. Song, S. Braynov, H. Rao. A B-to-C Trust Model for On-line Exchange, Americas Conference on Information Systems, pp. 784-787, Boston, 2001.