Virtual Sales Agents for Electronic Commerce

1. Virtual Sales Agents for Electronic Commerce Monday, 2 July Wolfgang Wahlster German Research Center for Artificial Intelligence DFKI GmbH Stuhlsatzenhausweg 3 66123 Saarbruecken, Germany phone: (+49 681) 302-5252/4162 fax: (+49 681) 302-5341 e-mail: wahlster@dfki.de WWW:http://www.dfki.de/~wahlster

2. Outline • Virtual Sales and Shopping Agents • Virtual Webpages • Life-like Characters as Virtual Sales Agents • Virtual Sales Teams • Information Extraction Agents for E-Commerce • Wrapper Induction and Programming by Example • Ontological Annotation of Webpages • Encoding Rule Knowledge for E-Commerce Agents • Conclusions

3. Intelligent Agent Technology is a Prerequisitefor Advanced WebCommerce Shopbots for Automated Comparison Shopping Virtual Web Pages Text Analysis and Generation Multimedia Presentation Planning Information Extraction from HTML/XML Documents Advanced WebCommerce User Modeling and Language Generation Coordinated Text & Graphics Planning Machine Translation Robust Dialogue Understanding Advanced Speech Synthesis One-to-One Marketing Intuitive, Multilingual Access Multimodal Interfaces Dialogue with Virtual Sales Agents

4. Five Generations of Internet Applications 2000 Embedded Internet Agents Mobile Internet Services WWW EMail Research Net 1 2 3 4 5 Every Car has a homepage, Agents are main Internet users, Ubiquitious Computing t Internet Access via WAP and UMTS devices

5. What are Virtual Sales Agents? l appear as life-like characters l plan interactive behavior autonomously l can initiate interaction ACTIVE l understand the user‘s requests l answer clarification questions l allow mixed initiative dialogs l respond immediately to interruptions l criticism and clarification questions l direct manipulation RE- ACTIVE INTERNET AGENTS INTER- ACTIVE PROACTIVE l anticipate the user's needs l adopt the user's goals l provide unsolicited comments

6. Intelligent Web Services Provider Consumer Netbot sells  Information  Goods  Services buys  Information  Goods  Services  Intelligent Parallel Retrieval  Information Extraction and Summarization  Personalized Presentation  Matchmaking  Teleshopping Assistance  Telemarketing Assistance  Translation Services  Data Mining Services Knowledge about:  Usage Patterns  User Models  Consumer Profiles Web Sites

7. XML-based Negotiation between Shopping and Sales Agents Negotiation based on the Exchange of XML Documents Shopping Agent Customer Sales Agents Companies - Call for Bids - Offer - Criticism - Alternate Offer

8. Virtual Market Places with Human and Machine Agents

9. Performance Ranking of Comparison Shopping Agents Ranked List of all Shopping Agents for a Product Category Performance Ranking Agent Benchmark Problem Jango Pricescan Buybuddy Cadabra MySimon Roboshopper Comparison Shopping Agents

10. Three Generations of Web Sites First Generation Second Generation Third Generation Virtual Webpages Interactive Web Sites Static Web Sites Netbots, Information Extraction, Presentation Planners JavaScripts and Applets User Modeling, Machine Learning, Online Layout Database Access and Template-based Generation Fossils cast in HTML Dynamic Web Sites Adaptive Web Sites

11. The Idea of Personalized Netbots Personal Assistants e.g. MetaCrawler Softbots Indices, Directories, Search Engines Mass Services WWW Traveller’s Netbot: Tries to achieve traveller’s goals (finding and executing plans)  checks availability  finds best price  uses personal preferences (e.g. frequent flyer programme, seating preferences)  lets the traveller know, when seats become available (active help)

12. What is a Virtual Web Page? Virtual Memory, Virtual Relation, Virtual Reality... A Virtual Web Page l is generated on the fly as a combination of various media objects from multiple web sites or as a transformation of a real web page. l looks like a real web page, but is not persistently stored. l integrates generated and retrieved material in a coordinated way. l can be tailored to a particular user profile and adapted to a particular interaction context. l has an underlying representation of the presentation context so that an Interface Agent can comment, point to and explain its components.

13. Virtual Webpage Retrieved from 5 Different Servers

14. AiA: Information Integration for Virtual Webpages PAN Travel Agent Andi Car Route Planner Yahoo News Server Yahoo Weather Server Gault Millau Restaurant Guide Hotel Guide

15. The Generation of Virtual Webpages with PAN and AiA Address Hotel Agent Map Agent AiA Presentation Planner Pictures and Graphics Netbot PAN Pieces of Text Components of virtual Webpages Virtual Web Presentation Coordinates for Pointing Gestures Trip Data Input for Speech Synthesis Persona Server Icons for Hyperlinks Constraint- based Online Layout Weather Agent Train & Flight Scheduling Agent Major Event Agent

16. Persona as a Personal Travel Consultant

17. The Combination of Retrieved and Generated Media Objects for Virtual Webpages Multi-Domain Problem Specs NETBOT Multiple Data Sources Distributed Information Information Structures l Relations, Lists l KR Terms Media Objects l Texts, Sounds, Videos l Pictures, Maps, Animations Retrieved Results

18. The Combination of Retrieved and Generated Media Objects for Virtual Webpages Information Structures l Relations, Lists l KR Terms Media Objects l Texts, Sounds, Videos l Pictures, Maps, Animations Retrieved Results Select Canned Media Objects Design New Media Objects Coordinate Media Objects Transform Media Objects l Icons, Clip Art l Frames, Sounds l Graphics, Animation l Text, Speech, Mimic l Temporal Synchroni- zation l Spatial Layout l Clip, Convert, Abstract l Zoom, Pan, Transition Effects Select & Design Reuse & Transform

19. Operational Models of Referential Semantics for Robots and Netbots (Wahlster 1999) Robot Netbot Set of Subsumption Relations in an Ontology Set of Subsumption Relations in an Ontology “Screw” “Departure Time” Set of Recognizers Set of Wrappers Physical Objects WWW Objects ... ... Screw 1 Screw N DT 1 DT N

20. The Role of Ontological Annotations for the Generation and Analysis of Virtual Webpages (Wahlster 1999) Webpages with Ontological Annotations Webpages without Ontological Annotations Information Extraction Agent Presentation Planner Virtual Webpage With Ontological Annotations in: SHOE, OML, XOL,OIL, DAML and Persona Annotation in PML Information Extraction Agents TriAS Presentation Agent Persona

21. A Natural Language Agent for Finding Pre-Owned Porsche Cars Boxter, not red, must have AC, less than 20k

22. Towards Mobile and Speech-based E-Commerce Using UMTS Phones l UMTS phones (Wireless Application Protocol for Cellular Phones) l WML as a markup language for interactive content l Mobile access to virtual shops allows price comparisons during real shopping l Multimodal dialog: Voice In (Speech) - Web Out (Graphics, Hypertext) l Voice input using advanced speech understanding technology l Easy to use: customers simply say what they want

23. Enhanced ECommerce through Personalization System is able to flexibly tailor product presentations to the individual user and the current situation. An animated character serves as “Alter Ego” of the presentation system. Personalized Presenters at DFKI

24. Personified Agents Increase the User's Trust in the System's Presentation Experimental evidence for effects of modality on the user's trust (van Mulken, 1999) The system gives recommendations, which turn out to be wrong in some cases. How much does a user trust the system's advice depending on the modality of a presentation? 1.0 0.8 0.7 0.6 0.5 Self-animated Persona, Speech, Gesture, Facial Expression, Pointing Speech, Graphical Highlighting Text, Graphical Highlighting

25. Impact of the modality of a Presentation on the User's Trustfulness Result:Persona > Speech > Text Conclusion: If the presentation is more human-like, recommendations are more readily followed For l decision support systems l tutoring systems l recommendation systems l virtual sales agents personified interface agents have a clear advantage: They increase the user's trust in the information presented by the system

26. PPP’s Persona Server implements a generic Presentation Agent that can be easily adapted to various applications Visual Appearances Behaviors Hand-drawn l Presentation Gestures Cartoon l Reactive Behaviors Bitmaps l Idle-time actions l Navigation actions Persona Server Auditory Characteristics Video Bitmaps l Sound effects, auditory icons l Voice: male, female Generated Bitmaps from 3D-Models

27. Use of a Life-like Character for Electronic Commerce Digital Assistant Selector

28. DFKI‘s PET-Technology: Flexible Realization of Virtual Sales Agents Agent in its own Frame Sales Agent on Desktop Sales Agent on Webpage simple implementation limited presentation behavior advanced presentation behavior complex implementation very advanced presentation behavior download of sales agent

29. Classification of Persona Gestures Talking Posture 1 cautious, hesitant appeal for compliance avoids body-gestures Talking Posture 2 active, attentive self-confident uses body-gestures Gesture Catalogue

30. Context-Sensitive Decomposition of Persona Actions High-Level take-position (t t ) point-to (t t ) 1 2 4 3 Persona Actions Context-Sensitive r-stick-pointing (t t ) move-to (t t ) Expansion 1 2 3 4 (including Navigation Actions) Decomposition r-turn (t t ) 1 21 r-hand-lift (t t ) into 3 31 r-step (t t ) Uninterruptable 21 22 Basic Postures f-turn (t t ) r-stick-expose (t t ) 22 2 31 4 Bitmaps ... ... ... ...

31. Extensions of the Representation Formalism Distinction between production and presentation acts (i.e. Persona- or display acts) Explicit representation of qualitative and quantitative constraints Production Act Presentation Act Introduce S-Show S-Position Elaborate-Parts Create- Graphics S-Wait Label Label S-Create- S-Depict Window S-Speak S-Point S-Speak S-Point Qualitative constraints: Create-Graphics meets S-Show, ... Metric constraints: 1 <= Duration S-Wait <= 1, ...

32. PET: Persona-Enabling Toolkit Objective: l Enable non-professional computer users to populate their web pages with lifelike characters PET comes with: l a set of characters and basic gestures l an easy-to-learn Persona markup language Developer’s PET will include: la character design tool which enables users to build their own characters Technical Realization: l Based on XML and Java

33. The Persona Markup Language <html> <head> <title> Persona Test </title> </head> <body> <persona bitmap=“cartoon” ...> <uselib url= .../> <do name=“greet”/> <do name =“speak” args=“hello”/> </persona> </body> </html> Specification of the character to be used Specification of Persona actions Features: • XML-based • easy to learn

34. Functional View of PET Bitmaps Webpage with Reference to Java Applet URL of Webpage with Persona Tag <html> ... <APPLET archive=“personaplayer.jar”...</APPLET> ...</html> PET Application Server <html> <head> <title> Persona Test </title> </head> <body> <persona bitmap=“cartoon” ...> <uselib url= .../> <do name=“greet”/> <do name=“standard”/> <do name =“speak” args=“hello”/> </persona> </body> </html> PET Parser Persona Scripts waitscreen 4 gesture greet 0 0 null gesture laugh 0 0 null ... PET Generator Persona Engine Behavior Monitor Event Handler Character Composer Audio Data

35. The Bidirectional Control Flow onPersona-Enabled Webpages l Mouse Clicks l Mouse Movements Triggers actions of the Persona l Text Input l Speech Input l Menu Input l Direct Manipulation Input Triggers operations on elements of the webpage Web Persona

36. Sending Interface Agents to Clients: Plug-Ins or Applets? Plug-Ins Applets l Add features (character players) to browser l Download triggered by user l Requires disk space on client l Unrestricted access to client l Less appropriate for WebCommerce, Guides l Agents integrated in 3D environments l Appropriate for Entertainment Examples: lExtempo's Jennifer James (Hayes-Roth et al. 98) l PFMagic's virtual petz l Java animation code sent over the net l Automatic loading l Requires no disk space on client l Restricted access to client l Appropriate for WebCommerce, Guides l Agents integrated in 2D environments l Less appropriate for Entertainment Examples: l DFKI's Web Persona (Müller et al. 98) l ISI's Adele (Johnson et al 98) New in AiA/PAN: Balanced combination of Applets and Servelets Efficient distribution of client-side Java and server-side Java for driving the Interface Agent

37. Porsche 9 11 & Boxter

38. Persona Active Elements (PAE) • l Active Images • An active image starts a persona action when clicked. • l Addressable Objects • An addressable object is an object which can be addressed and • manipulated by Persona via its name and its position. <ACTIVEIMAGE SRC=“image” HREF=“url” NAME=“image name” STATUS=“status message” ALT=“tooltip” CACTION=“persona action onClick” OACTION=“persona action on MouseOver” ...> <PDIV DIVNAME=“name of the element” DVFRAME=“frame name” TOP=“anchor-y” LEFT=“anchor-x”>some HTML elements</PDIV>

39. A Virtual Sales Agent for OTTO – World’s Largest Tele-Ordering Company

40. DFKI’s Ecommerce Presentation Planner has been extended to accommodate for various target platforms through the introduction of a mark-up language layer Presentation Planner PET- PML Agent Script WML SMIL PET Persona Player WML-Browser MS-Agent Controller SMIL Player

41. Simulated Dialogues as a Novel Presentation Technique Presentation teams convey certain rhetorical relationships in a more canonical way l Provide pros and cons The single presenters can serve as indices which help the user to classify information. l Provide information from different points of view, e.g. businessman versus tourist Presentation teams can serve as rhetorical devices that allow for a continuous reinforcement of beliefs l involve pseudo-experts to increase evidence

42. Presentation Teams for Advanced ECommerce I recommend you this SLX limousine.

43. Underlying Knowledge Base Representation of domain l FACT attribute car_1 consumption_car_1 Value dimensions for cars adopted from a study of the German car market l safety, economy, comfort, sportiness, prestige, family and environmental friendliness l FACT polarity consumption_car_1 economy negative Difficulty to infer implication of dimension on attribute l FACT difficulty consumption_car_1 economy low

44. Example of a Dialogue Strategy Question: How much gas does it consume? Answer: It consumes 8l per 100 km. Negative Response: I’m worrying about the running costs. Dampening Counter: Forget about the costs. Think of the prestige! Header: (dampening_counter ?agent ?prop ?dim) Constraints: (*and* (positive ?agent) (pol ?prop ?other_dim positive)) Inferiors: (Speak ?agent (“Forget about the ” ?dim “!”)) (Speak ?agent (“Think of the ” ?other_dim “!”))

45. Multiple Interface Agents for User-adaptive Decision Support User-Adaptive Search Planning weighted propositions ... ... Multiple Decision Support Agents Spare parts for this car are rather expensive! But, it’s fast!

46. MAUT (Multi-Attribute Utility Theory) - I formalism for the evaluation of structured objects basic idea: identification of relevant dimensions for the evaluation of an object total evaluation of an object : • evaluation of the object regarding the dimension : • definition of the relationships between dimensions and attributes within the ontology  decision support: connection between currently focused data and user preferences

47. MAUT (Multi-Attribute Utility Theory) - II 10 10  10 0 0  0 0 0 1 1 10000 0 hotel user´s interest on the dimension 0.1 0.5 0.4 cheapness culture sportiness dimension relative weight of an attribute for the dimension 0.8 0.5 0.2 0.5 ...   10 evaluation function   0 0 1 price golf tennis

48. MAUT - Example  hotel 1: l tennis l golf l price: 5000 DM  hotel 2: l tennis l price: 2000 DM  

49. Research Topics: Multiple Interface Agents Interactive Presentation Teams Corpus-based Approach to Gesturing Empirical Evaluation of Presentation Teams

50. Non-Interactive Presentation Teams I recommend you this SLX limousine.