Intelligent Agents – state of the art –

1. Intelligent Agents– state of the art – Aleksander Pivk Materials collected at: America School on Agents and Multi-agent Systems(University of Southern California)

2. 69th Solomon Seminar SYSTEM output input ENVIRONMENT What is an agent? • The main point about agents is that they are autonomous: capable of acting independently, exhibiting control over their internal state. • Thus: an agent is a computer system capable of autonomous action in some environment.

3. 69th Solomon Seminar What is an agent? • Trivial (non-interesting) agents: • thermostat • An intelligent agent is a computer system capable of flexibleautonomous action in some environment.By flexible, we mean: • reactive; • pro-active; • social.

4. 69th Solomon Seminar Intelligent Agents and AI • – A little intelligence goes a long way! – • Oren Etzioni, speaking of commercial experience of NETBOT, Inc.: • Microsoft Office Assistant • We made our agents dumber and dumber…until finally they made some money.

5. 69th Solomon Seminar off; if s=temperature OKon; otherwise Purely Reactive Agents • Some agents decide what to do without reference to their history – they base their making decision entirely on the present, with no reference at all to the past. • We call such agents purely reactive:action: S A • A thermostat is a purely reactive agent:action(s)=

6. 69th Solomon Seminar AGENT see action output input ENVIRONMENT Perception • Introduce the perception system: • The see function is the agent’s ability to observe its env., whereas the action function represents the agent’s decision making process. • New functions:see: S P {maps environment states to percepts}action: P*A {maps sequences of percepts to actions}

7. 69th Solomon Seminar AGENT see action state next output input ENVIRONMENT Agents with State • Lets consider agents that maintain state: • Have some internal data structure, used to record inf. about the env. state and history. • Let I be the set of all internal states of the agent. • Functions:see: S P {maps environment states to percepts}action: I A {maps from internal states to actions}next: I  P  I {maps an internal states and percept to IS}

8. 69th Solomon Seminar Application & Research Domains • Learning Agents • Embodied Agents • Logics for Agents • Coordination, Cooperation, Collaboration • Market-based Multi-agent Systems

9. 69th Solomon Seminar Learning Agents • Why should agents learn? • Learning user and world models, action-to-utility mappings, problem solving • Learning modalities • from users (observation, interaction, being told) • from other agents (collaborative filtering, from experts) • from experience (supervised, reinforcement, probabilistic models) • Learning techniques • neural/decision networks, decision trees,reinforcement learning, instance based learning • Assistant agents (work effort, productivity)

10. 69th Solomon Seminar Learning for Information Agents • Information agents • Access information from a variety of data sources • Integrate the data from these sources • Monitor and provide notifications • Technical challenges • Turning semi-structured data into structured data • Ensuring continued access to the data • Resolving naming inconsistencies across sources • Building agents that efficiently execute their tasks

11. 69th Solomon Seminar Country Information Agent World Governments Agent NATO Members CIA World Factbook 1995 1996 1997

12. 69th Solomon Seminar Prediction Historical Flight Data Learned Flight Delay Predictor Historical Weather Data Flight Delay Prediction Agent YahooWeather Agent

13. 69th Solomon Seminar Real Estate Notification Agent New Listing: 3br 2bath200K Send EmailNotification

14. 69th Solomon Seminar Travel Planning Agent

15. 69th Solomon Seminar Wrapper Induction Problem description: • Web sources present data in human-readable format • take user query • apply it to data base • present results in “template” HTML page • To integrate data from multiple sources, one must first extractrelevant information from Web pages • Task: learn extraction rules based on labeled examples • Hand-writing rules is tedious, error prone, and time consuming

16. 69th Solomon Seminar Example of Extraction Task NAMECasablanca Restaurant STREET 220 Lincoln Boulevard CITY Venice PHONE(310) 392-5751

17. Assumes items are always in fixed, known order … Name: J. Doe; 1 Main; 111-1111. <p>Name: E. Poe; … Introduces several types of wrappers LR: Advantages: Fast to learn & extract Drawbacks: Cannot handle permutations and missing items Must label entire page 69th Solomon Seminar Phone Addr Name WIEN [Kushmerick et al ‘97, ‘00] Name: ; : ; : .

18. Learns a transducer Advantages: Also learns order of items Allows item permutations & missing items Uses wildcards (eg, Number, AllCaps, etc) Drawback: Must “see” all possible permutations 69th Solomon Seminar Addr Name Phone SoftMealy [Hsu & Dung, ‘98] Addr: ; ; Name: ; Phone: Phone: .

19. Learns underlying HTML template WHIRL “soft logic” to measure document similarity Name: html_table_tr_td Address:html_table_tr_td_td Advantages: Learns from unlabeled data Explicitly exploits HTML structure Disadvantage: Not as expressive as previous ones Works only at the level of “HTML nodes” 69th Solomon Seminar WHIRL Wrappers [Cohen ’99]

20. Hierarchical wrapper induction Decomposes a hard problem in several easier ones Extracts items independently of each other Each rule is a finite automaton Advantages: Powerful extraction language (eg, embedded list) One hard-to-extract item does not affect others Disadvantage: Does not exploit item order (sometimes may help) 69th Solomon Seminar STALKER [Muslea et al, ’98 ’99 ’01]

21. 69th Solomon Seminar Extraction Rules Extraction rule: sequence of landmarks SkipTo(Phone) SkipTo(<i>) SkipTo(</i>) Name: Joel’s <p> Phone: <i> (310) 777-1111 </i><p> Review: …

22. 69th Solomon Seminar More about Extraction Rules Name: Joel’s <p> Phone: <i> (310) 777-1111 </i><p> Review: … Name: Kim’s <p> Phone (toll free) : <b> (800) 757-1111 </b> … Name: Kim’s <p> Phone:<b> (888) 111-1111 </b><p>Review: … Start:EITHERSkipTo( Phone :<i> ) OR SkipTo( Phone ) SkipTo(: <b>)

23. 69th Solomon Seminar EC Tree Learning the Extraction Rules GUI Inductive Learning System Extraction Rules Labeled Pages

24. 69th Solomon Seminar Example of Rule Induction Training Examples: Name: Del Taco <p> Phone (toll free) : <b> ( 800 ) 123-4567 </b><p>Cuisine ... Name: Burger King <p> Phone : ( 310 ) 987-9876 <p> Cuisine: … Initial candidate: SkipTo( ( )

25. 69th Solomon Seminar Example of Rule Induction Training Examples: Name: Del Taco <p> Phone (toll free) : <b> ( 800 ) 123-4567 </b><p>Cuisine ... Name: Burger King <p> Phone : ( 310 ) 987-9876 <p> Cuisine: … Initial candidate: SkipTo(() …SkipTo(Phone) SkipTo(:) SkipTo(() ... SkipTo(<b> ()...SkipTo(Phone) SkipTo(() ...SkipTo(:)SkipTo(()

26. 69th Solomon Seminar Multi-view Learning Two ways to find start of the phone number: SkipTo( Phone: ) BackTo( ( Number )) Name: KFC <p> Phone: (310) 111-1111 <p> Review: Fried chicken …

27. 69th Solomon Seminar RULE 1 RULE 2 - + + + - - Co-Testing + - Labeled data Unlabeled data

28. 69th Solomon Seminar Name: Chez Jean <p> Phone: (310) 666-1111 <p> Review: ... Name: Burger King <p> Phone: (818) 789-1211 <p> Review: ... Name: Café del Rey <p> Phone: (310) 111-1111 <p> Review: ... Name: KFC <p> Phone:<b> (800) 111-7171 </b> <p> Review:... Co-Testing for Wrapper Induction BackTo( (Number) ) SkipTo( Phone: ) Name: Joel’s <p> Phone: (310) 777-1111 <p>Review: ... Name: Kim’s <p> Phone: (213) 757-1111 <p>Review: ...

29. Embodied Agents

30. 69th Solomon Seminar Embodied Agents • Animated agent research integrates: • Artistic animation • Computer graphics • Intelligent agents • Why build animated agents? • For more effective communication • For artistic effect • As models for robotic or human agents • When behavior cannot be scripted • e.g., due to interactions with people and other agents • In agents, we begin to see dynamic models of thought and action

31. 69th Solomon Seminar Animated Pedagogical Agents • Animated characters that: • Interact with students in learning envs. • Help keep learning on track • Act as guides, tutors, teammates • Engage in instructional dialog • Enhance motivation and interest • APA’s require: • Realistic, lifelike behavior • A rich set of cognitive and social abilities for effective learning

32. 69th Solomon Seminar Steve: An Embodied Intelligent Agent for Virtual Environments • J. Rickel, L. Johnson, M. Thiebaux, et al. • 3D agent that interacts with students in virtual environments • Can work together with multiple students and multiple users

33. 69th Solomon Seminar Cognition Steve Domain knowledge General capabilities Motor commands Current state Translate into movements, speech Filter, assemble into coherent view Motor Control Perception Broadcast to environment Monitor events Commands to environment Event notifications Virtual Environment Steve’s Architecture (detailed)

34. 69th Solomon Seminar Mission Rehearsal Exercise

35. As a process As an outcome 69th Solomon Seminar Coordination Decisionmaking takes others into account Individuals are not thwarted by others Cooperation Individual decisions further the collective welfare Individuals appear to be “working together” Collaboration Individuals work toward shared goals Common goals achieved well and/or efficiently Co. & Co. & Co.

36. 69th Solomon Seminar Example: Hidden Pictures • Simple (visual) search task • How would YOU work as a part of a team to solve it ?

37. 69th Solomon Seminar Market-based MAS • Marketspace: class of agent interaction env. • What you need to know: • Economic foundations and principles: • Game theory • Price system (general equlibrium) • Auction theory • Design issues and experience • Market models and mechanisms • Trading Agents

38. 69th Solomon Seminar Business games • http://www.cmu.edu/comlabgames-----------------------------------------------------------At the comlabgames website,www.cmu.edu/comlabgames, there are three modules for designing, playing and analyzing the experimental outcomes of games:two person strategic form games, multi-person extensive form games, and auctions and markets. The original comlabgames website is visited on average 50,000 of time each week, and is linked to hundreds of other sites. UCLA mirrors the original site. Comlabgames is very easy to use, and the students just bring their laptops to class, design the games, run them and then analyze thedata. • Vesna Prasnikar, Marko Grobelnik

39. 69th Solomon Seminar Logics for Agents • Symbolic Reasoning • an agent contains an explicitly represented, symbolic model of the world • makes decisions (action to take) via symbolic reasoning • problems: • transduction: how to translate the real world in accurate, adequate symbolic description (speech understanding) • representation/reasoning: how to symbolically represent inf., and how to get agents to reason with this inf. (planning) • Theorem Proving Agents: agent decides what to do by using logic to encode the theory stating the best action in any situation (predicates + rules) • Agent oriented programming: AGENT0 and PLACA

40. 69th Solomon Seminar Logics for Agents • Practical Reasoning (BDI Logic) • is a matter of weighting conflicting considerations for and against competing options, where the relevant considerations are provided by the agent desires/values about and what the agent believes (Bratman). • directed towards actions (theoretical towards belief) • consists of two activities: • deliberation: deciding what state of affairs we want to achieve • means-end reasoning: deciding how to achieve them • implemented BDI agents: IRMA, PRS, Desiderata, LORA