1 / 11

Cobot: A Social Reinforcement Learning Agent

Cobot: A Social Reinforcement Learning Agent. Presented By Deepali Abhyankar. Cobot. RL-based agent for LambdaMOO LambdaMOO – A complex, open-ended, multi-user chat environment

berk-neal
Download Presentation

Cobot: A Social Reinforcement Learning Agent

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Cobot: A Social Reinforcement Learning Agent Presented By Deepali Abhyankar

  2. Cobot • RL-based agent for LambdaMOO • LambdaMOO – A complex, open-ended, multi-user chat environment • Cobot – Interacts with the LambdaMOO users and learns to perform interesting and entertaining actions based on the users feedback

  3. Actions performed by Cobot • Proposing conversation topics • Introducing new users • Engaging in common wordplay routines • To perform actions that seem meaningful, useful or amusing to the users.

  4. Challenges • Choice of an appropriate State space • Multiple Reward sources • Inconsistency and drift of user rewards and desires • Variability in user understanding • Data sparsity • Irreproducibility of experiments

  5. LambdaMOO • Oldest continuously operating MUD (Multi user Dungeons). • A series of interconnected rooms • Rooms populated with users and objects which move between them. • Users communicate through speech and Verbs • A large collection of verbs exists. (1) Buster is overwhelmed by all these deadlines. (2) Buster begins to slowly tear his hair out, one strand at a time. (3) HFh comforts Buster. (4) HFh [to Buster]: Remember, the mighty oak was once a nut like you. (5) Buster [to HFh]: Right, but his personal growth was assured. Thanks anyway, though. (6) Buster feels better now. • The objects are created by the users themselves who devise actions and and control access by other users.

  6. Cobot • Cobot is a software agent that resides in lambda moo. • Connects via telnet. • Is a user with all the rights and responsibilities from the point of view of the LambdaMOO server. • Wanders into the Living Room, where he spends most of his time. • Notes the various events that occur here.

  7. Functions performed by Cobot • Gathering and reporting social statistics. • Can search the web to answer specific questions posed to him. • Topic Change: Introduce a conversational topic. • Initiate a Roll call: someone who is tired of Monica Lewinsky may emote “TIRED OF LEWINSKY ROLL CALL.” Sympathetic users agree with the roll call. • Make a comment describing the current social state of the Living Room. • Introduce two users who have not yet interacted in front of Cobot.

  8. RL State Features • Maintain separate state spaces • Cobot can be viewed as running a large number of separate RL processes in parallel, with each process having a different state space. • The state space for a user contains a number of features containing statistics about that particular user.

  9. State space of Cobot for a generic user

  10. Each user’s state space is effectively infinite, as there are real-valued state features. • Linear function approximation is used for each user’s policy. • Cobot’s RL actions are chosen according to a mixture of the policies of the users present.

More Related