1 / 12

Motivated Reinforcement Learning for NPC in Game Worlds

This paper explores the use of motivated reinforcement learning to create non-player characters (NPCs) in computer game worlds that can evolve and adapt to their environment. It discusses the motivation, objective, method, experiments, and conclusions of the research.

michellep
Download Presentation

Motivated Reinforcement Learning for NPC in Game Worlds

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Motivated Reinforcement Learning for Non-Player Characters in Persistent Computer Game Worlds Advisor : Dr. Hsu Presenter : Chia-Hao Yang Author : Kathryn Merrick, Mary Lou Maher SIGCHI  06

  2. Outline • Motivation • Objective • Introduction • Method • Experiments • Discussion • Conclusions • Habituation SOM • Q-learning

  3. Motivation • Many NPC possess a fixed set of pre-programmed behaviors and lack the ability to adapt and evolve in time with their surroundings.

  4. Objective • To create NPC that can both evolve and adapt with their environmental.

  5. Introduction • Current technologies for NPCs • Reflexive agents • Only recognized states will produce a response • State machines & rule-based approaches • EX : Baldur Gate & Dungeon Siege • Learning agents • It can modify their internal structure to respect to some task. • Black and White • Reinforcement learning agents • The agent records the reward signal. • Then chooses an action which attempts to maximize the long-run sum of the values of the reward signal. • Tao Feng

  6. S(t-1) – S(t-2) S(t) – S(t-1) Method • Motivated reinforcement learning agents • It use a motivation function to directs learning. • Skill development is dependent on the agent’s environment & these skills are developed progressively over time. Q-learning

  7. Experiments • In order to experiment with MRL agent, we implemented a village scenario in Second Life. • Support character • Trades people • Location, object, inventory sensor • Move to object, pick up object, use object effector • Ex : the pick, when used on the mine, will produce iron which can converted to weapons when used near the forge

  8. Experiments • Partner character • Vendor character • Location, object sensor • Move to object effector • Ex : In Ultima Online players can set up vendor characters to sell the goods they have crafted.

  9. Conclusions • This paper has presented MRL agents as a means of creating non-player characters which can both evolve and adapt. • MRL agents explore their environment and learn new behaviors in response to interesting experiences, allowing them to display progressively evolving behavioral patterns.

  10. Habituation SOM • An HSOM consists of a standard Self-Organizing Map with an additional habituating neuron connected to every clustering neuron of the SOM.

  11. Q-Learning • It’s a part of reinforcement learning algorithm which has been widely used for many applications such as robotics, multi agent system, game, and etc. • It allows an agent to learn through training without teacher in unknown environment. • Modeling the Environment • putting similar matrix name Q in the brain of our agent reference

  12. Q-Learning • algorithm • example …… reference

More Related