120 likes | 135 Views
This paper explores the use of motivated reinforcement learning to create non-player characters (NPCs) in computer game worlds that can evolve and adapt to their environment. It discusses the motivation, objective, method, experiments, and conclusions of the research.
E N D
Motivated Reinforcement Learning for Non-Player Characters in Persistent Computer Game Worlds Advisor : Dr. Hsu Presenter : Chia-Hao Yang Author : Kathryn Merrick, Mary Lou Maher SIGCHI 06
Outline • Motivation • Objective • Introduction • Method • Experiments • Discussion • Conclusions • Habituation SOM • Q-learning
Motivation • Many NPC possess a fixed set of pre-programmed behaviors and lack the ability to adapt and evolve in time with their surroundings.
Objective • To create NPC that can both evolve and adapt with their environmental.
Introduction • Current technologies for NPCs • Reflexive agents • Only recognized states will produce a response • State machines & rule-based approaches • EX : Baldur Gate & Dungeon Siege • Learning agents • It can modify their internal structure to respect to some task. • Black and White • Reinforcement learning agents • The agent records the reward signal. • Then chooses an action which attempts to maximize the long-run sum of the values of the reward signal. • Tao Feng
S(t-1) – S(t-2) S(t) – S(t-1) Method • Motivated reinforcement learning agents • It use a motivation function to directs learning. • Skill development is dependent on the agent’s environment & these skills are developed progressively over time. Q-learning
Experiments • In order to experiment with MRL agent, we implemented a village scenario in Second Life. • Support character • Trades people • Location, object, inventory sensor • Move to object, pick up object, use object effector • Ex : the pick, when used on the mine, will produce iron which can converted to weapons when used near the forge
Experiments • Partner character • Vendor character • Location, object sensor • Move to object effector • Ex : In Ultima Online players can set up vendor characters to sell the goods they have crafted.
Conclusions • This paper has presented MRL agents as a means of creating non-player characters which can both evolve and adapt. • MRL agents explore their environment and learn new behaviors in response to interesting experiences, allowing them to display progressively evolving behavioral patterns.
Habituation SOM • An HSOM consists of a standard Self-Organizing Map with an additional habituating neuron connected to every clustering neuron of the SOM.
Q-Learning • It’s a part of reinforcement learning algorithm which has been widely used for many applications such as robotics, multi agent system, game, and etc. • It allows an agent to learn through training without teacher in unknown environment. • Modeling the Environment • putting similar matrix name Q in the brain of our agent reference
Q-Learning • algorithm • example …… reference