1 / 25

CS491j CSMPG

CS491j CSMPG. Lecture Eight: AI Deux. Roadmap. Last Time: Logic, Reasoning, Planning Today: Learning, Modeling, Emotions . Learning. Learning has numerous definitions What is its role in gaming?. Offline Learning. Learning is often slow In games much of what is to be learned is fixed.

LeeJohn
Download Presentation

CS491j CSMPG

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. CS491j CSMPG Lecture Eight: AI Deux CS 491J Computer Science of Multi-Player Games

  2. Roadmap • Last Time: Logic, Reasoning, Planning • Today: Learning, Modeling, Emotions CS 491J Computer Science of Multi-Player Games

  3. Learning • Learning has numerous definitions • What is its role in gaming? CS 491J Computer Science of Multi-Player Games

  4. Offline Learning • Learning is often slow • In games much of what is to be learned is fixed. • Learn offline, use learned policy or model online • Uses? Advantages? Disadvantages? CS 491J Computer Science of Multi-Player Games

  5. Online Learning • Learning online provides adaptability • Customized to a particular player • Responsive to changing environments • Limited learning resources • Mixed-Mode learning? CS 491J Computer Science of Multi-Player Games

  6. Supervised Learning CS 491J Computer Science of Multi-Player Games

  7. Learning: Classification • Learn a discrete decision problem: • Is this player attacking? • Am I winning or losing? • Is the player having fun? • Techniques • Decision Trees • Expert Systems/Case-Based Reasoning • Statistical Models CS 491J Computer Science of Multi-Player Games

  8. Learning: Function Aprox. • Learn a continuous output given inputs • Can be used for classification • How fast should I drive? • What is the probability I’ll win this battle? • How good is this player? • Techniques • Neural Networks • Statistical Models CS 491J Computer Science of Multi-Player Games

  9. Causality: Bayesian Networks • Often times we know the basic structure of how things happen • But we observe limited information • We can still reason about what happened CS 491J Computer Science of Multi-Player Games

  10. Example Network Friendly Attack On Lava Enemy Attack Unit Dies Unit Dies CS 491J Computer Science of Multi-Player Games

  11. Learning Bayes Nets • Structure learning is hard (and often unnecessary) • Probability distribution learning is hard (but tractable in many circumstances) CS 491J Computer Science of Multi-Player Games

  12. Reinforcement Learning • Learns a policy: • Given a state, how should we act? • Unsupervised Learning (Approx.) • Dependent on a “Reward” function • Dependent on a appropriate state space • Dependent on a good set of actions • Produced the best backgammon program (TD-Gammon) CS 491J Computer Science of Multi-Player Games

  13. The basic idea • Take many different actions (initially at random, then guided) • Obtain a reward • Propogate reward backwards across actions (The “credit assignment problem”) CS 491J Computer Science of Multi-Player Games

  14. Learning a Policy • Estimate the utility from the reward for the current state plus the expected reward for future states. • Given experience (i to j), update the utility: • U’(i) = U(i) + (delta)*(R(i)+U(j)-U(i)) CS 491J Computer Science of Multi-Player Games

  15. Valuing States and Actions • Given a state i and an action a, learn the utility or value of the action Q(i,a) • Same basic idea as TD-Learning, but update each pair, rather than each state: • Q’(i,a) = Q(i,a) + (delta)*(max (Q(j,a’)-Q(i,a))) CS 491J Computer Science of Multi-Player Games

  16. Internal State • So far we have considered agents that plan and learn, but what about the agent’s feelings • Creating believable characters. CS 491J Computer Science of Multi-Player Games

  17. Finite State Automata • A simple model for a characters inner state • Each node represents a state • Each edge represents an event in the world • Behavior is keyed to state in the machine CS 491J Computer Science of Multi-Player Games

  18. Example Time Hit Run Stand Time Sighted Sighted Pain Attack Sighted Zero Life Die CS 491J Computer Science of Multi-Player Games

  19. FSMs • Often used • Advantages? • Disadvantages? CS 491J Computer Science of Multi-Player Games

  20. Emotional modelling • Continous approaches • Each “emotion” receives a number • Events update the value of the “emotion” Various update rules are possible • The set of numeric values is the player’s state CS 491J Computer Science of Multi-Player Games

  21. Example CS 491J Computer Science of Multi-Player Games

  22. Learning Possibilities • Learn “realistic” states • Human labeled realistic/unrealistic behaviors • Learn to produce realistic behaviors CS 491J Computer Science of Multi-Player Games

  23. Further Directions • Multi-Layer emotional models • Natural Language Processing & Generation • Cooperative emotions (empathy & more…) CS 491J Computer Science of Multi-Player Games

  24. Coming Soon • 3/23: Proposal Day • Pitch a game to the class • 3/28,3/30: First 10 Student Presentations • Topic to me 3 weeks before • Slides to me 1 week before • 4/11: Project Proposals • Binding contracts for what you will produce CS 491J Computer Science of Multi-Player Games

  25. AI in Games: Discussion CS 491J Computer Science of Multi-Player Games

More Related