Player Modeling

1. Player Modeling James Morrison

2. Previously and Currently • Currently, many games have only player-selected difficulty- “manually adaptive AI” • This is implemented in many games, as far back as Doom, and also very recent ones • (game play demonstration)

3. Currently • Many games don’t even implement multiple difficulty levels • This leads to games being criticized for being too easy or too hard.

4. Manually Adaptive AI • This is may be the most used method of selecting difficulty, but it is far from best. • Consider the average gamer: What difficulty should be picked? How do you know? • What if the gamer specializes in a given genre of game? • What if it’s the same as the game in question? • Different? • As challenge has been linked to player satisfaction, what should be done?

5. The Fix • The fix for this problem is automatically adjusting to challenge the player. • This presents a few problems... • How to change the difficulty • Is just a single measure enough? • How to know how to adjust the difficulty • Detection of player actions/consequences • This is all handled with Player Modeling.

6. What is Player Modeling? • A player model is an abstraction of the player. • A few uses of player modeling: • Model the skill of the player so the game can adjust difficulty • Model weaknesses so they can be exploited • Model actions so the player can be predicted

7. Creating a Player Model • When designing a player model, one has to decide what traits should be modeled based on the game • For example, UsesSmokeGrenades works just fine for Counter-Strike, but not at all in Oblivion • What actions the player can take are a good starting point • i.e. UsesDodge, UsesPowerAttack

8. Model Details • Does a high rating in an attribute mean that the player is highly proficient or uses it often? • How do you integrate the model into the AI? • Updating and maintaining a model of any significant detail can be quite a task...

9. Usage vs. Proficiency • Example: UsesSmokeGrenades • A use model would have a high rating for the player if the player uses a high number of smoke grenades • A proficiency model would have a high rating for the player if the player uses smoke grenades intelligently • Use a grenade when retreating, or when outnumbered, for example

10. Initializing the Model • There are a few ways of initializing models for players in a game setting… • Initialize with ‘neutral’ values (.5 on a 0-1 scale, for example) • Pre-learn using game testers to figure out a good baseline for the ‘average’ player • Set the levels arbitrarily high or low

11. Initializing with Neutral Values • Initializing with a neutral value would mean that your AI starts on what is effectively it’s middle difficulty. • This leads to some players being bored (because it’s too easy) or too challenged in the case that it’s too hard. • This also means that all players start as generalists and move from there, instead of the game recognizing their preferences from the start.

12. Pre-Learning • This would set the values to what the majority of players find about right when they start the game. • This would require a significant amount of testers, and this also prevents the game from recognizing player’s preferences from the start if they significantly deviate from the average

13. Arbitrary Values • In this initialization, the values are set by the developer, using their judgment as to how to start the player. • This could lead, as in neutral values, to some players being bored or over-challenged. • This type of setup also allows players to pre-input their play preferences (a la character creation in Oblivion or many games based on the D&D system)

14. Updating the Model: • Two steps: • Deciding when the model needs updated • this involves finding out when a player has succeeded or failed at a particular task • This can be computationally very expensive, especially in the use of a passive skill • Calculating the New value for the model • updateTrait(traitName, observedValue) • Often a machine-learning algorithm called the least mean squares method is used… • traitValue=α*observedValue + (1- α)*traitValue • Where α is what’s known as the learning rate

15. An Example Player Model

16. Using the Model • Now that we have a model, and it’s initialized, how do we actually use it in our theoretical game? • This brings us back to it’s uses: • Model the skill of the player so the game can adjust difficulty • Model weaknesses so they can be exploited • Model actions so the player can be predicted • I’ll go in depth with all three of these…

17. Adjustment of Difficulty • This is the most obvious of the three, and possibly most useful • As the player progresses, tailor the encounters to the player • Usually, a challenging but beatable encounter is ideal.

18. Adjustment of Difficulty • Usually adjusting the difficulty to suit the player has the effect of increasing player satisfaction. • When done incorrectly, however, or when the wrong metric is used, it can feel unfair or too easy

19. Modeling in Oblivion • The game Oblivion two sorts of player modeling behaviors. • At the beginning of the game, the player’s actions are tracked, and a suggested class is offered. • This system works relatively well, and for many players, will give a good class based on how they performed in the beginning of the game. • Demo!

20. Later in Oblivion • After leaving the tutorial zone in Oblivion, the game begins keeping track of the character’s level. • This is used as a metric for adjusting the difficulty of encounters within the game. • What are some possible problems with this system? • Yet another gameplay demonstration

21. Modeling Weaknesses • This is specific to the proficiency type of modeling… • Using a model to detect player weaknesses can be useful both for helping the player and harming him. • At the beginning of a game, if the player is significantly deficient in some area of gameplay, more tutorial time could be spent on the deficiency. • Later, a deficiency can be used to increase the difficulty of a given encounter.

22. Modeling Actions • Specific to the Usage amount type of modeling… • If a player has a high rating in some aspect, it is likely that is the aspect of the game to be used. • As an example: If a player is known to use a lot of smoke grenades, keep the smoke grenade animations and sounds cached for faster loading times.

23. Variations and Other Uses • Player modeling however, has many more uses then just these few uses and one design. • Besides the obvious changes in how the model learns and changes (with a change in the updateTrait() function) there are many other possible variations.

24. Hierarchical Player Modeling • In a hierarchical model, only the leaf nodes are actual concrete descriptors. • All parent nodes have only derived values based on their children (usually the average)

25. An Example Hierarchical Model

26. Why Have a Hierarchy? • The hierarchical model allows more resolution in the model, while still allowing overarching statements to be made about the player. • i.e. the player is a stealth-centric character, but prefers the sap to the bow. • This increased resolution allows the designers to adapt even further to the player • Perhaps dropping better saps, and fewer arrows in the above example…

27. Player Modeling for Interactive Storytelling • Finding player preferences can also be used to direct the flow of the game. • In Knights of the Old Republic, your character moved towards the dark side or light side depending on his actions, and this changed the story and dialogue to some extent. • Theoretically, a system has been designed that will give only player-interested side quests to the player.

28. Interactive Storytelling • In the second case, imagine a choose- your-own-story game, and where the player never intentionally makes the choices. • With different players acting differently, or the same player playing different characters, the play experience is changed every time • This was studied using the Neverwinter Aurora toolset, and was shown to have higher player-satisfaction values

29. An example Story • The story follows very roughly the story of red riding hood (if the wolf was a troll, and the grandmother was a wizard)

30. The Results of the Study • The Study showed that all user groups found more fun and more influential to the game world • This was especially true of the three specific groups shown. • The Study showed the certainty factors for 3 groups and the entirety of the study • Groups were • Females who found the game easy to follow • Females • Those who found the game easy to follow • Everyone else

31. Model Sharing • Another possible use for player models would be sharing the models between games. • This is mostly useful between games of the same genre, or more often, games of the same series (halo 1, 2, 3) as games have different actions available to the player

32. Model Sharing • Model sharing would allow players to truly take up where the left off playing another game. • Players starting ‘PlayerModeled game 2’ would be already modeled and the AI would then have no (or a very small) learning curve. • This reduces the time where the player is uninterested due to the game being too easy or too hard.

33. In Conclusion • Player Modeling is not so much an AI technique, but one that allows customization of the AI for maximum player enjoyment • Player Modeling can also be used to change the story or other aspects of the game to further increase the enjoyment of the player.

34. In Conclusion • Player modeling has yet to be fully implemented into a game, and is still being researched. • Some games have tried implementing simpler types of it, but they tend to fall short in some way.

35. Thanks for listening!