Playtesting

1. Playtesting

2. Playtesting is not… • Focus groups – gathering opinions on proposed game/feature ideas • QA testing – the hunt for bugs • Usability testing – is the user interface easy to understand and intuitive; how difficult is it to learn how to play • Playtesting – does my game achieve its goal (very often, is it fun?)

3. Early and often • The feedback from playtests is useless when it comes too late in the project to make changes • Infrequent playtests generate less revealing and less important discoveries • You always need to playtest more often than you think

4. Outside observers • Whenever possible, hire outsiders to conduct your playtests 1> Relieves playtesters from having to be tactful (you want brutal honesty) 2> Keeps your from justifying and defending your work (they don’t care) • If not, at least invite playtesters to help you most by being critical

5. Outside observers • If not outsiders, then conduct tests using those farthest from the work or you’re asking for debates (you can even divide the game between project teams)

6. Why? We love our babies • It’s hard to watch someone cut up your baby, we defend it ruthlessly • There are even social/psychological terms for this and other related phenomenon, such as “sunk cost fallacy” and “loss aversion” • The more you invest, the more you irrationally protect the investment

7. Dollar Auction • I am auctioning off a $100 bill • It goes to the highest bidder, who simply pays their bid and receives the $100 • But the second highest bidder must also pay their highest bid, yet gets nothing ANY BIDS?

8. How to begin… • IDEAL: say nothing and just let them play • Concession: explain the controls • Concession: explain the goal • Anything else will completely subvert your attempt to see what the player can discern on their own • Remember that you can’t be there when they actually play your game

9. Don’t taint the results • It’s hard NOT to subliminally tip your hand and ask leading questions: • “The second level was pretty easy, right?” Makes them embarrassed to say it was way too hard, they’ll say “it wasn’t that easy” • “How did you like the large open world?” “Large” and “open” are positive adjectives, they think they are supposed to naturally agree • Never make a statement with a question

10. Keep ‘em separated • If conducting long playtests where there are breaks or you have an open space, playtesters will want to chat with each other about the game. DON’T LET THEM! • You don’t want to muddy someone’s perspective, or allow assertive testers to dominate other people’s opinions • Unless your game RELIES on groups of people playing together in the same space, don’t let playtesters mingle

11. Let them suffer • It’s irresistible to help a playtester figure it out, rather than sitting in agony as they become frustrated and hate your game • You want to see the things they will try • Ask them what they are thinking as they fail, what they THOUGHT should happen (then change your game to make them right next time, fail to success…)

12. What is unclear? • Find out what images, icons, text, missions, features, etc. are not clear • If they don’t use certain buttons, ask what they think would happen if they pressed them (based on the icons) • Be invisible as much as possible, but actively solicit feedback when they DON’T try things you need to be tested

13. They don’t care • Never explain or justify your decisions to the playtesters. It’s instinctual when they criticize but you don’t need to convince them your choices were wise, you just need their honest feedback • The more you defend the less they attack, and you WANT the onslaught (joining in actually helps them speak out)

14. Target demographics • Test the target thoroughly but also test outside to get things you hadn’t realized • Example: Any user interface that can be understood by 40-50 year olds will be understood by 20-30 year olds. But the older testers provide great value in showing what is possibly “on the fence” for the younger testers

15. Get statistics • Not just what doesn’t work, but how many people did it not work for • You can ignore problems that only strike a minimal percentage of testers and you MUST address problems that most of them had • Breadth is more valuable than depth (more sessions with more people rather than longer sessions with fewer)

16. Statistics in game • Where possible, log important events • It’s useful to know where your players are struggling, how long certain tasks take, etc.

17. Kleenex testers • You want lots of fresh testers “Kleenex testers” who have never played your game • Getting testers through the first five minutes of your game is the toughest part

18. Recurring testers • Kleenex testers show you where your game should improve for first-timers • Recurring testers are required to discover replayability and when your game tires • Crucial for longer game experiences, particularly for MMO’s

19. Prepare questions ahead • Do NOT improvise a post-play interview • Follow surprises and interestingly unique aspects of a playtester’s experience • But have your crucial questions prepared in order to make sure they are included

20. Most important question to ask: “What are the three worst things about this experience?”