Types of Testing

1. Types of Testing Nov. 30

2. Five types of testing are relevant to our games • Internal Design Review • Quality Assurance Testing • Focus group testing • Usability Testing • Playtesting

3. Playtesting The goal of testing is to gain useful feedback from players to improve the overall experience of the game

4. Who to playtest with? • In the early stages, you playtest yourself • Move to confidants when you have a structure set up • Move to people you don’t know. • Move to target audience – representative sample (maintain diversity)

5. Some ground rules • DO NOT tell the player how the game is supposed to be played. Don’t give a speech on why you made your game, what it’s like, etc. • You have to let your game speak for itself • Give only minimal help, and only when a person is completely stuck • Above all…

6. Some ground rules • BE SILENT! • This is harder than it sounds • Many playtests are done using an objective person to run the test. The game designers will watch from behind a mirror or via video feed. This is to ensure they don’t influence the playtester. • You can’t hold the player’s hand. Their mistakes will provide valuable feedback, so you have to let them make mistakes. • No defending yourself to the playtester. • Take lots of notes (you will need them, but it also gives you something to do so you don’t have to interact that much with the playtester.)

7. Parts of a playtesting session • Introduction • Often scripted • Welcome and thank yous • Introduce yourself (not your game)

8. Parts of a playtesting session • Warm-up • Ask pre-written questions about the person • Ex.) “Tell me about some of the games you play.” • “How often do you purchase games?”

9. Parts of a playtesting session • Play session • Purpose is to get the player’s feedback on the experience • Before players begin, remind them that they are testing the game. Many playtesters feel like you are testing their skill and could get embarrassed. • Use the think out loud method • This is very important!

10. Parts of a playtesting session • The think out loud method • Ask the player to say what they are thinking. If they get stuck, ask them to explain what they are trying to do. • You may have to keep the person talking • Use questions like, “What are you doing?”

11. Parts of a playtesting session • Discussion of game experience • Develop a set of questions to ask the player after he or she has played your game. • Ex.) • Overall, what were your thoughts about the game? • What were your thoughts about the game play? • Were you able to learn how to play quickly? • What is the objective of the game? • How would you describe this game to someone who has never played it before? What would you tell them? • Now that you have had a chance to play the game, is there any information that would have been useful to you before starting? • Is there anything that you did not like about the game? If so, what? • Was anything confusing? Please take me through what you found to be confusing.

12. Parts of a playtesting session • Discussion of player experience • You may also include a feedback form

13. Parts of a playtesting session • Wrap-up • Thank person, give incentive if you have one

14. Playtesting • How many should you test? • 3-5 in each market • This process can be used with individuals or with a group • Individuals are best • Group dynamics can skew results • Spiral of Silence

15. Qualitative vs. quantitative • The method of playtesting we have just discussed gives qualitative data. • Playtesting also can yield quantitative data • Time it takes someone to read rules • Number of clicks it takes to perform a certain function • Feedback form could ask for rankings on scale of 1 to 10, for example

16. Playtesting • Come up with a hypothesis • What are you looking for? What are you trying to prove?

17. What are you testing for? • Some general areas you want to get feedback on: • Functionality • Internal completeness • Balance • Fun • Accessibility

18. Functionality • The system is established to the point where someone who knows nothing about it can sit down and play it. • (If you don’t have directions programmed into your game, type them up and hand them to the player).

19. Internal Completeness • There are no conflicting or missing rules • No loopholes – flaws in the system that users can exploit to gain an unfair advantage in a system, ruins the player experience • MarioKart glitch • No dead ends – player gets stranded in the game and can’t continue

20. Balance • Making sure the game meets the goals you have set for the player experience • System is of the scope and complexity you envisioned • Elements of the system work together without undesired results

21. Four areas of balance • Variables • The numbers associated with your game • Often in code • Speed of movement for example Ex.) Score deduction

22. Four areas of balance • Dynamics • Forces at work when your game is in action • All elements are present for game to work • Ex.) Super object that allows easy winning unbalances the game

23. Four areas of balance • Relationships • Relationship between players should be fair • If player earns a point, he or she might skip a turn so the other player has a chance (negative feedback) • If player earns a point, gets a free turn (positive feedback)

24. Four areas of balance • Skill • Match the level of challenge provided by the game system to the skill level of the user

25. Fun and Accessibility • Challenge, flow • Play • Story • Interesting choices that are meaningful, and have significance

26. Fun and Accessibility • Fun Killers: • Micromanagement • Stagnation • Insurmountable obstacles • Arbitrary events • Predictable paths

27. Fun and Accessibility • Accessibility – Don’t rely on directions. The player should be able to play without studying the directions carefully.

28. Usability Testing • Similar to playtesting • Testing that assesses how easy user interfaces are to use. • Jakob Nielsen suggests 15 users total broken up into three tests of 5 • Location is irrelevant (unless your company dominates market in that area)

29. Usability testing • Similar structure to playtesting

30. For your playtesting session • Develop the structure of your study, including feedback forms or questions you want to ask • Each group should recruit at one playtester • Bring enough copies of your game for each person playtesting to have their own copy

31. Final Project • Final Project Design and Usability • The final project should be functional. To test functionality, we will conduct playtests. You will provide a three-page report of your playtesting methods and results. In your report, discuss the feedback your playtesters provided and the subsequent changes you would make to your game.

32. Final Project • Final Project Narrative • The final project narrative should be a formal document, two to three pages in length, that is constructed as if you were trying to convince investors that they should support your efforts. Include specific goals, target audience and scope. Write in third person and assume that the reader has no previous knowledge of your idea. Include basic research (articles, books, etc.) that demonstrates there is a potential market/use for your game or interactive application.  • This is basically a rewrite of your original proposal.

33. Final Project • Topic Research • Write a two to three page paper that outlines the background material you've collected on the specific game topic. For example, if the game topic is gas prices, the paper should address all relevant issues (e.g. cost of gas, price of oil, location of oil wells, international reliance on oil, etc.) Include a bibliography.

34. Final Project • Storyboard • The storyboard should be an accurate presentation of the game you intend to build.   The frame-by-frame drawings and text need to be neat and organized, and contain as much detail and conceptual artwork as possible.  Present the storyboard as if you are trying to convince investors that they should support your efforts. Also turn in the rough draft of your storyboard from earlier in the semester.