Testing through user observations

1. Testing through user observations User Observation: Guidelines for Apple Developers, Kathleen Gomoll & Anne Nicol, January 1990 Notes based on: http://pages.cpsc.ucalgary.ca/~sheelagh/courses/481/assignments/2Usability/apple_guidelines_usability.html

2. User testing • User testing methods include: • Focus groups and interviews • Pros and cons? • Surveys with paper or web-based forms • Pros and cons? • Timed performance tests or keystroke protocols • Controlled laboratory experiments • Pros and cons? • User observation…

3. User observation • Watching and listening carefully to users as they work with a product • Why do it? • Gathers objective data about users • Relatively quick compared to other methods

4. Preparing for an observation • Set objectives: figure out what you're testing and what you're not. • Limiting scope of test makes it more likely you’ll get information that helps you solve a specific problem • What might be the objectives for your project? • Design the tasks: specific tasks that you want real users to accomplish • Write them out as short, simple instructions • What tasks would your user accomplish?

5. Preparing an observation (2) • Plan recording: video or audiotape? Why? • Determine setting: ideally, a quiet, enclosed room with a desk, appropriate hardware and software, video camera, and two microphones • Find representative users: people who have the same experience level as the typical user for your product • Pairs of users is sometimes good: they usually talk more than people working alone • Who might you get as your test users?

6. Conducting an observation (1) • Introduce yourself • Describe purpose of observation: • Set participant at ease • You're helping us by trying out this product in its early stages • We're looking for places where the product may be difficult to use. • If you have trouble with some of the tasks, it's the product's fault, not yours. Don't feel bad; that's exactly what we're looking for. • If we can locate the trouble spots, then we can go back and improve the product. • Remember, we're testing the product, not you

7. Conducting an observation (2) • Tell the user it’s OK to quit at any time? Why? • Explain any physical equipment. Such as? • Explain how to “think aloud”Why? • Help discover expectations for your product, intentions and problem solving strategies • We get a great deal of information from informal tests if we ask people to think aloud as they work through the exercises • It may be a bit awkward at first, but it's really very easy once you get used to it • If you forget to think aloud, I'll remind you • Would you like me to demonstrate?

8. Conducting an observation (3) • Explain that you cannot provide help. Why not? • Because we want to create a most realistic situation • Go ahead and ask questions for recording on tape • I’ll answer any questions you may have afterward • Describe the task and introduce the product • Give the participant written instructions for the tasks • Don’t demonstrate what you’re trying to test! • What tasks might you want to observe? • Ask if there are any questions before you start

9. Conducting an observation (4) • Conclude the observation • Explain what you were trying to find out during the test. Set the participant at ease • Answer any remaining questions • Discuss any interesting behaviors you would like the participant to explain • Use the results! • Don’t blame users for mistakes. Why not? • Document your results and revision plans • When should you do your observation?