236: II'nMI

1. 236: II'nMI Usability Testing

2. What is Usability Testing? Usability testing: What is it? • A way to assess the usability of a design with real-world users, before you go to the expense and trouble of actually building it. • What works (well) and what doesn't? • How do users actually use it? • Do they like/dislike it? • Usability goals: • Learnability, efficiency, memorability, error handling, satisfaction • Accessibility • Includes website testing, lab testing, mobile testing.

3. Why Bother? Why bother doing usability tests? • Because, if you don't, you run the risk of building a product that contains hideous usability flaws! • The cost of correcting these after the fact can be gargantuan: • Up to 200 times as much. • Arrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrgh ...

4. When to do Usability Testing When to do usability testing: • Formative testing/evaluation • Test/evaluate as the product is being developed • Assess success of ongoing effort • At least two test/evaluate/redesign cycles • Summative testing/evaluation • Test/evaluate after the product is finished • Good for comparing two similar products • Generally performed once

5. Usability Testing Cycle Usability testing cycle • Plan and prepare. • Run the test. • Collect the data. • Analyze the data. • Draw conclusions. • Document the results. • Redesign, based on the results. Iterate cycle as needed. • Generally >= 2 times. • Your first UT was for the paper prototype. This is your second. Let's take a closer look at the 7-step usability testing cycle.

6. 1. Plan and prepare. 1. Decide what do you want to learn from the test. • DD usability goals: efficiency, effectiveness, learnability, etc. 2. Decide on the best stakeholder/persona to accomplish the above. 3. Design test materials. • Introduction (script) • Testee scenario/tasks • Exit questions 4. Design and assemble the test environment. • Prototype, any additional props, etc. 5. Rehearse. 6. Run pilot test. 7. Revise test procedures and materials. 8. Recruit/schedule test users (3-5).

7. 1. Plan and prepare.

8. 2. Run the test. • Welcome the user. • Have user sign consent form. • We are not responsible for any psychoemotional damage ... ;-) • Get pretest information. • Name, age, occupation, other pertinent demographics • Read intro script to user. • Have it memorized! • Give user written list of tasks to perform. • Be there for user questions, but don't interact (more than necessary). • Zee Prime Directive!

9. 3. Collect the data. • Quantitative data • Metrics results • Time to complete tasks • Number of clicks, pages, menus, levels, errors • Ratings on a satisfaction questionnaire • Qualitative data • Encourage testees to "think aloud" • Record thoughts (audio, video, or written down) • Pathways, flow, problems, likes, dislikes • Observe critical incidents • Positive • Light bulb • Negative • Confusion/freeze/crash

10. Objective and subjective data

11. 4. Analyze the data: Quantitative. Quantitative data • Use a spreadsheet. • Enables you to • Collate data in a neat, orderly fashion • See patterns in the data • Use formulas to automatically calculate: • Percentage of testees who succeeded (or not) at each task • Average time to complete tasks • Average number of pages visited (buttons clicked, menu options selected) in each task • Frequency of specific problems • Etc. • Use of a spreadsheet for our in-class tests is not required. • But if you are feeling ambitious, go for it! • Create one on your own. • Or download: • Usability Test Data Logger Tool - free (donationware) spreadsheet analysis tool

12. 4. Analyze the data: Qualitative. Qualitative data • Spreadsheet! • Easiest to use the same quantitative data spreadsheet, to keep everything in one file. • Study the notes you took during testing for patterns. • Be specific in your remarks. • Good specific statement: Clicked on link to Data instead of Trials. • Poor vague statement: Clicked on wrong link.

13. 5. Draw conclusions. • Assess scope and severity of problems revealed during testing. • Problem scope • Local vs. global • Is the problem local (limited to the current page, screen, mode, etc.)? • Or is it global (application-wide)? • Problem severity • Minor • Users are somewhat annoyed/confused, but can still perform the task. • Major • Users are very annoyed/confused, and some might give up. • Show stopper • Users are so annoyed/confused, they all give up. • Plan and prioritize ways to fix (improve) these problems.

14. 6. Document the results. There are lots of different approaches for creating a usability test report (UTR). For example: • Usability.gov UTR template • Tower Records UTR You'll write a very simple, stripped-down UTR. • We'll take a look at the template in a moment ...

15. 7. Redesign, based on results. Prioritize redesign • Fix show stoppers. • Fix/improve major problems. • Improve minor problems.

16. Accessibility Accessibility • Usability should address all members of a user group • Approximately 20% of the population has a disability. • Your UI should be equally usable for users with/without disabilities. • Optional due to short time range (10 weeks) • It's the law! • For state and federal agencies in any case ... • Section 508 of the Rehabilitation Act of 1973 requires all state and federal agencies' electronic and information technology to be accessible to people with disabilities. • No doubt it will apply (quasi-)universally soon(ish).

17. Usability Test Template Let's take a look at the: • Revised SP 5 assignment • UT Prototype Usability Test assignment

18. Movie Time! • Usability Testing with Tobii T60 Eye Tracker (3.5) Usability Testing: Subject Two (7) • Usability Testing of Fruit(4.5) • Rocket Surgery Made Easy (25) • The Myth of Usability Testing (6.5)