Practical Approaches to Usability Testing

1. Julie Nowicki President Optavia Corporation Practical Approaches toUsability Testing WEB2000

2. Goals • Put usability testing in perspective • In the bigger picture of usability • Compared to other techniques • Understand the breadth of usability testing methods available • Be able to focus and choose the right method • Depending on the web site’s development stage • Balancing resource costs against benefits of various methods Practical Approaches to Usability Testing WEB2000

3. Topics • The role of usability testing • Definition and comparison to other techniques • Designing the usability test • Planning • Choosing specific methods • Tuxedos or jeans? • Summary: Getting what you want • And what your customers want! Practical Approaches to Usability Testing WEB2000

4. Usability E-commerce and Usability • Three E-commerce segments • B2C (Business-to-Consumer) • B2B (Business-to-Business) • B2E (Business-to-Employee) • All focus on the customer • Ability of customer toeffectively complete a task, withhigh satisfaction levelsis key to the bottom line Practical Approaches to Usability Testing WEB2000

5. Qualities of a usable Web site • Learnability • Can customers easily grasp how to use the site the very first time? • Memorability • Do customers remember how the next time? • Ease-of-use • Can customers navigate easily? • Is the design consistent and predictable? • Efficiency • Can customers complete tasks and transactions effectively? • Appeal • More than visual: do customers feel positive about the experience? Practical Approaches to Usability Testing WEB2000

6. How can usability testing help? Practical Approaches to Usability Testing WEB2000

7. Characteristics of a usability test • Uses real customers (users) • Customers perform actual tasks with the site or a prototype • Data is recorded • May be qualitative or quantitative • Includes customer comments • Provides objective information • Facilitated by a skilled specialist • Observed by a skilled specialist Practical Approaches to Usability Testing WEB2000

8. Limitations of usability testing • Based on a few customers • Guinea pig effect • Test environment cannot duplicate the real world • Users know they are being watched • Not thorough by nature • Testing a few tasks, not every aspect of site usage • Some issues are not readily apparent • Use of color and fonts • Accessibility Practical Approaches to Usability Testing WEB2000

9. Usability testing compared… • Web site log statistics • Automated use testing • Focus groups • QA and system testing • User acceptance testing Proven usability techniques: • Heuristic evaluation and guidelines reviews • Walkthroughs Practical Approaches to Usability Testing WEB2000

10. Tip Usabilityencompasses customer-centered design and evaluation. However… Usability testing is an evaluation technique. Usability testing is not a design technique. Practical Approaches to Usability Testing WEB2000

11. Customer-centered design process Analysis Understand objectives Understand customers Understand their tasks Design Develop design concept Specify the details Iterative! Evaluation Test with real customers (usability testing) Review with design specialists Practical Approaches to Usability Testing WEB2000

12. Actual Timeline

13. Anatomy of a usability test Plan and Design Conduct Report and Recommend Plan: Answer the big questions Select the test methods Get participants Practical Approaches to Usability Testing WEB2000

14. Overview of usability testing methods To prepare for the test, you must • Plan Answer some big-picture questions Understand objectives, customers, and tasks • Choose the test methods Considering a number of parameters, Pick the best usability testing methods for your situation Practical Approaches to Usability Testing WEB2000

15. Planning:Big decisions to make first • What are you trying to accomplish? • When will you do the test? • Based on product lifecycle • What are the customer profiles? • What tasks will customers do? • How will you present the results? • How will you make recommendations? • Explore design alternatives Practical Approaches to Usability Testing WEB2000

16. What are you trying to accomplish? • Find out how your site compares to the competition • Find out if your new design works for • Existing customers • New customers • Get statistically sound data Practical Approaches to Usability Testing WEB2000

17. Tips for setting objectives • Prioritize the four usability components • Learnability • Ease-of-use • Efficiency • Appeal • Be specific • NO “Registration is easy to use.” • YES “New customers should be able to complete the registration process in 5 minutes without help.” Practical Approaches to Usability Testing WEB2000

18. Tip Usability testing done early in the design cyclereduces risk. Practical Approaches to Usability Testing WEB2000

19. The Cost to Make a Change Usability Usability U-test a must Practical Approaches to Usability Testing WEB2000 Adapted from Rapid Application Development by James Martin

20. Tip • Don’t get fooled by online development tools! • Sketches are highly advantageous • They are easy to throw away • People don’t assume the system is done • People are more likely to suggest changes • Usability testing can be done with sketches Practical Approaches to Usability Testing WEB2000

21. Tip Usability testing done late in the production processhelps you planfor the next iteration. Practical Approaches to Usability Testing WEB2000

22. Who are the customers? • Marketing information such as • Age • Income • Education • Career • Experience with the Internet • Experience with online shopping • Anticipated frequency of use (your site) • Environment they work in • Home, office… Practical Approaches to Usability Testing WEB2000

23. What are the tasks? • Creating task scenarios is a must! • Don’t use language that leads the user • NO “Go to My Profile and change your address to…” • YES “You have moved. Change your address to…” Practical Approaches to Usability Testing WEB2000

24. Presenting the results • Consider allowing observers during the test • Make sure they do not interfere • Highlight tape • Compelling and efficient • Results of data collection • Summary of issues, prioritized by • Usability criticality • Implementation effort • Recommendations Practical Approaches to Usability Testing WEB2000

25. The next step Once you have • considered what you want to accomplish, and when • identified your customer profiles • determined the tasks you want customers to perform • decided how to communicate the results you are ready to design the test methods Practical Approaches to Usability Testing WEB2000

26. Test methods:Parameters that you must select • Equipment and facilities • What to measure • Quantitative and qualitative • How to collect measurements • Use of participants (protocol) • How many participants • Level of facilitation • Level of participant verbalization Practical Approaches to Usability Testing WEB2000

27. Equipment and facilities Conference room Paper notes One camera Paper notes Portable lab Logging software Lab facility Logging software Practical Approaches to Usability Testing WEB2000

28. Example Usability Labs From www.usabilitysystems.com From www.microsoft.com/usability Practical Approaches to Usability Testing WEB2000

29. Advantages of video tape • Records very valuable, compelling qualitative data • Page navigation • Expressions • Comments in context • Provides a real snippet of an actual customer experience • Watch out • Someone needs to understand the equipment • Time to setup, review, create highlight tape Practical Approaches to Usability Testing WEB2000

30. What to measure: Quantitative • Time on task • Number of errors • Successful completion of task (yes/no) • How you collect measurements is determined by facility and equipment Practical Approaches to Usability Testing WEB2000

31. What to watch for: Qualitative • Navigation patterns • Getting lost? • Predictability • Understand graphics? • Understand words? • Looking for it to work like something else? • Flexibility • Looking for a quicker way? • Looking for help? Practical Approaches to Usability Testing WEB2000

32. Protocols (Use of participants) • Individual • Paired • Comparison • Good for competitive analysis • Between-subjects: each customer uses one site • Within-subjects: each customer uses both sites Practical Approaches to Usability Testing WEB2000

33. How many participants? • 5-8 participants will find 80% of usability problems • 4 is good minimum number 3 4 5 6 7 8 9 10 11 12 Qualitative data Early prototype Qualitative and quantitative Later in development Paired or comparison protocols Statistically sound results Practical Approaches to Usability Testing WEB2000

34. Level of facilitation Facilitator in another room Facilitator in same room Discussion after each task Facilitator and participant interact during task Practical Approaches to Usability Testing WEB2000

35. Level of participant verbalization • Think-aloud • Heads-down work Practical Approaches to Usability Testing WEB2000

36. Interaction High Low Facilitation Low High Participant verbalization Practical Approaches to Usability Testing WEB2000

37. Putting it all together Informal Formal

38. Now that we’ve said that… • Can you do informal testing on a production site? • Sure! • Can you do quick, but formal testing in a lab on a prototype? • Sure! Practical Approaches to Usability Testing WEB2000

39. What if… • You’re in the fast-paced e-commerce environment • You want customer data but you need it • Now (yesterday) • Whenever you make significant changes • You want guidance in making design decisions or changes Practical Approaches to Usability Testing WEB2000

40. Variation Informal Formal

41. Resource costs will be determined by your choices • Cost of facilities and equipment • Time for each participant • Your time resources • Planning test • Conducting test • Reporting and decision-making • Staff resources and skill levels • Generally all go up as you move from informal to formal Practical Approaches to Usability Testing WEB2000

42. Results From Supervisor of Integrated Business Practices “When [we] introduced an electronic expense payment system, users declared the application incredibly efficient – as early as the testing phase. That never would have happened if we hadn’t had the usability group to help us meet our business objectives. No company should ever move forward on a project in today’s business environment without some usability in their back pocket.” Practical Approaches to Usability Testing WEB2000

43. Usability is more than just usability testing… It is a process Julie Nowicki President Optavia Corporation JNowicki@optavia.com 608 260 9000 www.optavia.com