Usability Testing

1. Usability Testing I. Omaima Al-Matrafi

2. Why Should You Care About Usability? • Have you ever… • gotten lost in a Web site? • left a site without finding the information you wanted? • waited too long for a page to download? • gone to a site you can’t view or read? • visited a site with outdated information? • Do you want people to visit and return to your site?

3. What is usability? • “A function of particular users performing particular tasks in a particular environment” (Smith et al. 68) • The “people who use the product can do so quickly and easily to accomplish their own tasks” (Dumas and Redish 4)

4. ISO Definition (9241-11) for Usability • “...the extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use.”

5. Definitions (cont.) Usability is a combination of factors that affect the user's experience with the product or system, including: • Ease of learning • How fast can a user who has never seen the user interface before learn it sufficiently well to accomplish basic tasks? • Efficiency of use • Once an experienced user has learned to use the system, how fast can he or she accomplish tasks?

6. Definitions (cont.) • Memorability • If a user has used the system before, can he or she remember enough to use it effectively the next time or does the user have to start over again learning everything? • Error frequency and severity • How often do users make errors while using the system, how serious are these errors, and how do users recover from these errors? • Subjective satisfaction • How much does the user like using the system? • Confusing, frustration and satisfaction

7. Usability • Is not functionality • Functionality is how the system works • Is not beta testing • Beta testing is unfocused and at end of process • Is not market research • Market research is about customer demands

8. Types of Usability Problems • Product doesn’t match job or task • Poor organization/layout • Unexpected occurrence of events • Product not self-evident • Requires recall rather than recognition • Inconsistent screens, messages, terminology • Design is inefficient • Cluttered or unattractive design • No feedback or poor feedback about status or errors • No exit or undo • Help or documentation is not helpful

9. Usability Problem Example: Inconsistent

10. Usability Problem Example: Poor Organization/Layout

11. Usability Problem Example: Product Not Self-evident

12. What is Usability Testing? • An empirical study of a product’s usability by observing actual users do real tasks with the product • The primary goal is to improve the usability of the product • 5 characteristic: • Real users • Real tasks • Specific usability goals/concerns • Observing and recording the testing • Data analysis

13. Planning: Define the Purpose • Look at goals of your organization. • Which goals is the Web suited to meet? • Prioritize the goals for your site. • Have one primary goal. • Optionally have secondary goals. • Example goals: • Provide a positive image of NASA / your organization • Provide information/function that the user needs

14. Five Steps for test process • Step 1: User Analysis & Profiles • Step 2: Decide what to Test • Step 3: Preparing for the Testing • Step 4: Conducting the test • Step 5: Analyzing the Data

15. Step 1: User Analysis & Profiles Knowing Your Users • What are your users’ characteristics? • Age, computer literacy, domain knowledge, access methods, browsers, work environments, handicaps, etc. • Collect this information through • Surveys / Questionnaires • Visits to their environment • A typical test includes 6-12 participants

16. Step 2: Decide what to Test • Choose an overall purpose • Example: How useable is our new website? • Determine objectives or what you are testing for. Examples: • Does our search engine provide usable results in the first 5 links returned? • Are search results clear to the users? • Choose type of test: • Performance: Can they do it? • Understandability: Can they understand it? • Read-and-locate: Can they find it?

17. Step 2: Decide what to Test con. • Select tasks: • Consider tasks with a high chance of user failure (complex tasks, one-of-a-kind tasks, highly abstract or technical tasks) • Consider tasks with a high cost of user failure (tasks that require support, like help or support calls, to complete; tasks where data could be damaged or lost) • Consider: • First impressions (look and feel) • First tasks • Tasks most performed • Critical tasks • Specific problem areas • New task for the product

18. Step 2: Decide what to Test con. • Select performance objectives (should be individualized for each task) • Time: How long to complete tasks, to find things, to performance procedures • Error/Success: user errors, attempts to do/find something, numbers of times section re-read, if the task was completed successfully • Five attributes of usability (Nielsen, 1994): • Learnability: system is easy to learn so users can get started quickly • Efficiency: system should be easy to use, resulting in high productivity • Memorability: system should be easy to remember • Errors: system should have low error rate and allow error recovery • Satisfaction: system should be pleasant to use

19. The goal and the concerns of the testExample: • General goal: users can use this program to read the documents easily and quickly. • Quantitative usability goals: users will be able to find the menu choice they want in less than 30 second with no more than two wrong choices and Know the function of each icon from the first time they see it. • General concern for this test: Can users find the correct menu choices and icons as they need them (ease of learn) • Specific concern of this test: 1-unrestanding how can open a PDF file to read it. 2- Accessing particular information easy and quickly by searching the document. 3- Increase the ability of enhancing the vision by zooming the page. 4- The ability of copying a specific part from this document. 5- The ability of making a snapshot for any part of document

20. Planning: User Tasks • Why will users come to your site? • What’s in it for them? • Best way to find out is to talk to them! • Create use scenarios to drive your design. • Example user tasks: • Find information about a particular product • Learn about newest projects • Find out about future presentations • Download copy of a document • Contact organization employees • Provide feedback

21. Planning: Example Scenario Scenarios try to capture a moment in time for the user. A sample scenario might read: John is working at XComptomake their systems more usable. He remembers that someone at MyCompis involved with usability engineering and wants to contact that person, either by phone or email, for more information on doing usability engineering at a government agency.

22. Step 3: Preparing for the Testing • Choose order of tasks: start easy, go sequential, or be random • Create written test materials: • Task list for users • Written welcome speech/ Intro to be read to user • Consent forms • Pre-task and post task questionnaires & interview questions • Other materials • Recruit participants & determine “payment” • Define team member’s roles: • Facilitator/Briefer (necessary): Often only team member to interact with users • Observation recorder/note taker (necessary) • Camera operator (optional) • Help desk operator (optional) • Test administer (optional)

23. Step 3: Preparing for the Testing Cont. • Create written test plan • Practice: conduct walkthroughs of the testing and if possible pilot test (the pilot test users could even be a team member) • Prepare test environment (day of test)

24. Pre-test questionnaire • Ensure that participant meets user profile • Gather additional detail about the participants’ knowledge and experience

25. Post-task questionnaire • Asks questions about a task as soon as it is completed • Captures feedback when it is fresh • Can help measure change in attitude toward the task • Keep it short

26. Description of the test • Tasks and their scenarios • The path for each task • Measuring usability • Typical Criteria for measuring usability • Consent Form • Methods: Details how you will run the test • Pre-Test ( user profile ), Post-Task questionnaires, Post test questioner, and Observation. • Task resources and Procedure: • time, HW, SW and procedure • Preparing the test • Test Team Members: define what each person will do

27. Step 4: Conducting the test • Greet & Brief participant: • Read/say welcome • Emphasize that you are not testing them, but the product and that they should act as natural as possible • Explain think-aloud protocol (if using) • Emphasize how user tells you she has completed a task • Stress that the testing is “anonymous” • Be unbiased (especially the Facilitator/Briefer) • Intervene carefully (avoid as much as possible) • Observe and record data • Debrief user

28. Step 5: Analyzing the Data • Collate data into findings: • Choose an approach: • Top-down approach: predetermine categories of findings (like navigation, design, terminology) and go through data looking for “hits” • Bottom-up approach: put each observation on a sticky note/note card, sort into categories and label categories • Determine time and errors/success • Examine findings for each user, user profile, and task • Use analysis techniques such as statistics (even averages help)

30. Step 5: Analyzing the Data con. • Analyze data: • Determine cause of problems • Determine scope/severity of problems • Make recommendations/changes • Report Findings

31. Usability Related Web Sites http://usability.gsfc.nasa.gov Goddard Usability Engineering Center http://www.usability.gov Usability resources by National Cancer Institute http://www.uie.com User Interface Engineering (Jared Spool) http://www.UPAssoc.org The Usability Professionals’ Association http://www.useit.com/alertbox The Alertbox: Current Issues in Web Usability (Jakob Nielsen) Book: A Practical Guide to Usability Testing (Dumas & Redish)