Usability Methods

1. Institute for International Research User Experience Design Forum January 24-25, 2002 San Francisco, California Usability Methods to ensure success of your web site Mike Wardeiner Kelly Heidman

2. Case Study Redesign of the internal home pages for NASA Glenn Research Center A focus on usability throughout the process ensured a successful release User Experience Design Forum January 24-25, 2002

3. Two Design Efforts • Information Design • Interface Design Separate but related efforts following same user-centered design process User Experience Design Forum January 24-25, 2002

4. Test Test Test Test Analyze Analyze Analyze Analyze Improve Improve Improve Improve User-Centered Design Process Multidisciplinary Project Team Software Engineers Usability Engineer Technical Writers Trainers KM Specialist and Users Design Conceptual Design Detailed Design and Prototyping Requirements User Profiles User Needs and Requirements Development Implementation Marketing strategy Program and Test Prepare Documentation and Help Systems Finished Product Easy to Learn Easy to Use User Experience Design Forum January 24-25, 2002

5. Image makeover No one could “find anything” Content was scattered Only two navigational tools “Transporter” taxonomy First design Developed in early 1999 Taxonomy grew haphazardly Linked about 12 web pages Index and news Background – Why we needed a better design? User Experience Design Forum January 24-25, 2002

6. People/Places Which people/what places? Services Directory Services to whom? Publications Which publications? What this was like… User Experience Design Forum January 24-25, 2002

7. 1. User Task Analysis • Understand how the users will use the web site • What tasks are critical • What tasks are most frequent • Step by step how users complete tasks • User interviews, observation to gather input Usability means designing interface to allow user to perform their tasks efficiently and effectively See Task Analysis Form in appendix User Experience Design Forum January 24-25, 2002

8. Task Analysis Case Study • Primary user task is to find information • Studied usage statistics to identify trends • Critical tasks are linking a site to homepage and Transporter • Most frequent task is to find cafeteria menu • Different methods used to locate information (by subject, organization, search, etc.) User Experience Design Forum January 24-25, 2002

9. 2. User Profiling • Identify user groups • Job duties • User tasks to be performed • Level of job experience • Level of computer experience Identify traits that will make a difference in how a group of users will use the system See User Profile Form in appendix User Experience Design Forum January 24-25, 2002

10. User Profiles Case Study • Engineers, scientists, technicians, managers, secretaries will all look for different information related to their job • Different levels of computer/web experience from novice to expert • Web developers perform some different tasks from other web users User Experience Design Forum January 24-25, 2002

11. 3. Heuristic Review • Evaluate interface based on set list of standards/guidelines • Guidelines applicable to the following areas: • Appearance • Navigation • Functionality • Consistency • Language • User Feedback • Error Prevention and Correction See Gerry Gaffney Web Site Evaluation Checklist in appendix. User Experience Design Forum January 24-25, 2002

12. User Experience Design Forum January 24-25, 2002

16. Propeller image not intuitive link to home page Icons not intuitive buttons, especially NASA logo Non-standard text hyperlink colors caused confusion in identifying visited links. List of links does not provide any clear organization of content. No clear indication of internal vs. external links. Black background and colored text not easy to read. Overuse of color for no apparent reason. Transporter not visible without scrolling

17. 4. Usability Test Existing Site • Evaluate existing site to establish baseline user performance • Baseline performance can be used to set usability goals • Usability test to discover additional performance problems to improve User Experience Design Forum January 24-25, 2002

18. Usability Test Steps • Observe users performing typical, real-life tasks • Measure usability by gathering data on the following areas: • Time to learn task • Time to complete tasks • Quantity of errors while performing task • Retention of learning over time • User subjective satisfaction • Analyze results, factoring in user profiles User Experience Design Forum January 24-25, 2002

19. Usability Test Documents • Pretest questionnaire to gather user profile information on evaluators • Evaluation tasks that target questionable areas of the design • Data collection forms • Post test questionnaire to gather user opinions • Data analysis summaries • Evaluation and recommendations report See Pretest Questionnaire, Test Intro, Tracking form, Post test Questionnaire, User Ratings, Test Results, Evaluation and Recommendation reports in Appendix User Experience Design Forum January 24-25, 2002

20. Usability Test Findings Case Study • Established baseline success rate (50% success for locating items within 3 attempts) • Transporter not used much due to location (not visible) and user lack of understanding • Users didn’t know what LeWeb is • Users didn’t know how to link web pages to intranet home page • Users didn’t know to link web pages to Transporter • Users didn’t understand internal vs. external web pages • Users employ different search methods for locating items User Experience Design Forum January 24-25, 2002

21. 5. Establish Usability Goals • Usability goals provide target to indicate when design is done • Goals based on measurable performance • Time to complete tasks • Errors made while performing task • Improved performance over time • How much training / help was needed • Subjective satisfaction of users User Experience Design Forum January 24-25, 2002

22. Usability Goals Case Study • Users should be able to locate 50% of the items onfirst try • Users should be able to locate 75% of the items within 3 tries • Individual web sites no more than 3 clicks away • 75% of users should judge new design to be better than old design • Project team hesitant about setting goals because: • Change is not well accepted by GRC community • Users are accustomed to current interface User Experience Design Forum January 24-25, 2002

23. Information Design Background, Development, Outcomes

24. What is Information Design? • Organization/Navigation • Taxonomies, site “blueprints”, metadata, controlled vocabularies, thesauri, search systems, site maps, indices • Content Management/Indexing • Content classification, mapping, see and see also references • What to include and what not (also driven by 508) Electronic card catalog – finding the information See Data Organizing samples in appendix User Experience Design Forum January 24-25, 2002

25. Managers spend over 6 weeks each year searching for information Productivity loss – 35% of productive time spent searching Liability for wrong/dated information Why is this important? Solutions supported via user profiling User Experience Design Forum January 24-25, 2002

26. Information Design Best Practices • Provide sense of location through taxonomy, hierarchy, navigation, layout, branding • Addresses goals of user and designer • Breadth versus depth • Consistent via authority rules or hierarchical structure Clear to user and always involves USER TESTING User Experience Design Forum January 24-25, 2002

27. Taxonomy Focus • General principles of scientific classification • Show groups and relationships • Identify the terms and then decide the possible term definitions User Experience Design Forum January 24-25, 2002

28. First Homepage Topics • Authority rules/term focus example: • Connections Link: • Submissions should be limited to resources of value to the Lewis community User Experience Design Forum January 24-25, 2002

29. Developing the Taxonomy • Consider • Existing documents, manuals, online content • Description of business groups/processes • Planned applications/processes/future content See Information Design Checklist in appendix User Experience Design Forum January 24-25, 2002

30. Depth not > 3 layers Incorporate key links (help, index) Focus on business processes Apply “custom” authority rules Memory through meaningful relationships Associative learning Human indexers Utilized subcategories with main topics Developing the Taxonomy Case Study Center Finance Budget & Accounting Buy it, Ship it, Track it User Experience Design Forum January 24-25, 2002

31. Before the Design • Know: • Your site goals, organization mission • Your audience makeup • Your content inventory (type and classification) • Ex. Static – privacy statements • Review similar sites in your industry User Experience Design Forum January 24-25, 2002

32. 6. Card Sorting • Write down each topic on an index card • Give the cards to users and have them group into similar piles • Avoid pre-labeling groups • Collate the results User Experience Design Forum January 24-25, 2002

33. Card Sort Case Study • Captured existing sites on cards • Team sorted the cards • Tried to fit into JPL’s taxonomy ( built around their processes) *poor idea • Resulted in our first six categories User Experience Design Forum January 24-25, 2002

34. 7. Interface/Information Design Prototyping • Initial “throw-away” design • Low fidelity prototype allows for rapid change with minimal effort • True interface functionality not required • Hand-drawn on paper is lowest fidelity Rapid prototyping enables iterative design approach User Experience Design Forum January 24-25, 2002

35. Interface/Information Prototype Case Study • Built html text-only site following taxonomy from card sort • One hour total development time Prototype of information design provides taxonomy for interface User Experience Design Forum January 24-25, 2002

36. Glenn Research Center About You and Glenn Core Research Finance Information Depot Leader's View Support Services User Experience Design Forum January 24-25, 2002

37. Glenn Research Center • Home • About You and Glenn • Health and Family • Leisure • Money • Safety • Work User Experience Design Forum January 24-25, 2002

38. Glenn Research Center • Home • About You and Glenn - Health / Family • EAP Employee Assistance • Fitness Center • LLF • Medical User Experience Design Forum January 24-25, 2002

39. 8. Usability Test Taxonomy • Purpose is to verify results of card sort with additional users • “Real” users attempt to locate 30 topics • Use low-fidelity prototype • Gather performance measures for validation • Analyze performance results • Modify information design based on results Evaluating the information design. User Experience Design Forum January 24-25, 2002

40. Usability Test Results Case Study • Successful find rate only at 73% (within 3 attempts – just under goal of 75%) • First time success rate at 50% (met goal) • 3 main categories are too unclear • Need to expand from 6 to 8 categories • Needed to relocate topics into other categories • Renamed some main headings • Gathered additional information on user attitudes toward current intranet User feedback made decisions clear User Experience Design Forum January 24-25, 2002

41. Taxonomy Views User Experience Design Forum January 24-25, 2002

42. Interface Design User Experience Design Forum January 24-25, 2002

43. Interface Design Issues • Provide new image; cleaner look • Search vs. Transporter • Simplify use of Transporter options if used • Use of “hot links” buttons (need rules to define which ones get chosen as a “hot link”) • Provide multiple search mechanisms • More intuitive process needed to link web pages to homepage • Address other identified problems with current interface User Experience Design Forum January 24-25, 2002

44. User Experience Design Forum January 24-25, 2002

45. 9. Usability Test Interface • Tasks targeted specific areas: • Search vs. Transporter • Link to NASA main page • Link to GRC external page • Use of “hot link” buttons • How to link a web page • Find an organization’s homepage • Repeated links to topics within the same main category to test user recall User Experience Design Forum January 24-25, 2002

46. Usability Test Results Case Study • Successful find rate of over 90% within 3 attempts (surpassed goal of 75%) • First time success rate of 77% (surpassed goal of 50%) • Minor name changes will increase success rate • 7 of 9 users preferred new design vs. existing User performance and user satisfaction surpassed usability goals. User Experience Design Forum January 24-25, 2002

47. Search feature was not a Transporter Hot link buttons not noticed Taxonomy was intuitive Not recognized as links Easy to link a new page User Experience Design Forum January 24-25, 2002

48. Recommended Changes Case Study • Keep Transporter name and functionality • Make Transporter a main feature • Eliminate NASA logo and GRC hanger as image links • Remove black background from behind “hot links” buttons to separate them from header • Modify some link names User Experience Design Forum January 24-25, 2002

49. User Experience Design Forum January 24-25, 2002

50. User Experience Design Forum January 24-25, 2002