Usability with Project 16/04/10

1. Usability with Project16/04/10 Susanne Frennert

2. What is usability? • The user having an acceptable experience when they use the product ! • What are the characteristics of an acceptable experience? • Successful (effectiveness) • Acceptable amount of time (efficiency) • Pleasure rather than stress (satisfaction) • ISO 9241-11 definition: usability as a goal • The extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use High usability means a system / product / website is: easy to learn and remember; efficient, visually pleasing and fun to use; and quick to recover from errors

3. Cost-justifying usability

4. The need to cost-justify • Not all companies (and managers) appreciate the benefits of usability • They may cite other factors as more important • Examples: • Tight deadlines • Functionality already developed • Limited money/resources

5. Cost-justifying usability • However, these are often “false economies” • Examples: • Deadlines: • Releasing the “wrong” product on time is as bad (or worse) as releasing the “right” product late • Existing functionality: • Existing functionality should have nothing to fear from usability, if it is the “right” functionality • Limited resources: • Putting the “right” resource in at the “right” time can make the overall project more efficient

6. Organisation Types

7. Good usability are good business • Reduced development and software maintenance costs • Increased product sales, market share, and revenue • Improve customer satisfaction and loyalty • Increased site visits and conversion • Decrease customer support costs • Improve employee productivity • Improved brand perception and media coverage

8. Reduced development and Software Maintenance Costs Integration of usability testing can contribute to decreased development costs by • basing feature decisions on solid user testing and evidence decreases the possibility of serious problems later in the process • many costs are associated with maintenance deal with unmet, or unanticipated, user requirements “An average of 48% of application code is devoted to the user interface  and 50% of the development time required for the entire application is  devoted to the user interface portion. Given this estimate, incorporating  usability into the design of the UI that customers and end users are interacting with is  critical.” (Weinshenck 2005) 

9. Increased Product Sale, Market Share, and Revenue On e-commerce or software as a service site, many potential product sale are lost due to poor site usability. A 2001 study found that 5% of online shoppers typically fail to find the product or service they are looking for, with as many as 65% of online shopping visits resulting in failure (Souza 2001; Mayhew & Tremaine 2005) Usability can be also used to increase advertising revenue; a revamp of Yahoo! Overture’s Search marketing tool found a 9% increase in click-through rate after only two weeks and “phenomenal” customer response (Cooper 2008)

10. Increased Site Visits and Conversions Usability can also increase the frequency with which users visit the site, contributing directly to the overall market share and customer base. eBay redesign their “Sell your Item” form, which had a high rate of drop-off for first time users, based on their user experience consulting and usability testing. A predicted 1% improvement in the drop off rate contributed to a revenue increase of $1.2 million a year (Braun 2002; Wilson & Rosenbaum 2005)

11. Improve Customer satisfaction and Loyalty A product that is easy to use and fun to use creates satisfaction and loyalty in its users One early study of e-commerce trust found that good navigation and presentation were essential in creating trust, and one of the authors found a link between quality in design and trust in the channel and the company’s brand (Karat & Lund 2005)

12. Decreases customer support costs The average call to a software development center is estimated to cost between $12 and $28 per call, and can cost up to $250   (Karat & Lund 2005; Mauro 2005) “A leading manufacturer of printers released a product that had a serious usability problem in the installment and operation of the printer driver software. This problem was so difficult to solve that more than 50% of the first 100,000 users called the customer service line. The cost was nearly $0.5 million per month. The total cost of replacing the printer driver was $900,000. It was clear upon analysis that the problem could have been identified and corrected at a fraction of the cost if the product had been subject to even the simplest usability testing” (Mauro 2005)

13. Improve Employee Productivity Supporting improved, enjoyable-to-use products can increase employee satisfaction and improve the quality of customer service, as well as reducing re-work and “downtime”. These factors contribute to overall improved job satisfaction, which can lead, long term, to improve employee retention (Robb & Pfefer 2003) “After Oracle changed the navigation structure on its database manager, database administrators were able to perform their duties 20% faster” (Black 2003)

14. Brand perception and media Coverage Products with great user experience are recognized for it. Users talk to each other about products they like, increasing sales due to world-of-mouth. Positive interactions can affect the brand's “halo”. “Apple’s iPhone, while accounting for only a small blip in the worldwide mobile phone sale, had the most media coverage of any product launched in history, and will likely have long-term effects on the mobile business” (Maurio 2008)

15. Rosson and Carroll “Usability Engineering” • “Cost-benefit analysis of usability activities contributes to more systematic usability engineering …” • “… BUT benefits are difficult to quantify, so estimates will often be overly conservative.” Issues: • Benefits (e.g. customer satisfaction) are harder to quantify and predict accurately • Costs are very concrete and easy to identify • Thus, can be difficult to justify usability…

16. Example eBay(Kelly Braun, usability engineer at eBay design labs)Sell Your Item Form (SYI)

17. Sell your Item Form at eBay(Kelly Braun, usability engineer at eBay design labs) • Look at page statistics and see where people leave process • Estimate usability impact • Use data to gain insight for re-design • They assumed the complexity of the page was contributing to the drop out rate. • How much revenue could be generated if they reduced the drop off rate by 1%?

18. SYI revenue increase calculations(Kelly Braun, usability engineer at eBay design labs) • Reduce drop out by 1%  Increase # of listings by 1% • 1.1 million listings per day  11,000 more listings per day. • Minimum listing fee = $0.30 • 11,000 listings x $0.30 = $3,300 per day • $1,204,500 in listing fees per year

19. SYI Usability Costs(Kelly Braun, usability engineer at eBay design labs) • One usability engineer for 3 months (team input and testing) • Testing assistant for 2 weeks • Participant incentives • Additional UI design (1 week) to address usability issues and redesign.

20. Cost-benefits analysis of usability A cost benefit analysis determines how well, or how poorly, a planned action will turn out. • A cost benefits analysis finds, quantifies, and adds all the positive factors. These are the benefits. • Then it identifies, quantities, and subtract all the negative factors, the costs. • The difference between the two indicates whether the planned action is advisable.

21. Example Cost-benefit Analysis for Usability testing In short, comparing these benefits and cost for the first year alone, an increased sales profit of GBP 34,770 is estimated.

22. Example Cost-benefit Analysis for Heuristic Evaluation In summary, comparing these benefits and costs for the first year alone, a benefit of GBP41,240 is expected.

23. Tailor your technique to the “problem” • What do the customer want to find out • Time to market • Who needs it and when • Hidden agendas

24. Scenario A company is planning to launch a service to replace an existing service. Marketing want to run a focus group to understand how users are using the current service to see how the new service can be improved. Usability is new in the company. Is a focus group the best choice to answer their questions? Alternatives?

25. What’s wrong with focus groups? They are good for: • Exploring what users or customers want • Gaining consensus on a concept But.. • Provide little depth from any participant • Capture opinion, not behavior • Strong group members can skew results • Provide weak qualitative data You can get better data for the same cost with in-depth interviews

26. Selecting the “right” tool?(Withney Quesenbury and Daniel Szuc, 2005) • To understand the environment… think observation, not usability testing • To test a concept… think rapid prototyping, not detailed specs • To find market advantages… think about an early competitive usability test

27. Methods for learning about the users

28. Methods for collecting business data

29. Methods for analysing information and tasks

30. Methods for evaluating designs in progress

31. Selling usability(Withney Quesenbury and Daniel Szuc, 2005) • Start with a small usability effort Example: If a company never done any form of usability testing, run an informal usability trial with an internal user. When the company management sees that even such a simple activity produces good insights, they may be more confident about doing employing user trials and using usability engineers.

32. Selling usability(Withney Quesenbury and Daniel Szuc, 2005) • Have team members participating. People are more open to ideas they have helped create. Participating lets them see that it’s not “magic” but real, valid techniques Example: Use participatory walk-troughs to get everyone thinking about the problem

33. Selling Usability(Withney Quesenbury and Daniel Szuc, 2005) • Don’t attack the product or design –the people who created it are in the audience

34. Intangible costs of Human Factors software lifecycle • 1 - The selection of non-critical design decisions for user studies • 2 - The establishment of too high a level of usability • 3 - Falling into the trap of overdesign • 4 - Communication problems between Human Factors specialists and software designers

35. Selection of non-critical design decisions for user studies • Not all design decisions affect the overall “quality” and “acceptability” of the final UI • How do you determine which ones are worth investigating? • Examples: Arial or Helvetica? 11 pt or 12 pt? Icon names? • Most usability experts rely on experience and intuition to decide… • … but they can be wrong! Costs: • Wasted user study time • Discovery of important changes may be put off until later Solution: • Make your user studies flexible enough to allow discovery of new problems

36. Establishment of too high a level of usability • Some UI performance targets simply cannot be met • Example: “This system must be learnable within 1 week” • This may appear to be a reasonable target … • … but what if the task itself takes >1 week to learn? Costs: • Design and development time chasing unrealistic performance targets Solution: • Set realistic targets • This can be informed by your task analysis studies • Also, aim for incremental improvements in performance over design cycles

37. Falling into the trap of overdesign • Prototyping software (UIMS) can make it very easy to add more features (“bells and whistles”) • Examples: borders, colours, icons, images, etc. • How to know when to stop??? • How to know what are valuable additions? Costs: • Designer’s time + implementation time Solution: • Strong management control

38. Communication problems between Human Factors specialists and software designers • There is a knowledge gap between HF specialists and the coders/developers… • Unless there is a shared common language and understanding, it is difficult to communicate effectively Costs: • Time lost in establishing a common language/understanding • Time lost developing the wrong thing (or the thing wrongly) • Time lost designing solutions that simply cannot be implemented Solutions: • HF specialists learns to develop own UI • Better corporate culture of communication

39. Intangible benefits from Human Factors software lifecycle • 1 - Adoption of features that save time • 2 - Avoiding system sabotage problems • 3 - Enhancing the ability to solve conceptual problems using the software system

40. Adoption of features that save time • Feature adoption by users is reduced when the system is so complex that users give up trying to learn the advanced features • i.e. they satisfice • Users will typically adopt the least complex system that offers them the required functionality • Even if it is less efficient • The Product Acceptance Analysis should identify unnecessary features • The User Tests should reduce feature complexity

41. Avoiding system sabotage problems • Being asked to use a system that is inappropriate, difficult or inadequate can lead to employee frustration • This frustration can lead to employees “taking it out on the system” • Example: entering incorrect or incomplete data • Example: reporting false system failures • This is referred to as “system sabotage” • Focus groups in Market Analysis and Product Acceptance stages are designed to address this issue • They test the receptivity of the target users to the new system • Also, the Task Analysis ensures that the final product matches the needs of the users as closely as possible

42. Enhancing the ability to solve conceptual problems using the software system • Increasing the user’s cognitive load to use the system … • … decreases the user’s cognitive capacity for “solving the problem” • So although the problem may be “solved” … • … there may have been a “better” solution • User Testing is designed to remove complexities from the system … • … and thus support users’ creative problem solving

43. Recommendations for Human Factors inclusion

44. Cost of running User Tests vs. size of user population User testing not cost-effective User testing cost-effective $ Benefits of testing Break-even point Cost of testing Size of user population Page 44

45. Summary of cost-benefit analysis • For any large user group, the benefits will usually outstrip the costs … • …often by a significant margin • For smaller user groups, the case is less clear-cut … • … so either decrease costs … • e.g. through “discount” usability methods • … or examine the intangible benefits more closely

46. Exercise

47. Exercise • Conclude your user trials • If you want feedback on your group presentation – then prepare slides for next Friday • NOTE – this is optional, but recommended

48. Tips for an effective presentation of usability results • Executive summary • Describe methodology (depending on your audience, you might need to explain or demo the product, describe the usability testing process, or provide background information. At a minimum , your audience will usually want to know something about the participants in the test and the tasks they were asked to perform) • Who, what, when, where, and how • Describe how tests were conducted • Profile users and describe sampling • Detail data collection methods • Succinctly explain the analysis • Provide screen captures (pictures really do work better than words in most cases) • Include tables and graphs • Provide examples • Identify strengths and weaknesses • Recommend improvements