1 / 54

Software Usability Course notes for CSI 5122 - University of Ottawa

Software Usability Course notes for CSI 5122 - University of Ottawa. Section 4: Usability in the Software Engineering Process Timothy C. Lethbridge < tcl@site.uottawa.ca > http:// www.site.uottawa.ca /~ tcl /csi5122. An Assertion.

muriel
Download Presentation

Software Usability Course notes for CSI 5122 - University of Ottawa

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Software UsabilityCourse notes for CSI 5122 - University of Ottawa Section 4: Usability in the Software Engineering Process Timothy C. Lethbridge <tcl@site.uottawa.ca> http://www.site.uottawa.ca/~tcl/csi5122

  2. An Assertion • Lack of usability is the most critical problem facing software engineering • In other words, the biggest gains in • software quality • productivity • cost-reduction • user satisfaction • profitability, etc. … would come from focusing on usability and related issues. Section 4: Usability and the SE Process

  3. Evidence for the assertion • Failures of projects due in significant part to usability • E.g. FAA air traffic control systems • Observations of ‘great’ software vs. not-so-great with respect to usability • Great: • Google (many of its products) • Microsoft Office (some of it) • Mac-OS (much of it) • Not-so-great: • Much in-house custom software • Software development environments (many aspects) • Web sites (far too many of them) • I have observed many users formally and informally • My survey of practitioners (next slide) • Analysis of benefits and cost savings Section 4: Usability and the SE Process

  4. Practitioner survey about 75 topics taught to computer science students • Lethbridge, T.C. (2000), “What Knowledge is Important to a Software Engineer?”, IEEE Computer, May, pp. 44-50. Section 4: Usability and the SE Process

  5. More on the survey • HCI was second in terms of “knowledge gap” • Where importance most exceeds current knowledge • The top 5 (out of 75) • Negotiation • HCI/user interfaces • Leadership • Real-time system design • Management • I believe not much has changed since 1998 Section 4: Usability and the SE Process

  6. Sources of resistance to adopting usability practices • Primarily: Inadequate education of students, practitioners and managers • Which leads to: • Lack of introspection ability of software developers • Unable to think like users • Lack of integration of usability into core development processes • Persistent beliefs • Most software is usable enough • Usability can be fixed at the “end” • Usability should be left to the HCI experts Section 4: Usability and the SE Process

  7. Causes of the problems in education • Faculty don’t have the background either • The field of HCI is seen as distinct and separate • The field is perceived as too soft: • There is a tendency to focus teaching and research on hard deterministic areas • There is little industry push for research in this area • There is little employer pull for greater education Section 4: Usability and the SE Process

  8. Some partial solutions • The SE and HCI communities need to work more closely together • Core UI development and evaluation topics should be considered jointly part of the two fields • Improve education by ensuring HCI permeates curriculum models, certifications, accreditation, etc. • Enable corporations to be certified for their usability capabilities • Would help “pull” education up by creating a need for professionals • Clients might come to learn that better software comes from such companies Section 4: Usability and the SE Process

  9. Should aspects of HCI be considered also integral to Software Engineering? • I argue yes: • Usability should have no special status as compared to reliability, efficiency, maintainability, etc. • Design involving users and their needs must drive software development Section 4: Usability and the SE Process

  10. Should aspects of HCI be considered also integral to Software Engineering? • Yes, but this does not preclude HCI specialists • Analogies: • A software engineer must be capable of designing architectural elements for storing data and information • But there will always need to be database specialists • Similar argument can be made for • Security experts, performance experts, requirements experts, etc. • Core UCD techniques (coming slides) can be part of the SE field and part of a broader HCI field Section 4: Usability and the SE Process

  11. User-Centred DesignRevisited Section 4: Usability and the SE Process

  12. User centered design (UCD) • A term encompassing a range of software engineering approaches that focus on the user • We have talked about many of the techniques Section 4: Usability and the SE Process

  13. UCD main stages • Pre-design • Design stage • Post-design Section 4: Usability and the SE Process

  14. UCD Pre-design techniques: • Understand classes of user • Understand tasks • Perform a competitive analysis • i.e. usability testing of competing products • Set usability goals • using the metrics we discussed earlier Section 4: Usability and the SE Process

  15. UCD Design-stage techniques • Parallel design • Many developers generate UI ideas in parallel on their own • They meet together and pick the best ideas • Has maximum chance of discovering good ideas, without constraining the possibilities too soon Section 4: Usability and the SE Process

  16. UCD Design-stage techniques - 2 • Involving users • Critiquing design • Discovering problems • with the UI • with task descriptions • through user testing • Contributing ideas • But users are not designers so don't ask them what do you want? Section 4: Usability and the SE Process

  17. UCD Design-stage techniques - 3 • Coordinating the interfaces of all parts of the system • Including documentation, training etc. • Applying guidelines and heuristic rules • Prototyping Section 4: Usability and the SE Process

  18. UCD Design-stage techniques - 4 • Performing empirical testing using simulations and prototypes • To discover problems • Measuring usability of the simulations and prototypes • Performing iterative design • Design • Evaluate • Redesign Section 4: Usability and the SE Process

  19. UCD Post-Design stage • Gather on-going feedback about potential improvements Section 4: Usability and the SE Process

  20. Another term related to UCD • “Outside-in development” Section 4: Usability and the SE Process

  21. Economics of Usability Section 4: Usability and the SE Process

  22. An example of economic analysis in UCD: Economics of heuristic evaluation • We will look at • Number of people needed • Cost-benefit analysis Section 4: Usability and the SE Process

  23. How many evaluators should you use in a heuristic evaluation? • Given: • N = total number of problems to be found • Clearly depends on: • Quality of the original design or work performed so far • Size and complexity of the system • λ= proportion of all usability problems found by a single evaluator: • Studies have shown this averages 34% (range 19% to 51%) • The percent of the total number of different usability problems found by aggregating reports from i evaluators is: • PercentProblemsFound(i) = 1-(1-λ)i Section 4: Usability and the SE Process

  24. We can plot the effect of this formula for different λ Section 4: Usability and the SE Process

  25. We can also look at specific figures for the average case where λ=34% • Evaluators Percent Problems found • 1 34 • 2 56 • 3 71 • 4 81 • 5 87 • 6 92 • 7 95 • Given a desired percentage of problems to be found, we can determine the number of evaluators needed • by inverting the formula: • i = ln(1-P)/ln(1-l) • It is not economical to try to find every problem but where should we stop? Section 4: Usability and the SE Process

  26. How much cost savings can we achieve by finding a given problem? • This depends on many factors, but • The worst problems are the easiest to find and will be found first • As we put more effort into finding problems, we will find problems that are less and less severe • We can model the cost savings from fixing a problem as seconds of wasted time per user session • Very approximate, but useful • Some problems will only occur once, then never again because the user will learn to avoid them Section 4: Usability and the SE Process

  27. Visualizing the diminishing returns • Estimated number of seconds per session saved by fixing the nth worst problem found in a particular project. Section 4: Usability and the SE Process

  28. Deciding what to fix and the resulting savings • From the previous chart, find the total savings if we just solve the first n problems • We can divide this by the n to find the average savings per problem had we just found the first n • A typical answer for this would be 2 seconds Section 4: Usability and the SE Process

  29. Deciding what to fix and the resulting savings • Some data from the previous chart, where there were about 200 problems eventually found • nth problem s/session saved average savings had we just • found found the first n problems • 1 10.6s 10.6s • 10 7.4s 8.9s • 20 4.9s 7.4s • 40 2.2s 5.4s • 60 1.0s 4.1s • 80 0.45s 3.2s • 100 0.20s 2.6s • In a large organization, we can gather data to calibrate our own charts and tables like the above. Section 4: Usability and the SE Process

  30. Deciding how many evaluators to include:Costs • Fixed costs to organize the evaluation: (no matter how many evaluators): • Planning • Preparing materials • Writing report and communicating results to developers • Typical estimate for a small system • = 75 hours work • = $4500 Section 4: Usability and the SE Process

  31. Costs continued • Variable costs for each evaluator • Weighted salary for time spent • Cost of analyzing the evaluator's report • Typical estimate for an average system • 8 hours work for each evaluator ($480) • 2 hours work analyzing the report ($120) • Total = $600 • Total costs = $4500 + 600i • Where i is the number of evaluators Section 4: Usability and the SE Process

  32. Deciding how many evaluators to include:Benefits • Total benefits vary substantially depending on nature of system. • In some published studies, the benefit averaged $15000 per problem • On the following slides are two benefit calculation examples Section 4: Usability and the SE Process

  33. Benefits example 1:small mass-market system • Three factors to consider: • 1. Time saved per session per problem • Will obviously vary from problem to problem and session to session • Reasonable average 2s saved per session per problem (similar to data shown earlier) • 2. Typical number of sessions of use per system • Assume an average of 50 sessions • 3. Number of users (licenses) • Assume an average of 3000 users Section 4: Usability and the SE Process

  34. Benefits example 1 - continued • Therefore: total time = 2s * 50 sessions * 3000 users = 300000 seconds per problem! = 83.3 hours of saved time • If weighted at $60 per hour: • Potential benefit is $5000 per problem • Assuming only half the problems will get fixed: • Benefit = $2500 per problem found • Or added value to a license per problem fixed = 2s * 50 sessions = 100s = $1.66 Section 4: Usability and the SE Process

  35. Benefits example 2:small in-house system • The same three factors to consider: • 1. Same average time savings per session (2s) • 2. More average sessions over lifetime (200) • 3. Fewer users (40) • Therefore: total time = 2s * 200 sessions * 40 users = 16000 s / problem = 4.4 hours of saved time • Using the same weighted value of $60 per hour: • Potential benefit is $266 per problem • or $133 per problem actually fixed. Section 4: Usability and the SE Process

  36. Cost benefit analysis • So will a usability study be economical? • Let’s put the costs and benefits together Section 4: Usability and the SE Process

  37. Example 1: Mass-market system • Insp Total benefit % probs total ben/cost • cost /prob found benefits ratio • 2 5700 2500 56 280000 49 • 3 6300 2500 71 355000 56 • 4 6900 2500 81 405000 59 • 5 7500 2500 87 435000 58 • 6 8100 2500 92 460000 57 • 7 8700 2500 95 475000 55 Section 4: Usability and the SE Process

  38. Example 2: In-house system • Insp Total benefit % probs total ben/cost • cost /prob found benefits ratio • 1 5098 132 34 8976 1.8 • 2 5700 133 56 14896 2.6 • 3 6300 133 71 18886 3.0 • 4 6900 133 81 21546 3.1 • 5 7500 133 87 23142 3.1 • 6 8100 133 92 24472 3.0 • 7 8700 133 95 25270 2.9 Section 4: Usability and the SE Process

  39. Odds and Ends Section 4: Usability and the SE Process

  40. Prototyping • A prototyping tool should: • Be easy to learn and use • Possess good visual capabilities • Allow easy extension • Give full support for the types of interface being developed • Support modular programming Section 4: Usability and the SE Process

  41. Types of prototyping • Horizontal prototyping • Complete UI, but no underlying functionality • Vertical prototyping • In-depth functionality of only certain parts • Scenario prototyping • Selected paths only Section 4: Usability and the SE Process

  42. The design rationale document • Allows you to keep control of the various alternatives considered and decisions made • Explain your usability decisions so others know the reasoning you followed Section 4: Usability and the SE Process

  43. Discount usability engineering • User and task observation • Cheap prototyping and scenarios • Simplified thinking aloud • Heuristic evaluation • Key goal, communicate qualitative information to developers in order to improve interfaces • as opposed to qualitative measures • A study was performed of iterative design involving discount usability engineering • Improvements of 38% after each stage Section 4: Usability and the SE Process

  44. Consumability - Closely allied to usability • Ability for customers to get rapid return on investment. • Factors include • all aspects of usability • installability / uninstallability • lack of inter-product dependency/interaction • low footprint: lack of run-time resource consumption • backward-compatibility • integratability • appropriate prioritization of iterations and features to yield business value • low purchase cost Section 4: Usability and the SE Process

  45. Rating corporate maturity with respect to usability Section 4: Usability and the SE Process

  46. The Model for Usability Maturity (MUM)(proposed by Lethbridge!) • A 5-level CMM-like categorization of organizations • Based on their interaction with human end-users in software development • (Each level beyond 1 builds on the previous levels) • Level 1: Haphazard • Level 2: Defined input from users / usability awareness • Level 3: Iterative interaction with users / design for usability • Level 4: Controlled and measured involvement of users • Level 5: Continually improving usability Section 4: Usability and the SE Process

  47. Level 1: Haphazard • If you can get users to use the system, then it is considered good enough! Section 4: Usability and the SE Process

  48. Level 2: Defined input from users / usability awareness • Users involved in requirements reviews • Feedback from users at reviews is incorporated into the next stage • Design team members have basic training in usability • Design team adheres to usability standards for look and feel • Reuse of well-understood controls, styles etc. Section 4: Usability and the SE Process

  49. Level 3: Iterative interaction with users/ design for usability • Users actively involved in decision making • Use case / task analysis • Competitive analysis • Design with careful attention to usability guidelines • Usability design decisions are carefully analyzed and the decisions are recorded • E.g. options analysis, tables of pros and cons Section 4: Usability and the SE Process

  50. Level 3: Continued • User input and feedback with repeated prototypes • From paper prototypes to functional prototypes • Informal qualitative observations of users or heuristic evaluations • Discount usability engineering Section 4: Usability and the SE Process

More Related