CS 544 Discount Usability Engineering

1. CS 544 Discount Usability Engineering Heuristic Evaluation Acknowledgement: Some of the material in these lectures is based on material prepared for similar courses by Saul Greenberg (University of Calgary), Ravin Balakrishnan (University of Toronto), James Landay (University of California at Berkeley), monica schraefel (University of Toronto), and Colin Ware (University of New Hampshire). Used with the permission of the respective original authors.

2. Usability Inspection • is the generic name for a set of methods based on having evaluators inspect or examine usability-related aspects of a user interface • usability inspectors can be usability specialists, software development consultants with special expertise, end users with task/content knowledge, or other professionals • 4 ways to evaluate an interface: • automatically (compute usability measures using evaluation software) • empirically (usability testing with real users) • formally (using models to calculate usability measures) • informally (rule of thumb and general knowledge of evaluators) • usability inspection is an informal evaluation

3. Discount Usability Engineering Developed by Nielson, based on using four techniques: • User and task observation • Scenarios • Simplified thinking aloud • Heuristic evaluation

4. Interface Design and Usability Engineering • Articulate: • who users are • their key tasks Brainstorm designs Refined designs Completed designs Goals: Task centered system design Participatory design User-centered design Graphical screen design Interface guidelines Style guides Psychology of everyday things User involvement Representation & metaphors Participatory interaction Task scenario walk-through Evaluate Usability testing Heuristic evaluation Field testing Methods: high fidelity prototyping methods low fidelity prototyping methods Throw-away paper prototypes Products: User and task descriptions Testable prototypes Alpha/beta systems or complete specification

5. Discount usability engineering • Cheap • no special labs or equipment needed • the more careful you are, the better it gets • Fast • on order of 1 day to apply • standard usability testing may take a week • Easy to use • can be taught in 2-4 hours

6. Heuristic Evaluation • Developed by Jakob Nielsen • Principles used to “evaluate” a system for usability problems • Becoming very popular • user involvement not required • catches many design flaws • Small set (3-5) of evaluators examine UI • independently check for compliance with usability principles (“heuristics”) • different evaluators will find different problems • evaluators only communicate afterwards • findings are then aggregated • Can perform on working UI or on sketches

7. Why multiple evaluators? • Every evaluator doesn’t find every problem • Good evaluators find both easy & hard ones

8. Heuristic Evaluation Process • Evaluators go through UI several times • inspect various dialogue elements • compare with list of usability principles • consider other principles/results that come to mind • Usability principles • Nielsen’s “heuristics” • supplementary list of category-specific heuristics • competitive analysis & user testing of existing products • Use violations to redesign/fix problems

9. Neilson’s Heuristics (original) • H1-1: Simple & natural dialog • H1-2: Speak the users’ language • H1-3: Minimize users’ memory load • H1-4: Consistency • H1-5: Feedback • H1-6: Clearly marked exits • H1-7: Shortcuts • H1-8: Precise & constructive error messages • H1-9: Prevent errors • H1-10: Help and documentation

10. searching database for matches H2-1: Visibility of system status • keep users informed about what is going on • example: pay attention to response time • 0.1 sec: no special indicators needed, why? • 1.0 sec: user tends to lose track of data • 10 sec: max. duration if user to stay focused on action • for longer delays, use percent-done progress bars

11. H2-1: Visibility of system status • keep users informed about what is going on • Appropriate visible feedback What mode am I in now? What did I select? How is the system interpreting my actions?

12. H2-2: Match between system & real world • speak the users’ language • follow real world conventions • (bad) example: Mac desktop • Dragging disk to trash • should delete it, not eject it!

13. H2-2: Match between system & real world • speak the users’ language • follow real world conventions • e.g. withdrawing money from a bank machine

14. H2-3: User control & freedom • “exits” for mistaken choices, undo, redo • don’t force down fixed paths How do I get out of this?

15. H2-3: User control & freedom • “exits” for mistaken choices, undo, redo • don’t force down fixed paths • Strategies: • Cancel button (for dialogs waiting for user input) • Universal Undo (can get back to previous state) • Interrupt (especially for lengthy operations) • Quit (for leaving the program at any time) • Defaults (for restoring a property sheet)

16. H2-4: Consistency & standards • Consistency of effects • same words, commands, actions will always have the same effect in equivalent situations • predictability • Consistency of language and graphics • same info/controls in same location on all screens/dialog boxes • forms follow boiler plate • same visual appearance across the system (e.g. widgets) • e.g. different scroll bars in a single window system! • Consistency of input • consistent syntax across complete system

17. H2-4: Consistency & standards These are labels with a raised appearance. Is it any surprise that people try and click on them?

18. H2-5: Error prevention • try to make errors impossible • modern widgets: only “legal commands” selected, or “legal data” entered • Provide reasonableness checks on input data • on entering order for office supplies • 5000 pencils is an unusually large order. Do you really want to order that many?

19. H2-6: Recognition rather than recall • Computers good at remembering things, people aren’t! • Promote recognition over recall • menus, icons, choice dialog boxes vs command lines, field formats • relies on visibility of objects to the user (but less is more!)

20. H2-7: Flexibility and efficiency of use • Experienced users should be able to perform frequently used operations quickly • Strategies: • keyboard and mouse accelerators • abbreviations • command completion • menu shortcuts • function keys • double clicking vs menu selection • type-ahead (entering input before the system is ready for it) • navigation jumps • e.g., going to window/location directly, and avoiding intermediate nodes • history systems • WWW: ~60% of pages are revisits

21. H2-7: Flexibility and efficiency of use Keyboard accelerators for menus Customizable toolbars andpalettes for frequent actions Split menu, with recently used fonts on top Double-click raises toolbar dialog box Double-click raises object-specific menu Scrolling controls for page-sized increments

22. H2-8: Aesthetic and minimalist design • No irrelevant information in dialogues • Bad example:

23. H2-9: Help users recognize, diagnose, and recover from errors • error messages in plain language • precisely indicate the problem • constructively suggest a solution

24. Errors we make • Mistakes • arise from conscious deliberations that lead to an error instead of the correct solution • Slips • unconscious behaviour that gets misdirected en route to satisfying goal • e.g. drive to store, end up in the office • shows up frequently in skilled behaviour • usually due to inattention • often arises from similarities of actions

25. Types of slips • Capture error • frequently done activity takes charge instead of one intended • occurs when common and rarer actions have same initial sequence • change clothes for dinner and find oneself in bed (William James, 1890) • confirm saving of a file when you don’t want to delete it I can’t believe I pressed Yes...

26. Types of slips • Description error • intended action has much in common with others that are possible • usually occurs when right and wrong objects physically near each other • pour juice into bowl instead of glass • go jogging, come home, throw sweaty shirt in toilet instead of laundry basket • move file to trash instead of to folder • Loss of activation • forgetting what the goal is while undergoing the sequence of actions • start going to room and forget why you are going there • navigating menus/dialogs and can’t remember what you are looking for • but continue action to remember (or go back to beginning)! • Mode errors • people do actions in one mode thinking they are in another • refer to file that’s in a different directory • look for commands / menu options that are not relevant

27. Designing for slips • General rules • Prevent slips before they occur • Detect and correct slips when they do occur • User correction through feedback and undo • Examples • mode errors • have as few modes as possible (preferably none) • make modes highly visible • capture errors • instead of confirmation, make actions undoable • allows reconsideration of action by user • e.g. Mac trash can can be opened and “deleted” file taken back out • loss of activation • if system knows goal, make it explicit • if not, allow person to see path taken • description errors • in icon-based interfaces, make sure icons are not too similar, • check for reasonable input, etc.

28. Generic system responses for errors • General idea: Forcing functions • prevent / mitigate continuation of wrongful action • Gag • deals with errors by preventing the user from continuing • eg cannot get past login screen until correct password entered • Warn • warn people that an unusual situation is occurring • when overused, becomes an irritant • e.g., • audible bell • alert box • Do nothing • illegal action just doesn’t do anything • user must infer what happened • enter letter into a numeric-only field (key clicks ignored) • put a file icon on top of another file icon (returns it to original position)

29. Generic system responses for errors • Self-correct • system guesses legal action and does it instead • but leads to a problem of trust • spelling corrector • Lets talk about it • system initiates dialog with user to come up with solution to the problem • compile error brings up offending line in source code • Teach me • system asks user what the action was supposed to have meant • action then becomes a legal one

30. H2-10: Help and documentation • Help is not a replacement for bad design! • Simple systems: • walk up and use; minimal instructions • Most other systems: • feature rich • some users will want to become “experts” rather than “casual” users • intermediate users need reminding, plus a learning path • Many users do not read manuals • prefer to spend their time pursuing their task • Usually used when users are in some kind of panic, need immediate help • indicates need for online documentation, good search/lookup tools • online help can be specific to current context • paper manuals unavailable in many businesses! • e.g. single copy locked away in system administrator’s office • Sometimes used for quick reference • syntax of actions, possibilities... • list of shortcuts ...

31. Types of help • Tutorial and/or getting started manuals • short guides that people are likely to read when first obtaining their systems • encourages exploration and getting to know the system • tries to get conceptual material across and essential syntax • on-line “tours”, exercises, and demos • demonstrates very basic principles through working examples • Reference manuals • used mostly for detailed lookup by experts • rarely introduces concepts • thematically arranged • on-line hypertext • search / find • table of contents • index • cross-index

32. Types of help (cont.) • Reminders • short reference cards • expert user who just wants to check facts • novice who wants to get overview of system’s capabilities • keyboard templates • shortcuts/syntactic meanings of keys; recognition vs. recall; capabilities • tooltips • text over graphical items indicates their meaning or purpose

33. Types of help (cont.) • Context-sensitive help • system provides help on the interface component the user is currently working with • Macintosh “balloon help” • Microsoft “What’s this” help • brief help explaining whatever the user is pointing at on the screen • Wizards • walks user through typical tasks • but dangerous if user gets stuck • Remember the MS “paperclip”?

34. How to perform evaluation • At least two passes for each evaluator • first to get feel for flow and scope of system • second to focus on specific elements • If system is walk-up-and-use or evaluators are domain experts, no assistance needed • otherwise might supply evaluators with scenarios • Each evaluator produces list of problems • explain why with reference to heuristic or other information • be specific and list each problem separately

35. How to perform evaluation (cont.) • Why separate listings for each violation? • risk of repeating problematic aspect • may not be possible to fix all problems • Where problems may be found • single location in UI • two or more locations that need to be compared • problem with overall structure of UI • something that is missing • hard w/ paper prototypes so work extra hard on those • note: sometimes features are implied by design docs and just haven’t been “implemented” – relax on those

36. Severity ratings • Used to allocate resources to fix problems • Estimates of need for more usability efforts • Combination of • frequency • impact • persistence (one time or repeating) • Should be calculated after all evals. are in • Should be done independently by all judges

37. Severity ratings (cont). 0 - don’t agree that this is a usability problem 1 - cosmetic problem 2 - minor usability problem 3 - major usability problem; important to fix 4 - usability catastrophe; imperative to fix

38. Summary • Advantages • the “minimalist” approach • a few general guidelines can correct for the majority of usability problems • easily remembered, easily applied with modest effort • discount usability engineering • cheap and fast way to inspect a system • can be done by usability experts, double experts, and end users • Problems: • principles are more or less at the motherhood level • can’t be treated as a simple checklist • subtleties involved in their use • doesn’t necessarily predict user/customers overall satisfaction • may not have the same “credibility” as user test data (one solution to this is to include the design team and developers in the usability evaluation, as in a “pluralistic walkthrough”)

39. Summary • Research result: • between 4-5 evaluators appear to be sufficient in most cases to identify 80% of total usability problems • user testing and usability inspection have a large degree of non-overlap in the usability problems they find (i.e., it pays to use both methods) • Cost-benefit: • usability engineering activities are often difficult to justify and carry out in a timely way, but many activities can be done quickly and cheaply, and produce useful results • usability inspection turns less on what is “correct” than on what can be done within development constraints • ultimate trade-off may be between doing no usability assessment and doing some kind

40. You now know • 10 Heuristics for evaluating designs: • H2-1: Visibility of system status • H2-2: Match between system & real world • H2-3: User control & freedom • H2-4: Consistency & standards • H2-5: Error prevention • H2-6: Recognition rather than recall • H2-7: Flexibility and efficiency of use • H2-8: Aesthetic and minimalist design • H2-9: Help users recognize, diagnose, and recover from errors • H2-10: Help and documentation • Principles can be used to systematically inspect the interface for usability problems

41. Readings and References • BGBG 80-86 (Evaluating Systems and Their User Interfaces) • BGBG 170-181 (Usability Inspection Methods) Optional: • Neilsen, J. (1993). Usability Engineering. Read Chapter 5: Usability Heuristics, 115 – 163.