Usability Methods: Cognitive Walkthrough & Heuristic Evaluation

1. Usability Methods: Cognitive Walkthrough & Heuristic Evaluation Dr. Dania Bilal IS 588 Spring 2008 Dr. D. Bilal

2. Purposes • Measures multiple components of the user interface • Addresses relationships between system and its users • Bridges the gap between human and machines

3. Purposes • Measures the quality of system design in relation to its intended users • Involves several methods, each applied at appropriate time of the design and development process

4. Usability Attributes • As described by Neilsen • Learnability • Efficiency • Memorability • Errors & their severity • Subjective satisfaction

5. Learnability • System must be easy to learn, especially for novice users • Hard to learn • systems are usually designed for expert users • Learning curve for novice and expert users

6. Efficiency • System should be efficient to use so that once the user has learned how to use it, the user can achieve a high level of productivity • Efficiency increases with learning

7. Memorability • System should be easy to remember, especially by casual users • No need to learn how to use system all over again after a period of not using it

8. Errors • System should have a low error rate • System should provide user with a recovery mechanism • Minor errors • Major errors

9. Minor Errors • Errors that did not greatly slow down user’s interaction with the system • User is able to recover from them • through system feedback • through awareness of error made

10. Major Errors • Difficult to recover from them • Lead to faulty work if high in frequency • May not be discovered by the user • Errors can be catastrophic

11. Subjective Satisfaction • System should be likeable by users (affective) • Satisfaction varies with • purpose of system • user goals

12. Assumptions • The designer’s best guess is not good enough • The user is always right • The user is not always right • Users are not designers • Designers are not users • More features are not always better • Minor interface details matter • Online help does not really help Source: Nielsen, J. (1993). Usability Engineering. San Diego: Morgan Kaufman.

13. Cognitive Walkthrough Method • Involves experts acting on behalf of actual users • Characteristics of typical users are identified & documented • Tasks focusing on aspects of design to be evaluated are developed

14. Cognitive Walkthrough Method • An observer “experimenter” is present • Prepares tasks • Takes notes • Provides help, etc. • Coordinates and overseas final report

15. Cognitive Walkthrough Method • Expert walkthrough interface on each task • Expert records problems that user may experience • Assumptions about what would cause problems and why are noted • Benchmarks may be used for each task

16. Sample Questions for Walkthrough • Will the user know what to do to complete part of or the whole task successfully? • Can the user see the button or icon to use for the next action? • Can the user find specific subject category from the hierarchy?

17. Cognitive Walkthrough • Each expert documents experience about walkthrough for each task • Critical problems documented • Problems and what cause them are explained • Draft report/notes are compiled and shared with other experts and Experimenter

18. Debriefing Session • Experts and experimenter meet & discuss findings • Experimenter shares his/her observational notes with experts • Findings include success stories & failure stories, as applicable • Consolidated report is generated

19. Walkthrough Report • Include questions experts for each of the tasks and the consolidated answer Use benchmarks and map out the finding for each task • See Assignment 4: Usability for additional information on benchmarks

20. Heuristic Evaluation • Evaluators interact with an interface several times and map interface to specific heuristics or guidelines • Example: Nielsen’s ten heuristics • Each evaluator generates a report • Reports are aggregated and final report is generated • An observer may be present

21. Stages of Heuristic Evaluation • Stage 1: Debriefing session • Experts told what to do • Written instructions provided to each expert • Heuristics provided to each expert as part of written instructions • Verbal instructions may be included

22. Stages of Heuristic Evaluation • Stage 2: Evaluation sessions • Each expert tests system based on heuristics • Expert may also use specific tasks • Two passes are taken through interface • First pass: overview and familiarity • Second pass: Focus on specific features & identify usability problems

23. Stages of Heuristic Evaluation • Stage 3: Debriefing session • Experts meet to discuss outcome and compare findings • Experts consolidate findings • Experts prioritize usability problems found & suggest solutions

24. Neilsen’s Heuristics • Ten heuristics found at • http://www.useit.com/papers/heuristic/heuristic_list.html • Additional rules, see Text. • Some heuristics can be combined under categories and given general description.

25. Usability Heuristics • http://www.usabilityfirst.com/methods • http://www.useit.com/papers/heuristic/heuristic_evaluation.html (how to conduct a heuristic evaluation) • http://www.uie.com/articles (collection of articles) • http://www.uie.com/articles/usability_tests_learn/ Learning about usability test (Jared Spool) • http://www.useit.com/papers/heuristic/severityrating.html (Severity rating)