Usability Engineering

1. Usability Engineering Mary Czerwinski Microsoft Research

2. Agenda • Why usability test? • What is usability testing? • An example • Toward user-centered design • Other resources mpc--Microsoft Research

3. Why usability test? • Cost savings (well documented, see Neilsen, 1993) • Not always directly visible (PSS calls, resales, product returns, distributed productivity benefits to user, sw development costs) • Competitive market--user expectations • Political demands • Help might not mpc--Microsoft Research

4. What is usability? • Useful - Does it do what is needed? (teach, find, manage $, calibrate/extend the hw, escape) • Usability Is it easy to learn? Is efficient to use? Do no or only few errors occur? Is it easy to remember? • Desirability - Is it fun to use? Do you want to keep using it again and again? mpc--Microsoft Research

5. Development • Iterative test and design • Heuristic evaluation • Spec reviews • Low/Hi fidelity Quality Assurance Usability in your product cycle the earlier the better! Planning • Establish usability goals • Field research--tasks • Cognitive modeling • Competitive testing • Participatory design • UI design guidelines • Applied research • PSS communication • Roundtables • Low fidelity prototyping • Focus groups • Competitive testing • Field testing • PSS communication mpc--Microsoft Research

6. Ideal software • Software features should be discoverable Users can easily identify the controls or features that match the tasks they want to do Users can quickly identify what the software lets them do EVEN if they didn’t previously know this action was possible--principle of gradual disclosure mpc--Microsoft Research

7. Ideal software • Software features should be learnable If the feature is new or unfamiliar, the software (and on-line help) should contain information that guides the user through learning what the feature is for and how it works • Software features should be memorable After time spent on other activities, the user should be able to return to a feature and use it correctly mpc--Microsoft Research

8. Research and ideal software • Research on human cognitive abilities: • attention; visual perception • memory; learning • Research on human-computer interaction • applied research; task-oriented studies • heuristics for software design • Literature is a great resource for studies! mpc--Microsoft Research

9. Basic Cognitive Principles Memory • Associations are built by repetition • Scaffold model - more likely to remember items that have many associatons • Recognition is easier than recall • Working memory has small capacity (time & size) • Long-term memory has large capacity (time & size) mpc--Microsoft Research

10. Attention • Attention is a resource - gets divided between the different senses, different tasks • Automatic well-learned processes don’t require very much attention which means we can concentrate on new items • Context can • provide information • make observer focus on one part of the display • prime an observer so they’re biased towards what you want them to see mpc--Microsoft Research

11. Basic Cognitive Principles Visual Perception • We excel at pattern recognition • We automatically try to organize visual displays - gestalt principles • Motion, contrast, color, outlining can provide salience Also, motor control (becoming more important) mpc--Microsoft Research

12. Memory, Attention, and Visual Perception Dynamically Interact Basic Cognitive Principles MEMORY PERCEPTION What is this feature? Does it match the task? Recognition Pull info from memory Feedback ATTENTION Motor Control mpc--Microsoft Research

13. Early focus on users... • Feature Prioritization, Task Analysis • Participatory Design, Focus Groups • Heuristic Evaluation, Design Guidelines • Advisory Panels/Design Buddies • Field Studies • Cognitive Modeling exercises • Competitive Testing • Work with PSS mpc--Microsoft Research

14. Toward a user-centered design • Modeling customers’ activities (even mental ones) Understand activities, then create a solution A way to share information as a team • Generating multiple solutions • Developing usability goals Measuring against clear, quantifiable goals mpc--Microsoft Research

15. Usability Goals Table mpc--Microsoft Research

16. Usability goals--data analysis • Collecting data: video, protocols, subjective ratings and objective observations; debrief • Averages: times, % error time, # of trials before success, # of experimenter interventions, subjective ratings, # of task interrupts, % completed • Usability issues with # of Ss • Look for patterns and lines of converging evidence mpc--Microsoft Research

17. Recruiting participants • Need to find external users that match your target population • External users preferred to avoid “bias” • Work with marketing to understand user characteristics and screen for them • Screen via phone or Internet • Background questionnaire at beginning of study session • NDA, video consent mpc--Microsoft Research

18. Getting users comfortable • Greet the users promptly • Offer beverage, washroom • Give tour of lab, explain cameras, confidentiality • Emphasize it’s the software design that’s the issue, not the user’s expertise • Adjust chair, table heights--ergo concerns • Be enthusiastic, encouraging, listen! mpc--Microsoft Research

19. During the study session… Record EVERYTHING (and videotape) • Every keystroke, comment (use video) • Too hard to know ahead of time what’s important or a trend • Don’t give too much help (3 levels) and record: 1. Encouragement (“You’re doing great!”) 2. Hint (“Have you looked under Find?”) 3. Walk them through the task (“1st go here, then here…”) mpc--Microsoft Research

20. At session’s end…. • User Satisfaction Questionnaire (Chin, Diehl & Norman or your own) • Debrief Remind of key successes, failures and review Disclose total purpose of the study • Provide gratuity and thank participant • Escort out of the building mpc--Microsoft Research

21. Interpreting results… • Be quick and back up all of your usability issues with data (quantify it if possible) • Check your recommendations out with the team--some might not be possible • Recommend well thought-out UI changes; treat them like bugs • Retest to see if the changes fixed the problems mpc--Microsoft Research

22. Usability Issues Table • Table contains the issue #, its severity, the Ss that experienced the issue, its description, and its recommendation • Optionally, you can “status”--e.g., did group agree to fix the issue and how, when? • Another alternative is to use your corporation’s bug reporting tools mpc--Microsoft Research

23. Golden Rule(s) • List of top priority issues to team in 24 hours from last subject • Report with recommendations out in 1 week • Provide screen shots of tested UI and the recommended design solution • This implies learning a “quick and dirty” drawing or prototyping tool (Gives you more influence and shows you care about fixing the problem) mpc--Microsoft Research

24. Development stage: design, test & redesign • Not traditional “waterfall” model • Developing low-fi/hi-fi prototypes Broad and shallow Narrow and deep • Testing with a few users • Redesigning based on feedback • Testing again mpc--Microsoft Research

25. Drawing conclusions • Usability issues and recommendations • Updated usability goals table • Important to mark specifics down and publish for archival purposes • Usability issues should be tracked with PSS if unresolved mpc--Microsoft Research

26. Toward beta…. • Identify usability “showstoppers” before ship; fit and finish (e.g., audio tweaks, text) • Competitive benchmarking • Prioritize usability enhancements for next version • Field research to understand real usage of products in context and usability opportunities mpc--Microsoft Research

27. Cautions about lab testing • Doesn’t usually tell you what to design--structured user visits and interviews do • YOU set the tasks, the design, and the analysis • Best case performance • Look for patterns of behaviors--the usability issues with the UI design mpc--Microsoft Research

28. Accessibility issues • Include users with disabilities during the design phase--Lighthouse for the Blind, etc. • Also available--Design guidelines for accessible products mpc--Microsoft Research

29. Important considerations... • Ethical treatment of Ss, consent forms and NDAs • Statistical power and significance • Guided exploration v. free discovery, learning v. initial use • Validity, reliability, and generalizability • Objectivity mpc--Microsoft Research

30. Resources: Books • Rubin, J. 1995. Handbook of Usability Testing. • Dumas, J. & Redish, J. 1993. A practical guide to usability testing • Hellander, M. (Ed.) 1988; 1998. Handbook of human computer interaction: I & II. • Hix, D. & Hartson, H. 1993. Developing user interfaces • Nielson, J. 1993. Usability engineering • Preece, J. 1995. Human-computer interaction mpc--Microsoft Research

31. Resources: URLs • http://www.hfes.org • SIGCHI - http://www.acm.org/sigchi/ • HCI Index - http://is.twi.tudelft.nl/hci/ • HCI Resources on the Net - http://www.ida.liu.se/labs/aslab/groups/um/hci/ • HCI-BIB - http://www.cmd.uu.se./html/hcibib.html/ mpc--Microsoft Research

32. Questions?