1 / 25

Comp 15 - Usability and Human Factors

Learn about different usability evaluation methods such as interviews, surveys/questionnaires, and inspection, and how they can be used to improve the usability of systems. This material was developed by Columbia University and funded by the Department of Health and Human Services.

bmorgan
Download Presentation

Comp 15 - Usability and Human Factors

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Comp 15 - Usability and Human Factors Unit 5a - Usability Evaluation Methods This material was developed by Columbia University, funded by the Department of Health and Human Services, Office of the National Coordinator for Health Information Technology under Award Number 1U24OC000003.

  2. Overview • Usability • Evaluation methods • Interviews • Surveys / questionnaires • Usability inspection methods • Heuristic evaluation • Cognitive walkthrough • Usability testing Health IT Workforce Curriculum Version 2.0/Spring 2011

  3. Usability Health IT Workforce Curriculum Version 2.0/Spring 2011

  4. Why Usability Matters: The Clinical Story Health IT Workforce Curriculum Version 2.0/Spring 2011

  5. Why Usability Matters: The Story from Patients and Consumers Health IT Workforce Curriculum Version 2.0/Spring 2011

  6. Interviews/Focus groups Questionnaires/Surveys Ethnographic Observations Usability Inspection Methods Usability Testing Controlled Cognitive Experiments Usability Evaluation Methods Health IT Workforce Curriculum Version 2.0/Spring 2011

  7. Usability Evaluation Health IT Workforce Curriculum Version 2.0/Spring 2011

  8. Interviews:Four Main Types Health IT Workforce Curriculum Version 2.0/Spring 2011

  9. Interviews:Four Main Types (cont.) Health IT Workforce Curriculum Version 2.0/Spring 2011

  10. Phases of the Interview Health IT Workforce Curriculum Version 2.0/Spring 2011

  11. Usability Interview Example Health IT Workforce Curriculum Version 2.0/Spring 2011

  12. Focus Groups • Commonly used method in marketing and social science research • Typically 3 to 10 participants representative of the sample population • For example: attending physicians, residents, nurses, pharmacists & administrators • Discussion led by trained facilitator • Preset agenda to guide discussion • Sufficient flexibility to pursue emergent issues • Facilitator guides and prompts discussion • Encourages less talkative participants to speak up • Discourages more verbose participants from dominating Health IT Workforce Curriculum Version 2.0/Spring 2011

  13. Questionnaire / Surveys • Common method for collecting user demographic data and soliciting opinions • Can reach a rather large group of people • Questions are typically closed, though open comments may be solicited • Some use a mix of negative and positive questions • Require respondent to be more attentive Health IT Workforce Curriculum Version 2.0/Spring 2011

  14. Rating Scales: Likert • Mostly commonly used • Measure opinions, attitudes and beliefs • Widely used for evaluating user satisfaction • Statement representing a range of opinions • I found the system to be very easy to use • Rate response on a continuum from one extreme (e.g., highest or strongest agreement) to another (lowest or strongest disagreement) Health IT Workforce Curriculum Version 2.0/Spring 2011

  15. Sample Usability Likert Questions Health IT Workforce Curriculum Version 2.0/Spring 2011

  16. Rating Scales: Semantic Differential • Explore a range of bipolar attitudes about a particular item • Participant is asked to place mark in one of a number of positions between 2 extremes to indicate agreement • I found the website to be (attractive  ugly) • I found the materials to be (clear  confusing) Health IT Workforce Curriculum Version 2.0/Spring 2011

  17. Online Surveys:Two Types Health IT Workforce Curriculum Version 2.0/Spring 2011

  18. Survey Monkey • http://www.surveymonkey.com • Illustration of online survey • Focuses on user experience • Different forms of interaction • Check boxes for multiple selection • Radio buttons for choosing among alternatives or rating items on Likert scale • Configurable and flexible Health IT Workforce Curriculum Version 2.0/Spring 2011

  19. Usability Inspection Methods:Expert Reviews Health IT Workforce Curriculum Version 2.0/Spring 2011

  20. Heuristic Evaluation • Developed by Jakob Nielsen and Rolf Molich • Method for structuring the critique of a system using simple & general heuristics • Several analysts critique a system to identify & diagnose usability problems • Display based • Each evaluator assesses the system & notes violations of usability principles • In a second pass, analyst rates severity of problems • Can be used in all phases of design including storyboards, prototypes and fully functioning systems Health IT Workforce Curriculum Version 2.0/Spring 2011

  21. Usability Principles (Nielsen, 1993) • Visibility of system status • Match between system and the real world • User control and freedom • Consistency and standards • Help users recognize, diagnose, and recover from errors • Minimize memory load • Emphasize recognition rather than recall • Flexibility and efficiency • Motivation and engagement Health IT Workforce Curriculum Version 2.0/Spring 2011

  22. More Specific Heuristics (Gerhardt-Powals, 1996) • Automate unwanted workload: • Free cognitive resources for high-level tasks • Eliminate mental calculations, estimations, comparisons, & thinking • Reduce uncertainty; display data in a clear and obvious manner • Reduce cognitive load by bringing together lower level data into a higher-level summation • Present new information with meaningful aids to interpretation: • Use a familiar framework, everyday terms, metaphors Health IT Workforce Curriculum Version 2.0/Spring 2011

  23. More Specific Heuristics (cont.) • Use names that are conceptually related to function • Context-dependent • Attempt to improve recall and recognition • Group data consistently meaningful ways to decrease search time • Limit data-driven tasks: • Reduce time assimilating raw data, appropriate use of color & graphics • Include only information needed by the user at a given time Health IT Workforce Curriculum Version 2.0/Spring 2011

  24. Problem Description, Recommended Solution & Commentary P. Description is clear, key points should be more visually prominent. S. Use bullets or bolded Text Heuristics Violated: Consistency Standards P. Picture is extraneous and eats up valuable space. S. Summary positioned above the picture. Heuristics Violated: Minimalism P. An excess of text S. Convey points more succinctly, use bullets and eliminate sentences Heuristics Violated: Minimalism P. User is not fully aware of scope of session and perhaps program S. Perhaps an advance organizer and a timeline would be helpful Heuristics Violated: Visibility P. Can't return to session from Discussion by clicking back arrow. Heuristics Violated: Control/Flexibility P. Change Program is a misleading label for editing preferences. Heuristics Violated: Match P. Help is not sufficiently informative and context sensitive. S. Useful if system provides some context-sensitive guidance. Heuristics Violated: Help P. Can;t return to English after switching to Spanish Heuristics Violated: Reversible action/Control Heuristic Evaluation Health IT Workforce Curriculum Version 2.0/Spring 2011

  25. Problem Severity Health IT Workforce Curriculum Version 2.0/Spring 2011

More Related