• 360 likes • 524 Views
Cognitive Walkthroughs and heuristic evaluation. Evaluation - definitions. to assess the extent of the system’s functionality (i.e. does it map onto the user’s task requirements) to assess the effect (not affect, generally) of UI on the user to identify any specific problems with the system.
E N D
Evaluation - definitions • to assess the extent of the system’s functionality (i.e. does it map onto the user’s task requirements) • to assess the effect (not affect, generally) of UI on the user • to identify any specific problems with the system
Evaluation is ... • Essential • Time consuming • Often difficult to organise • e.g. the availability of subjects / participants • But there is an easy(ish) alternative to working with end-users
Cognitive walkthroughs • Structured approach to usability analysis • Intended to supplement NOT replace empirical approaches (empirical = with users in this instance) • A number of different versions exist... • ‘Hand simulation of the cognitive activities of a user’ Polson et al, 1992
Performing a walkthrough • Create a usage scenario • Identify a task within the scenario • Perform the walkthrough • Record the problems found
To begin... • Scenario: • e.g. someone writing a letter • Task: • To copy and paste some text • sub-tasks • Copy text into clipboard • Select point to enter (pasted) text • Issue paste command
Tasks for copy & pasting text Note the order!
Examining the sub-goals • Will the user try to achieve the right goal? • What knowledge is needed to achieve the right subgoal? Will the user have this knowledge?
Satisfying goals • Will the user notice that the correct action is available? • Will the user associate the correct action with the sub-goal they are trying to achieve?
Feedback • Will the user perceive the feedback? • Will the user understand the feedback? • Will the user see that progress is being made towards the solution of their task?
Tasks for copy & pasting text 1. Apply Q1& 2 to sub-goal B 2. Apply Q1 & 2 to C 3. Apply Q3-7
Recording problems • Any questions answered to the negative indicate potential usability problems • Describe the problem in as much detail as possible • When the problem occurred • Why the problem occurred • The consequences of the problem
Now your turn... • Using your task analysis of requesting a loan conduct a cognitive walkthrough • Report the problems • consider how they might be fixed
heuristic evaluation ‘heuristic’ - ‘used of problem solving techniques that proceed by trial and error’ (related to Greek ‘eureka’) Longman Concise English Dictionary, 1985 a method of usability evaluation where an analyst finds usability problems by checking the user interface against a set of supplied heuristics or principles Lavery, Cockton and Atkinson, 1996
who should do heuristic evaluation? • use more than one evaluator • ideally should not be the designer • ideally should be a usability specialist • technical authors also useful • each carries out independent inspection, then aggregate findings • evaluators may need help • unless ‘walk-up-and-use’ application • could provide typical usage scenario
effectiveness of increasing number of evaluators (after Nielsen, 1993)
evaluator performance Nielsen 1992: • same interface evaluated by 3 groups • novice (knowledge about computers only) • usability experts • usability and specialist domain experts
how to do heuristic evaluation • go through interface, compare against recognised usability principles • first pass flow of interaction and general scope • subsequently focus on specific elements • typically 1 - 2 hours in total • output: list of usability problems cross-referenced to usability guidelines
learnability memorability errors efficiency subjective satisfaction usability metrics
but how to set the target level... • skill and intuition... • better than last version • better than the competition • client targets • set a range of levels • unacceptable • minimum • target • ideal
STOP remember materials for next week
usability testing - planning • draw up a test plan • see separate handout ‘Checklist for usability test plans’ • informal use by the test team • formal use for QA procedures • consider whether to use video/audio recording
usability testing - users • test users should represent target users • remember sales staff as a special user group • may need to give basic training • getting hold of users • internal users should be easy • customers from user groups may help • paid volunteers : students, classified ads.. • take account of older users if relevant
usability testing - designing test tasks • representative & provide reasonable coverage • do-able but not trivial • consider relating to a larger scenario • provide a written task description • present in increasing level of difficulty • decide whether to use verbal protocols
relative effectiveness • Karat el (1992) • expert individual & group walkthroughs • used guidelines and tasks • usability testing • users identified and described problems • testing identified most problems • including some severe ones missed by experts
effectiveness continued • walkthroughs useful when resources linited, or for early design • team walkthroughs better than individuals • techniques are complementary • cost effectiveness similar • also formal experimental trials
usability testing - procedure • preparation • remember to switch off screen-savers, email, etc. • introduction • testing • debriefing • questionnaires if used • also ask about the testing process • write up quickly
learnability time to reach specified level of proficiency, e.g. complete a specified, representative task note that learning is a continuum memorability test users on commands after trial session errors number of errors in completing specified task subjective satisfaction rating scales physiological measures efficiency times for experts to complete specified task(s) frequency of ‘non-productive’ actions ratio of used to unused commands usability metrics