1 / 54

3.4 Heuristics Evaluation

3.4 Heuristics Evaluation. What is wanted. Analysis. Design. Evaluation. Heuristic Evaluation. J. Nielsen & R. Molich Use heuristics General principles Domain-specific rules ‘Discount usability’ method How to do Several evaluators for 75% of the problems Independent review.

keely
Download Presentation

3.4 Heuristics Evaluation

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. 3.4 Heuristics Evaluation What is wanted Analysis Design Evaluation

  2. Heuristic Evaluation • J. Nielsen & R. Molich • Use heuristics • General principles • Domain-specific rules • ‘Discount usability’ method • How to do • Several evaluators for 75% of the problems • Independent review

  3. Nielsen’s 10 Heuristics Error prevention Recognition rather than recall Help and documentation Visibility of system status Match between system and the real world User control and freedom Consistency and standards Flexibilityand efficiency of use Help users recognize, diagnose, and recover from errors Aesthetic and minimalist design

  4. 3.5 Observations with Think-aloud What kinds of data can we collect? What is wanted Analysis Design Evaluation

  5. Think-Aloud • Users asked to talk • Advantages • Simple to do • Useful insights • Used in design and in evaluation • Disadvantages • May alter how users act; 霍桑效應 • Unnatural response changes how users act • Negatively affect quantitative measurement (e.g., time)

  6. Protocol Analysis • Protocol • Record of an evaluation session • Types • Paper and pencil, audio, video recording, computer logging, user notebooks • Transcription • Coding • Coding scheme • Ex: 資訊系統專案團隊成員互動型態之探索

  7. Post-Task Walkthrough • Replay the recordings • When to use • Why ‘undo’? • Why not button B? • Advantages • Sufficient time to frame questions • Minimize non-task-related talk during observation • Limitations • Loss of freshness

  8. 3.6 Observation with Eye Tracking 3.6.1 Analysis Tools

  9. Fixation • 定義 • 個體專注於視覺刺激中特定位置時 • 眼球相對固定不動時所停留的位置 • 兩項參數 • 固定不動的最小時間 (e.g., 80ms) • 眼球轉動的最大視角 (e.g., 0.5)

  10. Heatmap • Colored display for 2D data array • Advantages • Perceptual & locational • Parameters • Which users • Fixation length or # of fixations • Color scale

  11. Two Types of Heatmaps Fixation count heatmap Absolute gaze duration heatmap

  12. Guidelines for Using Heatmaps (X) let’s-track-and-see-what-happens approach (2) Define the 2D-data represented (1) Used for visualizing quantitative data (3) Define a fixation; e.g., 100ms, 0.5 (4) Define the upper bound for color scale (5) Describe the timeduration when fixations were collected

  13. Case Study: ASCO Original ASCO homepage Proposed ASCO homepage With which users can be quicker to find information?

  14. CoLiDeS(Comprehension-based Linked Model of Deliberate Search)

  15. CoLiDeS(Comprehension-based Linked Model of Deliberate Search)

  16. Bad Examples of Visual Search         Attention deployment Target processing Visual design Ease of understanding

  17. Participants and Apparatus • 12 participants • Age: 30-55 • ASCO member • Recruited at 2005 ASCO Annual Meeting in Orlando • Use experience: once a month (mean) • Apparatus • 17” monitor (1024 x 768 pixels) • a mouse • Tobii 1750 eye tracker

  18. Procedures • Within-subjects design • Counterbalanced against sequence effect • Randomized 25 tasks (Design 1) 20-min unrelated exercises 25 tasks (Design 2) Randomized Randomized Any confounding effect between old vs. new?

  19. TASK I: Find a list of upcoming conferences Original webpage Proposed webpage

  20. Analysis Results

  21. What Color Scheme Is Better? Red-Orange-Yellow-Blue-Gray Red-Orange-Yellow-Green-Gray

  22. Grayscale Scheme

  23. White Background?

  24. Gaze Plot&Gaze Replay • What they are? • Place • Order • Size • Limitations • 有時候看不到數字 • 有時候看不到網頁 • Time-consuming • Gaze replay Qualitative nature

  25. Areas of Interest (AOI)

  26. 3.6 Observation with Eye Tracking 3.6.2 Recruiting Participants

  27. Communicate to the Participant • Disclose the exposure to eye tracking • Ethically • Ensure no surprise to subjects • Nothing fishy to subjects • Don’t over-explain eye tracking • Subjects too aware of eye movement • Subjects might quit the test • Ex: “is that infrared going to hurt me?”

  28. How to Say It? • Simple introduction to eye tracking • 在實驗過程中，我們會觀察你使用網頁的方式 • 我們會應用科技來追蹤眼睛的移動 • 接下來我們將請教一些關於眼睛的基本問題 • 以確定你適合進入這個實驗

  29. Screener Questionnaire

  30. Impossible Eyetracker Calibration Bifocals/Trifocals 白內障(cataract)/ 青光眼(Glaucoma)

  31. Impossible Eyetracker Calibration 眼睛植入物(eye implant) 眼睛植入物(eye implant)

  32. Impossible Eyetracker Calibration Screen reader Permanently diluted pupil

  33. Impossible Eyetracker Calibration Extreme height Bushy eyelashes

  34. 問卷後說明 • Invitation to participate • Compensation • Session duration • Never be nervous • Detail all kinds of tracking • Schedule an appointment • Show test location • Share contact information

  35. 3.6 Observation with Eye Tracking 3.6.3 Pre-experiment Preparation

  36. Environmental Issues Avoid strong light over monitor Avoid eye dryness

  37. Typical Eye-Tracking Lab Setup

  38. 3.6 Observation with Eye Tracking 3.6.4 Formal Experiment

  39. Before the Sessions Begin… 開場白 任務說明 • Thanks • 研究目的(白話) • 強調觀察法的必要性 • 解釋實驗過程由任務組成 • 盡可能保持平常心態 • 鼓勵受試者認真 • 強調沒有標準答案 • 協議約定與重要性說明 • 用範例解釋 • 中斷說明

  40. Before the Sessions Begin… • 受試者的權利、義務 • 獎酬與收據 • 同意書 • 資料的使用與保密 • 眼動儀校調說明 • 畫面錄存與錄音

  41. Giving a Practice Task • 如何固定受試者坐姿 • 「請按照你平常使用網路時的坐姿，並請記得熟悉一下滑鼠跟鍵盤，並且調整椅子高低與位置。」 • What Nielsen’s subjects did • 協議預演 • 「我做完了！」 • Really important

  42. Do Eye-Tracking Calibration Participants sit in the center of the monitor Get a lock

  43. Do Eye-Tracking Calibration Be never shy about getting a lock Calibration

  44. Verbal Feedback in Getting Lock • 當校調失敗時… • 「我們必須再跑一次」 • 「不行，這次沒辦法」 • 「我們還會再做一次」 • Choose the best one you think

  45. Just Before the Experiment Begins Look at the monitor Final reminders • 直盯螢幕 • 聆聽施測者 • 提問或回應 • Not to move around • Keep a clear path • 有權利要求休息 • 非能力檢驗 • 提醒任務完成協議 • 最後的發問

  46. Begin a task

  47. Example Facilitator’s Protocol Sheet

  48. 3.6 Observation with Eye Tracking 3.6.5 Obstacles to Calibration

  49. Some Behaviors Affecting Tracking Brimmed hat Looking over glasses

  50. Some Behaviors Affecting Tracking Drinking Hand on the Chin

More Related