1 / 53

Unit 3: evaluation

Unit 3: evaluation. 3.1 Cognitive Walkthrough . What is wanted. Analysis. Design. Evaluation. Cognitive Walkthrough. Exploratory learning Users tend to love learning by exploration Similar to software code walkthrough. Inputs of Cognitive Walkthrough. Questions Asked by Evaluator.

fred
Download Presentation

Unit 3: evaluation

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Unit 3: evaluation

  2. 3.1Cognitive Walkthrough What is wanted Analysis Design Evaluation

  3. Cognitive Walkthrough • Exploratory learning • Users tend to love learning by exploration • Similar to software code walkthrough

  4. Inputs of Cognitive Walkthrough

  5. Questions Asked by Evaluator

  6. Train ticket vending machine from [Newman & Lamming 1995] 1. Choose destination or enter fare Newark Rahway Metropark Metuchen Edison New Brunswick 2. Indicate journey type One-way Round-trip 3. Deposit money coins bills LIFT for ticket and change 4. Press to receive ticket and change Cognitive Walkthrough Example

  7. Scenario A novice user wants to buy a one-way ticket to New Brunswick. She has a $5 bill in her pocket + some change. Task-definition question Q0: “What is the user trying to achieve?” Answer: “Purchase a one-way ticket to New Brunswick.” Cognitive Walkthrough Example

  8. 1. Will the correct action be evident to the user? 1. Choose destination or enter fare Newark Rahway Metropark Metuchen Edison New Brunswick 2. Indicate journey type One-way Round-trip 3. Deposit money coins bills LIFT for ticket and change 4. Press to receive ticket and change Cognitive Walkthrough Example 問題：使用者要做出正確動作會不會很困難？(X)

  9. 1. Will the correct action be evident to the user? (subgoals provided to user) Cognitive Walkthrough Example 1. Choose destination or enter fare Newark Rahway Metropark Metuchen Edison New Brunswick 2. Indicate journey type One-way Round-trip Yes 3. Deposit money coins bills LIFT for ticket and change 4. Press to receive ticket and change

  10. 2. Will the user connect the correct action’s description/label with his/her intent? Cognitive Walkthrough Example 1. Choose destination or enter fare Newark Rahway Metropark Metuchen Edison New Brunswick 2. Indicate journey type One-way Round-trip 3. Deposit money coins bills LIFT for ticket and change 4. Press to receive ticket and change

  11. 2. Will the user connect the correct action’s description/label with his/her intent? Cognitive Walkthrough Example 1. Choose destination or enter fare Newark Rahway Metropark Metuchen Edison New Brunswick 2. Indicate journey type One-way Round-trip Yes 3. Deposit money coins bills LIFT for ticket and change 4. Press to receive ticket and change

  12. 3. Will the user properly interpret the system’s response? (know if chose right/wrong?) Cognitive Walkthrough Example 1. Choose destination or enter fare Newark Rahway Metropark Metuchen Edison New Brunswick 2. Indicate journey type One-way Round-trip 3. Deposit money coins bills LIFT for ticket and change 4. Press to receive ticket and change

  13. 3. Will the user properly interpret the system’s response? (know if chose right/wrong?) Cognitive Walkthrough Example 1. Choose destination or enter fare Newark Rahway Metropark Metuchen Edison New Brunswick 2. Indicate journey type One-way Round-trip Yes (if button lights up…) 3. Deposit money coins bills LIFT for ticket and change 4. Press to receive ticket and change

  14. 1. Will the correct action be evident to the user? Cognitive Walkthrough Example 1. Choose destination or enter fare Newark Rahway Metropark Metuchen Edison New Brunswick 2. Indicate journey type One-way Round-trip 3. Deposit money coins bills LIFT for ticket and change 4. Press to receive ticket and change

  15. 1. Will the correct action be evident to the user? Cognitive Walkthrough Example Yes 1. Choose destination or enter fare Newark Rahway Metropark Metuchen Edison New Brunswick 2. Indicate journey type One-way Round-trip 3. Deposit money coins bills LIFT for ticket and change 4. Press to receive ticket and change

  16. 2. Will the user connect the correct action’s description/label with his/her intent? Cognitive Walkthrough Example 1. Choose destination or enter fare Newark Rahway Metropark Metuchen Edison New Brunswick 2. Indicate journey type One-way Round-trip 3. Deposit money coins bills LIFT for ticket and change 4. Press to receive ticket and change

  17. 2. Will the user connect the correct action’s description/label with his/her intent? Cognitive Walkthrough Example 1. Choose destination or enter fare Newark Rahway Metropark Metuchen Edison New Brunswick 2. Indicate journey type One-way Round-trip Yes 3. Deposit money coins bills LIFT for ticket and change 4. Press to receive ticket and change

  18. 3. Will the user properly interpret the system’s response? (know if chose right/wrong?) Cognitive Walkthrough Example 1. Choose destination or enter fare Newark Rahway Metropark Metuchen Edison New Brunswick 2. Indicate journey type One-way Round-trip 3. Deposit money coins bills LIFT for ticket and change 4. Press to receive ticket and change

  19. 3. Will the user properly interpret the system’s response? (know if chose right/wrong?) Cognitive Walkthrough Example 1. Choose destination or enter fare Newark Rahway Metropark Metuchen Edison New Brunswick 2. Indicate journey type One-way Round-trip Yes … One-way $6.35 3. Deposit money coins bills LIFT for ticket and change 4. Press to receive ticket and change

  20. 1. Will the correct action be evident to the user? Cognitive Walkthrough Example 1. Choose destination or enter fare Newark Rahway Metropark Metuchen Edison New Brunswick 2. Indicate journey type One-way Round-trip One-way $6.35 3. Deposit money coins bills LIFT for ticket and change 4. Press to receive ticket and change

  21. 1. Will the correct action be evident to the user? Cognitive Walkthrough Example Yes 1. Choose destination or enter fare Newark Rahway Metropark Metuchen Edison New Brunswick 2. Indicate journey type One-way Round-trip One-way $6.35 3. Deposit money coins bills LIFT for ticket and change 4. Press to receive ticket and change

  22. 2. Will the user connect the correct action’s description/label with his/her intent? Cognitive Walkthrough Example 1. Choose destination or enter fare Newark Rahway Metropark Metuchen Edison New Brunswick 2. Indicate journey type One-way Round-trip One-way $6.35 3. Deposit money coins bills LIFT for ticket and change 4. Press to receive ticket and change

  23. 2. Will the user connect the correct action’s description/label with his/her intent? Cognitive Walkthrough Example 1. Choose destination or enter fare Newark Rahway Metropark Metuchen Edison New Brunswick 2. Indicate journey type One-way Round-trip One-way $6.35 3. Deposit money Yes coins bills LIFT for ticket and change 4. Press to receive ticket and change

  24. 3. Will the user properly interpret the system’s response? (know if chose right/wrong?) Cognitive Walkthrough Example 1. Choose destination or enter fare Newark Rahway Metropark Metuchen Edison New Brunswick 2. Indicate journey type One-way Round-trip One-way $6.35 3. Deposit money coins bills $5 LIFT for ticket and change 4. Press to receive ticket and change

  25. 3. Will the user properly interpret the system’s response? (know if chose right/wrong?) 1. Choose destination or enter fare Newark Rahway Metropark Metuchen Edison New Brunswick 2. Indicate journey type One-way Round-trip One-way $6.35 Recv’d: $5.00 3. Deposit money coins bills LIFT for ticket and change 4. Press to receive ticket and change Cognitive Walkthrough Example No, need feedback!  Add received amount display

  26. Not enough $… 1. Will the correct action be evident? 1. Choose destination or enter fare Newark Rahway Metropark Metuchen Edison New Brunswick 2. Indicate journey type One-way Round-trip One-way $6.35 Recv’d: $5.00 3. Deposit money coins bills LIFT for ticket and change 4. Press to receive ticket and change Cognitive Walkthrough Example

  27. Not enough $… 1. Will the correct action be evident? 1. Choose destination or enter fare Newark Rahway Metropark Metuchen Edison New Brunswick 2. Indicate journey type One-way Round-trip One-way $6.35 Recv’d: $5.00 3. Deposit money coins bills LIFT for ticket and change 4. Press to receive ticket and change Cognitive Walkthrough Example Probably not! ? ? ?

  28. Not enough $… 1. Will the correct action be evident? Cognitive Walkthrough Example Probably not! Add a new UI element to make it clear! 1. Choose destination or enter fare Newark Rahway Metropark Metuchen Edison New Brunswick 2. Indicate journey type One-way Round-trip One-way $6.35 Recv’d: $5.00 3. Deposit money coins bills Cancel & return money LIFT for ticket and change 4. Press to receive ticket and change

  29. Sample Cover Form for Cognitive Walkthrough

  30. Initial Design of VCR Remote Control

  31. Action Description

  32. Sample Form for Cognitive Walkthrough

  33. 3.2 Cognitive Task Analysis What is wanted Analysis Design Evaluation

  34. Cognitive Task Analysis • Core methodology used in cognitive science • Study human performance • In laboratory • In field • Decision making, reasoning, and information needs • Physical & mental • GOMS(Goals, Operators, Methods, and Selection rules)

  35. Goal • 「使用者必須完成的事物」 • 「花費努力的方向」

  36. Operator • 「針對目標所進行的動作(action)」 • 「移動滑鼠」、「按滑鼠左鍵」 • 「按滑鼠右鍵」、「移動DELETE鍵」 • 導致狀態改變 • Mentally or physically • 執行時間 • 操作時間獨立性 • Operartor估計的正確性模型整體的正確性

  37. Method 標定 刪除法 字元 刪除法

  38. Selection Rules • IF 段落字元數 > 10 • THEN Use 標定刪除法 • ELSE Use 字元刪除法

  39. Coarse GOMS model

  40. Fine GOMS model Goal Subgoal Operator

  41. Decomposition Criteria 操作時間可衡量性 操作時間獨立性

  42. Keystroke-Level Model (KLM)Operators

  43. KLM Example Design 1: Use 資源回收桶 Design 2: 檔案總管刪除選單

  44. Action Sequence in Design 1

  45. Operator Sequence in Design 1

  46. Action Sequence in Design 2

  47. Operator Sequence in Design 2

  48. Modeling with Ms How much $$? Password? Where the button? The account#? Final check

  49. Adding Ms to Operator Sequence in Design 1(Novice)

  50. Adding Ms to Operator Sequence in Design 1(Novice)

More Related