1 / 23

Formative Evaluation

Formative Evaluation. cs3724: HCI. ANALYZE. analysis of stakeholders, field studies. claims about current practice. Problem scenarios. Scenario-Based Design. DESIGN. Activity scenarios. metaphors, information technology, HCI theory, guidelines. iterative analysis of

aira
Download Presentation

Formative Evaluation

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Formative Evaluation cs3724: HCI

  2. ANALYZE analysis of stakeholders, field studies claims about current practice Problem scenarios Scenario-Based Design DESIGN Activity scenarios metaphors, information technology, HCI theory, guidelines iterative analysis of usability claims and re-design Information scenarios Interaction scenarios PROTOTYPE & EVALUATE summative evaluation formative evaluation Usability specifications

  3. Usability Engineering Reqs Analysis Design Evaluate Prototype

  4. Usability Engineering Reqs Analysis Design Evaluate Develop many iterations

  5. Usability Engineering Formative evaluation Summative evaluation

  6. User Interface Metrics • Ease of learning • learning time, … • Ease of use • perf time, error rates… • User satisfaction • surveys… Not “user friendly”

  7. Claims Analysis • + advantages • - disadvantages • E.g. file management: + +

  8. Claims Analysis • + advantages • - disadvantages • E.g. file management: • + fast learning • + spatial learning aid memory • switching between devices • long to locate a known file… • + flexible, power • + fast for experts • Learning commands • aesthetics lacking?...

  9. Usability Evaluation • Analytic Methods: • Heuristic Evaluation • Models: GOMS, Fitts Law • Empirical Methods: • Usability Testing • Wizard of Oz • Controlled Experiment

  10. Nielsen’s 10 Heuristics • Visible status, feedback • User control, undo, exits • Familiar, speak user’s language • Consistent, standards • Recognition over recall • Efficient, expert shortcuts • Aesthetic, minimalist • Prevent errors • Error recovery • Help, task-based

  11. Nielsen’s 10 Heuristics • Visible status, feedback wysiwyg • User control, undo, exits wizards • Familiar, speak user’s language Acrobat error msg • Consistent, standards Word, PPT • Recognition over recall web nav, phone menu • Efficient, expert shortcuts Word bold • Aesthetic, minimalist phone book • Prevent errors HomeFinder • Error recovery undo, back • Help, task-based IIS help doc

  12. Speak the User’s Language

  13. Help documentation Context help, help doc, UI

  14. Applying Heuristics • Lets look at a website…

  15. Early Usability Testing • Formative: helps guide design • Early in design process • when architecture is finalized, then its too late! • A few users (3-5) • Identify usability problems, incidents • Qualitative feedback from users • Quantitative usability measures

  16. Usability Test Procedure • Wizard of Oz: act out the system • Benchmark task set • No cheating! Don’t teach them how to use the UI • Repeat: • Give user a task • Ask user to “think aloud” • Observe, note mistakes and problems • Avoid interfering, hint only if completely stuck • Interview • Verbal feedback • Questionnaire • 30 min - 1 hour / user

  17. Analyze Results • Initial reaction: • “stupid user!”, “that’s developer X’s fault!”, “this sucks” • Mature reaction: • “how can we redesign UI to solve that usability problem?” • the user is always ‘right’ • Identify usability problems • Learning issues: e.g. can’t figure out or didn’t notice feature • Performance issues: e.g. arduous, tiring to solve tasks • Subjective issues: e.g. annoying, ugly • Problem severity: critical vs. minor • Redesign

  18. Pizza Ordering System • Scenario: • Volunteers! • Design ideas • Wizard of Oz usability test • Redesign

  19. In action: Pizza Ordering System • Scenario: VT students, studying/busy/tired, poor, wants discounts, starving, credit card / hokie card/ no cash, 2 in together, half-n-half, split cost • Volunteers! • Design ideas • Wizard of Oz usability test • Redesign

  20. Pizza R Us Drag your toppings: Whole Half-n-half peperoni mushrooms pineapple checkout PRICE: $10.00

  21. Size of pizza: • large • Small • First half: • Toppings: • Peperoni • Mushrooms • Pineapple • Price: 0.00 • 2nd half: • Toppings: • Peperoni • Mushrooms • pineapple

  22. % pza –ppm –large -…

More Related