1 / 29

HCI Validation

HCI Validation. Richard F. Riesenfeld University of Utah Fall 2004. Major Considerations - 1. Stage of design Conceptual, preliminary, detail Novelty of project Do we know what we are doing? Number of expected users How important is this? How amenable to change will it be?.

welter
Download Presentation

HCI Validation

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. HCI Validation Richard F. Riesenfeld University of Utah Fall 2004

  2. Major Considerations - 1 • Stage of design • Conceptual, preliminary, detail • Novelty of project • Do we know what we are doing? • Number of expected users • How important is this? • How amenable to change will it be?

  3. Major Considerations - 2 • Criticality of the interface • Are lives at stake if there are problems? • Cost of product • Allocation for testing • Time available for testing • Experience of designers and evaluators

  4. Expert Review methods - 1 • Heuristic evaluation • Experts critique it wrt established criteria • Guidelines review • Does it meet “spec” • Can be an overwhelming list • Bureaucratic approach

  5. Expert Review methods - 2 • Consistency inspection • Experts check of style, function, form, etc. • Cognitive walkthrough • Experts perform role of users • Try to assess its success from experience

  6. Expert Review methods - 3 • Formal usability inspection • Moot court • Countervailing opinions • Can be unwieldy

  7. Comparative Evaluations - 1 • Different experts see different issues • Can get caught with conflicting advice* • Limit the number of experts • Get a “bird’s-eye” view in the beginning • Throw images on a wall, etc. * “For every PhD, there is an equal and opposite PhD” * “

  8. Comparative Evaluations - 2 • Formal (statistical) methods • Form a hypothesis • Determine dependent variables • Identify independent variables * “

  9. Usability Labs, etc • Hard to employ because of time and money constraints in product development • Development cycle schedule • Budgets • Corporate/Cultural Attitude

  10. Controlled Experients • Statistical testing • Establish level of statistical significance • At the 95% confidence level we know… • Usability testing • Find flaws in the interface through more informal (inconclusive) methods • Empirical methods

  11. Human Subjects -1 • Careful, “This isn’t Kansas anymore!” • Many new dimensions need attention • Human respect and dignity • Voice generated check outs violated privacy • Military has NO privacy • Other cultures treat matters differently

  12. Human Subjects -2 • Real LEGAL issues, so get it right! • Informed consent • Understand your liability • Get it in writing, copy to each party • Government, or institutional rules • We are not accustomed to this • Need cognizant approvals • IRBs, etc. • Research proposals, etc

  13. Observations methods • Have subjects “think aloud” • Will subjects be honest, etc. • Use video recording • Field tests • Study the successes/failures of the interface • Getting access • Reliance on memories • “How is it going?” (We tend to react to most recent)

  14. Destructive Testing • Hey, can you break this? • Good for security • Good for games • Durability testing appropriate for some environments • ATM in high crime area • Military • Students, they can’t resist a challenge

  15. Competitive Testing -1 • Consumers Union, Road & Track, style • Take several into lab and have a “shoot out” • Expensive • Takes skill (like a movie review) • Depends on the criteria • Depends on good and representative judgment

  16. Competitive Testing -2 Major Limitations • Limited coverage of features • Depends on initial user experiences

  17. Surveys • Tricky business, can lead to nearly any conclusion • Population selection • Question choices • Size • Leading questions, other bias • Negative bias – users with complaints

  18. Online Surveys • More issues…

  19. Conclusions • HCI is a new game • Not exact science • Old methods not entirely applicable • Need newer, faster, light weight, flexible, informal, subjective, intelligent approaches

  20. Recommendations • Use good judgment • Trust good judgment • Yours • Others, whom you trust • Be open to criticism and suggestion

  21. Interpretation • What is being said? • What is the real issue? • What is the real fix? • RSI is a problem • Pain • Keyboard or mouse? • Need different devices, or speech, or simply a better mouse and keyboard?

  22. acm.org/~perlman/question.cgi • Please rate the usability of the system. • Try to respond to all the items. • For items that are not applicable, use: NA • Make sure these fields are filled in: System:Email to: • Add a comment about an item by clicking on its   icon, • or add comment fields for all items by clicking on Comment All. • List the most negative aspect(s): • List the most positive aspect(s): • Top of Form

  23. acm.org/~perlman/question.cgi • To mail in your results, click on: Mail Data • System:   Email to: • Optionally provide comments and your email address in the box. • List the most negative aspect(s): • List the most positive aspect(s): • Top of Form

  24. acm.org/~perlman/question.cgi • List the most negative aspect(s): • List the most positive aspect(s): • Top of Form

  25. RHS of Each Row

  26. acm.org/~perlman/question.cgi

  27. acm.org/~perlman/question.cgi • List the most negative aspect(s): • List the most positive aspect(s): • Top of Form

  28. acm.org/~perlman/question.cgi • List the most negative aspect(s): • List the most positive aspect(s): • Top of Form • List the most negative aspect(s): • List the most positive aspect(s): • Top of Form

  29. END HCI Validation

More Related