Usability & Evaluation in Visualizing Biological Data

1. Usability & Evaluationin Visualizing Biological Data Chris North, Virginia Tech VizBi

2. Usomics & Evaluationin Visualizing Biological Data Chris North, Virginia Tech VizBi

3. Myths about Usability • Usability = Voodoo

4. Science of Usability Phenomenon Engineering Measurement Science Modeling …analogy to biology

5. 1. Analyze Requirements 4. Evaluate 2. Design 3. Develop Usability Engineering • User-centric • Iterative • Engineering = process to ensure usability goals are met

6. Myths about Usability • Usability = Voodoo • Usability = Learnability

8. Myths about Usability • Usability = Voodoo • Usability = Learnability • Usability = Simple task performance

9. Impact on Cognition Insight gained: Spotfire GeneSpring

10. Myths about Usability • Usability = Voodoo • Usability = Learnability • Usability = Simple task performance • Usability = Expensive http://www.upassoc.org/usability_resources/usability_in_the_real_world/roi_of_usability.html

11. 1. Analyze Requirements 4. Evaluate 2. Design 3. Develop Usability Engineering

12. Requirements Analysis • Goal = understand the user & tasks • Methods: Ethnographic observation, interviews, cognitive task analysis • Challenge: Find the hidden problem behind the apparent problem

14. Analysts’ Process Pirolli & Card, PARC

15. Systems Biology Analysis • Beyond read-offs -> Model-based reasoning Mirel, U. Michigan

16. 1. Analyze Requirements 4. Evaluate 2. Design 3. Develop Usability Engineering

17. Why Emphasize Evaluation? • Many useful guidelines, but… • Quantity of evidence • Exploit domain knowledge Hunter, Tipney, UC-Denver

18. Science of Usability Phenomenon Measurement Modeling

19. Measuring Usability in Visualization Phenomena goal,problemsolving inference,insight perception,interaction system,algorithm visual • frame-rate • capacity • … • realism • data/ink • … • task time • accuracy • … • ? • market • ? 2 kinds of holes Measurements

20. Time & Accuracy • Controlled Experiments • Benchmark tasks

21. Results

22. + Consistent overall + Fast for single node analysis - Slow and inaccurate for expression across graph + Accurate for comparing timepoints p<0.05

23. Cerebral Munzner, UBC

24. Insight-based Evaluation • Problem: Current measurements focus on low-level task performance and accuracy • What about Insight? • Idea: Treat tasks as dependent variable • What do users learn from this Visualization? • Realistic scenario, open-ended, think aloud • Insight coding • Information-rich results

25. Insight? GeneSpring Spotfire Gene expression visualizations HCE Cluster/Treeview TimeSearcher

26. Cluster- Time- Gene- ViewSearcher HCESpotfireSpring Results Count of insights Total value of insights Average time to first insight(minutes)

27. Insight Summary

28. Users’ Estimation Cluster- Time- Gene- ViewSearcher HCESpotfireSpring Total value of insights Users’ estimated insight percentage

29. Insight Methodology • Difficulties: • Labor intensive • Requires domain expert • Requires motivated subjects • Short training and trial time • Opportunities: • Self reporting data capture • Insight trails over long-term usage – Insight Provenance

30. Trend towards Longitudinal Evaluation • Multidimensional in-depth long-term case studies (MILCS) • Qualitative, ethnographic • GRID: Study graphics, find features, ranking guides insight, statistics confirm • But: Not replicable, Not comparative Shneiderman, U. Maryland

31. Onward… • VAST Challenge • Analytic dataset with ground truth • E.g. Goerg, Stasko – JigSaw study • BELIV Workshop – BEyond time and errors: novel evaLuation methods for Information Visualization

32. Visual Analytics

33. Embodied Interaction 1) Cognition is situated. 2) Cognition is time-pressured. 3) We off-load cognitive work onto the environment. 4) The environment is part of the cognitive system. 5) Cognition is for action. 6) Off-line cognition is body-based. -- Margaret Wilson, UCSC GigaPixel Display Lab, Virginia Tech Carpendale, U. Calgary