Field studies

1. Field studies

2. User studies • Ubicomp: people use technology • Must conduct user studies • Also: • Focus groups • Ethnographic studies • Heuristic evaulations • Etc.

3. User studies • Laboratory studies: • Controlled environment • Field (in-situ) studies • Real world

4. Field studies • Appropriate for ubicomp: • Abundant data • Observe unexpected challenges • Understand impact on lives • Trade-off: • Loss of control • Significant time and effort

5. Three common types • Current behavior • Proof of concept • Experience with prototype

6. How to think about user studies? • Formulate hypotheses

7. Research steps • State problem(s) • State goal(s) • Propose hypotheses • Propose steps to test hypotheses • Explain how problem(s), goal(s) and hypotheses fit into existing knowledge • Produce results of testing hypotheses • Explain results • Evaluate research • State new problems

8. What is a hypothesis? • Proposing an explanation • Theory or hypothesis? • “This is just a theory.” • Some theories we live by (“just” not justified): • Newton’s theory of motion • Einstein’s theory of relativity • Evolutionary theory

9. Hypothesis • Must be tentative • Must predict

10. Hypothesis • Some criteria of scientificity • Self-consistent • Grounded (fits bulk of relevant knowledge) • Accounts for empirical evidence • Empirically testable by objective procedures of science • General in some respect and to some extent

11. On proposing hypotheses • Anomalous phenomena: • Strange and unfamiliar (Bermuda triangle) • Familiar yet not fully understood (cognitive load) • Is there already an explanation?

12. Types of hypotheses • Incremental • Fundamental shift: • Ptolemy (c. 90 – c. 168): geocentric cosmology • Copernicus (1473 – 1543): heliocentric cosmology

13. And then came… • Kepler (1571 – 1630): elliptical orbits

14. Fundamental shift example • Ulcer: • Stress? Spicy food? • Bacteria.

15. Types of proposed explanations • Causes • Correlation • Causal mechanisms • Underlying processes • Laws • Functions

16. Proposing causal explanations • Studies show that using a cell phone while driving increases the probability of getting into an accident. Why is that so? • Pick up ringing phone • Dial number • See but don’t perceive

17. Effects not always there • Cell phone + driving: • Usually no accident • Only one of the factors

18. Remote and proximate causes • Cell phone + driving: • Attention shift → missed signal → accident • Remote cause → proximate cause → effect

19. Correlation • A and B are correlated if: • A → B • B → A • C → A and C → B • A combination of (some of) the above • Coincidence • Correlation vs. causal relation: • Correlation doesn’t imply causal relation • Cannot determine cause direction (A → B or B → A)

20. Correlation • Positive, negative • None found ≠ none exists • Causal link → correlation: • May provide initial evidence for causal link • Less explanatory value than facts about causal links

21. Causal mechanisms • Mechanisms connecting remote causes and their effects. • E.g.: • Damaged artery in heart → clotting • Clotting → blocked artery • Blocked artery → heart attack • Aspirin inhibits clotting → lower risk of heart attack

22. Underlying processes • Photoelectric effect

23. Photoelectric effect • Einstein: 1921 Nobel Prize in Physics

24. Laws • General regularities in nature • Universal: • F = ma • Non-universal: • Statistical laws

25. Functions • What is the purpose of the phenomenon? FOR SALE A prime lot of serfs or SLAVES GYPSY (TZIGANY) Through an auction at noon at the St. Elias Monastery on 8 May 1852 consisting of 18 Men 10 Boys, 7 Women & 3 Girls in fine condition

26. Functions • William Harvey (1578 – 1657): • Heart pumps blood through circulatory system • No modern instruments! • Experiments with a numberof animals: • Various fish, • Snail, • Pigeon, etc.

27. Multiple methods together • Function → → causal mechanism → → underlying processes • National Ignition Facility(Dennis O’Brien @ UNH):Ignition with lasers → → Laser, target chamber → → Physics of nuclear fusion

28. Multiple methods together • Law → underlying processes • Isaac Newton (1643 – 1727), second law of motion:F = ma → Graviton?

29. Ockham’s razor • Crop circles: pranksters or aliens?

30. Ockham’s razor • William of Ockham (c. 1288 – c. 1348) http://en.wikipedia.org/wiki/File:William_of_Ockham.png

31. Do I have a hypothesis? • Yes. • Do you realize you do?

32. How to think about user studies? • Formulate hypotheses

33. Three common types • Current behavior • Proof of concept • Experience with prototype

34. Research steps • State problem(s) • State goal(s) • Propose hypotheses • Propose steps to test hypotheses • Explain how problem(s), goal(s) and hypotheses fit into existing knowledge • Produce results of testing hypotheses • Explain results • Evaluate research • State new problems

35. Current behavior • Insights and inspiration: • State problem(s), goal(s) • Propose hypotheses • Relatively long

36. Current behavior – example 1 • AJ Brush and KoriInkpen, “Yours, mine and ours?...” (pdf) (2005 movie inspiring title) • Home technology: users share, etc.

37. Current behavior – example 2 • Schwetak Patel et al. “Farther Than You May Think…” (pdf) • Hypothesis: Mobile phone a proxy to user location.

38. Three common types • Current behavior • Proof of concept • Experience with prototype

39. Research steps • State problem(s) • State goal(s) • Propose hypotheses • Propose steps to test hypotheses • Explain how problem(s), goal(s) and hypotheses fit into existing knowledge • Produce results of testing hypotheses • Explain results • Evaluate research • State new problems

40. Proof of concept • Technological advance: • Produce results: prototype • Explain results: prototype • Relatively short

41. Proof of concept – example 1 • J. Sherwani et al., “Speech vs. Touch-tone: Telephone Interfaces for Information Access by Low Literate Users” (pdf) (video) • Hypothesis: Speech better telephony interface than touch-tone for low literate users.

42. Proof of concept – example 2 • John Krumm and Eric Horvitz, “Predestination:…” (pdf) • Hypothesis: Destinations from partial trajectories. • Train/test algorithm on GPS tracks from 169 people • Used pre-existing data: • Krumm and Horvitz, “The Microsoft Multiperson Location Survey” • Collecting original data a significant contribution • Leverage!

43. Three common types • Current behavior • Proof of concept • Experience with prototype

44. Research steps • State problem(s) • State goal(s) • Propose hypotheses • Propose steps to test hypotheses • Explain how problem(s), goal(s) and hypotheses fit into existing knowledge • Produce results of testing hypotheses • Explain results • Evaluate research • State new problems

45. Experience with prototype • Users’ interaction with technology: • Produce results: prototype • Explain results: prototype • Relatively long

46. Prototype an example! • Others don’t care about: • Raw usage information • Usability problems • Intricate implementation details • Etc. • Generalize! • Scientific and good technical work

47. Experience – example 1 • C. Neustaedter, et al., “A Digital Family Calendar in the Home:…” (pdf) (video) • Hypothesis: At-a-glance awareness, remote access are significant benefits. • 4 households, 4 weeks each • (Best Student Paper, Graphics Interface 2007)

48. Experience – example 2 • Rafael Ballagas et al., “Gaming Tourism:…” (pdf) (video) • Hypothesis: Learning through a game. • 18 participants: 2 alone + 8 pairs (8 x 2 = 16)

49. Study design • Who is the consumer? • Manager(s) • Industry, academic lab • Professor(s) • E.g. thesis committee • Researchers • E.g. advisor’s collaborators • Reviewers • For paper, proposal, thesis • Funding agency • Report on progress, proposal for funding • Public • Friends, family, alumni, potential students, donors, potential employers

50. Study design • How can I explain this to a layperson? • What is key? What can be omitted? • How will I write this up? • Paper • Thesis • Report • Blog post • Start writing paper/thesis/report/blog post at the beginning of the study.