1 / 15

Evaluation in Experience-Focused HCI: Exploring the Virtual Intimate Object

Evaluation in Experience-Focused HCI: Exploring the Virtual Intimate Object. Future Applications Lab Viktoria Institute Gothenburg, Sweden 13 January 2006 Joseph ‘Jofish’ Kaye Culturally Embedded Computing Cornell Information Science jofish@cornell.edu.

zaza
Download Presentation

Evaluation in Experience-Focused HCI: Exploring the Virtual Intimate Object

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Evaluation in Experience-Focused HCI: Exploring the Virtual Intimate Object Future Applications Lab Viktoria Institute Gothenburg, Sweden 13 January 2006 Joseph ‘Jofish’ Kaye Culturally Embedded Computing Cornell Information Science jofish@cornell.edu

  2. Evaluation in Experience-Focused HCI Evaluation in experience-focused HCI is the poor cousin of design. As a field (if we are one!) we have excellent theoretical foundations (Dourish, Wright & McCarthy), a wide repertoire of design techniques (Equator Project, IDEO) and extensive tools for building (CS, EE, PD, ID). But traditional HCI evaluation techniques (GOMS, etc) are inadequate for ready-at-hand technologies used in ways unreplicable in the lab.

  3. Virtual Intimate Object (VIO) • A small red circle in Windows’ taskbar, or in a Mac window. • When circle is clicked, partner’s circle glows bright red, then fades over time. http://io.infosci.cornell.edu • Kaye, Levitt, Nevins, Golden & Schmidt. Communicating Intimacy One Bit at a Time. Ext. Abs. CHI 2005, ACM Press. • Kaye. I just clicked to say I love you. alt.chi, Ext. Abs. CHI 2006, ACM Press.

  4. Intimacy in HCI History of elegantly designed but ad-hoc interfaces for communicating intimacy in HCI; hard to evaluate • Fields & Thresholds (Dunne & Raby DoP2 '94) • Feather, Scent & Shaker (Strong & Gaver CSCW'96) • The Bed (Dodge CHI'97) • UbiComp'03 Workshop on Intimacy www.intimateornot.org • Digital Family Portrait (Mynatt et. al. CHI'01) • inTouch (Brave, Ishii, Dahley CSCW'98) • inStink, Honey I’m Home (Kaye '02, interactions '04) • ……

  5. How much does Ariel love Eric? Count the hearts! Disney (1997) The Special Edition Little Mermaid Coloring Book. Golden Books Publishing. Answer: 19

  6. Two VIO Studies • Study I • 10 couples in existing long-distance relationships (n=20) • 5 couples assigned to VIO (n=10) • 5 couples assigned to MinIO (n=10) • Paper logbooks; pre, post-tests, daily questions • Study II • 80+ subjects recruited through long distance relationship communities on LiveJournal • All subjects took pre-survey; all were asked to take post-survey • Four groups: VIO, VIO + daily survey, daily survey alone, nothing • All surveys online - Currently being analyzed

  7. Experience surveys: starting points • Gaver et. al: Cultural Probes • Geertz: Thick description • Dourish, Wright & McCarthy, etc: experience-based HCI • Iterative design: Design, Build, Evaluate..

  8. Thick Description • Winking isn’t just something your eye does • It’s culturally embedded • It’s only the thick description of the context and culture that lets us understand the role of the wink in sharing a conspiracy, or even parodying another sharing conspiracy. • Geertz, C. (1973) Interpretation of Cultures Ch. 1

  9. Cultural Probes • Gives context around a situation • Originally for inspiring the design part of the design  build  evaluate  iterative design cycle • Repurposed here for inspiring the evaluation part of the cycle • <provocative statement>We’re changing a key, generally unspoken assumption by proposing that evaluation is creative, arbitrary, and complex.</provocative> Gaver et. al. Cultural Probes, interactions 6(1) 1999

  10. Three topics • Questions/tasks about the technology being evaluated. • Questions/tasks about the situation. • Questions/tasks about instrument being used to gather information

  11. And three kinds of questions(for organization, not canonical) • Context: Where? When? Take a photo of… How long? How often? • Metaphor: What TV show? What song? What colour? What metaphor? • Valence: What’s the best? What’s the worse? What’s your favorite? What should we change? What shouldn’t we change? Draw the best, the worst.

  12. Question Design chart

  13. Getting beyond usability • Evaluating in context requires working in context • VIO becomes ready-at-hand, not present-at-hand, and so evaluation must be in context. • Users must have significant, continuous use, and you must have solved the usability problems otherwise most of what you get is usability issues – i.e. Mobile Probes paper Hulkko et. al. Mobile Probes Proc. NordCHI ‘04

  14. Discussion • Unpacking meanings of ‘evaluate’ • …tell me if people like it • …tell me if a variable that I can track has changed • …make a comparison between my stated (unstated?) goals/values/principles and the built technology • …to tell me what to do next • When? (Formative, iterative, final, discrete, continuous)

  15. Thanks to Maria Håkansson, Lars Erik Holmquist, Phoebe Sengers, the Culturally Embedded Computing Group, my subjects and my coauthors. Joseph ‘Jofish’ Kaye Culturally Embedded Computing Cornell Information Science jofish@cornell.edu This talk at jofish.com/talks. VIO at io.infosci.cornell.edu

More Related