1 / 62

Interfaces for Augmenting Face-to-Face Conversation

This study explores the use of miniature head-up displays and augmented reality to enhance face-to-face conversation. It investigates the integration of energy-harvesting devices, user modeling, context recognition, and everyday applications such as calendaring. The experiment aims to address common problems faced during spoken conversations in various settings, including classrooms, technical meetings, and opportunistic communication.

michalak
Download Presentation

Interfaces for Augmenting Face-to-Face Conversation

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Interfaces for Augmenting Face-to-Face Conversation Professor Thad Starner Contextual Computing Group GVU Center, College of Computing Georgia Institute of Technology

  2. A living experiment since 1993

  3. Miniature Head-up Displays

  4. Previous Work Energy harvesting Augmented reality Disk-based MP3 players; Wireless IM User modeling Context recognition

  5. Everyday Use • Originally developed for • Classroom notes • Hallway conversations • Expert use study [MobileHCI2003] • Augmenting memory • Face-to-face conversation • Is this a common problem?

  6. Spoken conversation • During work time • 35-80% in spoken conversation • 7-82% in technical meetings • 14-93% in opportunistic communication • Managers are most likely to be at this high end [Whittaker94, Whittaker95] • PDA to support conversation?

  7. Quick PDA Survey • How many people • Have owned a PDA at some point?

  8. Quick PDA Survey • How many people • Have owned a PDA at some point? • Have a PDA with you right now?

  9. Quick PDA Survey • How many people • Have owned a PDA at some point? • Have a PDA with you right now? • Typical results for technical audience • 50% owned a PDA • 15-25% carry it to talk (1/3-1/2 of PDA users)

  10. Application: Calendaring • One of the most common PDA applications • One of the most desired functions • Occurs routinely in social conversation • One-on-one • Conferences • Meetings • Anecdotal observation of dissatisfaction

  11. Scheduling Device Survey[CHI2004] • 138 subjects • Georgia Tech student center • 90% students; 88% age 18-25; 70% male • Survey • What is your primary scheduling system while mobile? • 8 Likert scale questions on effectiveness, ease of use, speed, and reliability • Open response questions • Schedule four appointments

  12. Appointment Tasks • “Could we meet sometime next Monday?” • “Could we schedule a time to meet in the second week of February?” (three months in future) • “Could we schedule a time to meet tomorrow?” • “Could we reschedule our appointment in February from the 10th to the 11th?”

  13. Videotaped Interactions Subject view Scheduling device Timed retrieval, navigation, and entry

  14. Actual vs. Claimed Usage

  15. Actual vs. Claimed Usage

  16. Actual vs. Claimed Usage

  17. Actual vs. Claimed Usage

  18. Abandonment rates • 43% of PDA users switched • 68% of planner users switched • Memory and scrap paper dominated • Hypothesis: Users switch to mechanisms that are faster to access (similar to Miller68?)

  19. Timing (in seconds)

  20. Timing (in seconds)

  21. Timing (in seconds)

  22. Timing (in seconds)

  23. Timing (in seconds)

  24. Timing (in seconds)

  25. Actual vs. Claimed Usage

  26. Calendar Study Summary • Overall usage speed similar • Access (retrieval+navigation) is different • Users tend to switch to faster access systems • Can we make faster access on-body interfaces?

  27. Dual Purpose Speech[UIST2004;UIST2005] • Design everyday-use agents that • Minimize time to retrieve and navigate interface • Minimize cognitive load • Use the user’s social dialog to cue the agent • Phone number example • Calendaring?

  28. DPS: Calendar Navigator Agent

  29. “Can I see you next week sometime?”

  30. “Let me see if I’m free on the 24th”

  31. “Let me see if I’m free on the 31st” “Yes, 3pm seems like a good time”

  32. “OK, I’ll put “meet Maribeth” at 3pm in my calendar”

  33. Experimental Results • Expert use (Wizard of Oz) • 47% faster than PDAs • 20% faster than paper calendars • Novice use (Wizard of Oz) • Quickly adapt • Faster than PDA • Errors • 82% correct action first attempt • 98% upon second repetition

  34. DPS: Dialog Tabs • Capture conversation for later processing • Low retrieval time and cognitive load • Low impact of speech recognition errors • Enable batch processing • Unable to identify scheduling conflicts directly • Always-visible feedback • Unintrusive during conversation • Continuous reminder of cached information • Quick access and search for processing

  35. DPS: Dialog Tabs

  36. DPS: Dialog Tabs

  37. DPS: Speech Courier • Semi-automatically send e-mail/voice-mail to a remote assistant • “Let me ask my assistant Cynthia to …” • “I should make sure to ask my grad student Kent to …”

  38. Notetaking During Conversation • Speech is often socially inappropriate • Extensive notetaking by expert wearable users but not other device users • Why? • Keyboard? • Head-up display? • Users?

  39. g b c d e f m o h i j k y n z p w x q v l t s r u a Twiddler • Mobile, one-handed • 3 x 4 grid of buttons • Top speed? • Learning?

  40. Mobile Text Entry • 1.3 billion mobile phone users • 500 million phones sold 2003 • $16 billion wireless messaging in 2003 • 1 trillion text messages per year Current mobile text entry methods very slow!

  41. Twiddler vs Mobile Phone Sony Ericsson T610 Twiddler

  42. Current Methods WPM Experience Keyboard LetterWise 21.0 550min desktop keypad T9 20.36 expert Nokia 3210 Multi-tap 15.5 550min desktop keypad TiltText 13.57 165min Motorola i95cl Multi-tap 11.04 165min Motorola i95cl T9 9.09 novice Nokia 3210 Multi-tap 7.98 novice Nokia 3210 Multi-tap 7.93 expert Nokia 3210 Multi-tap 7.2 n/a desktop keypad

  43. 67 Twiddler vs. Multitap [CHI2004;ISWC2004] • Longitudinal study • 2 x 20 factorial mixed design • Within subject • Twenty 20-minute sessions • $1*WPM*Accuracy per session ($4 minimum) • Extended to determine expert rates • MacKenzie and Soukoreff phrase set

  44. 67 Twiddler vs. Multitap 67 47

  45. Twiddler Summary • Desktop typing in a mobile phone keypad • Expert rates <25 hours (50wpm) • Typing class 68 hours (40wpm) • Touch typing in first session (20 minutes) • 26wpm in 400 minutes

  46. Cell Phone Prototype

  47. Mini-QWERTY? • Evidence of success for short e-mail • Practical for face-to-face notetaking? • Speed? • Touch typing? • Learning?

  48. Mini-QWERTY? [CHI2005]

  49. Mini-QWERTY? • Capable of desktop speed and fast learning • 30wpm initial (expert desktop typists) • 60wpm in 400 minutes • Touch typing possible (!) • Why not used in class? In conversation? • Two thumbs are too strenuous? • Socially inappropriate (currently)? • Head-up display?

  50. Enabling Conversation: ASL->English One Way Translator • Interpreters cost $80/hour • Inconvenient to schedule; not spontaneous • English is a second language for the Deaf • Domains: • Doctor’s office (privacy) • Lawyer’s office (privacy) • Airport directions (convenience) • Car accident (convenience) • Apartment hunting

More Related