1 / 9

Context-aware / Multimodal UI Breakout Summary

Context-aware / Multimodal UI Breakout Summary. James A. Landay, et. al. HCC Retreat July 7, 2000. James Landay Anoop Sinha Jimmy Lin Trevor Perring Greg Heinzinger Chris Long Ed Chi. Christine Halverson Gian Gonzaga Ken Fishkin John Lowe Adam Janin Russell Eames Elin Pedersen.

philip-mccarthy
Download Presentation

Context-aware / Multimodal UI Breakout Summary

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Context-aware / Multimodal UI Breakout Summary James A. Landay, et. al. HCC Retreat July 7, 2000

  2. James Landay Anoop Sinha Jimmy Lin Trevor Perring Greg Heinzinger Chris Long Ed Chi Christine Halverson Gian Gonzaga Ken Fishkin John Lowe Adam Janin Russell Eames Elin Pedersen Participants

  3. Applications • Alert management • sites beacon context • “this is a quiet place, no interruptions please” • e.g., movie theater or restaurant • devices use context to avoid interruptions • wearable t-shirt that jams local cell phones! • “Elvis has left the meeting” • easily share documents from meetings • beam tokens of documents to participants or • use shared context to find docs later • e.g., I was in a meeting with Ken at Lake Tahoe, find docs

  4. Context Events • Signal changes • like a windowing system • Can use as triggers to cause other actions • change my phone forwarding when I change locations • Can be immediate or logged for later tacit information mining

  5. Context Implementation Issues • Apps need to share context easily • build-in to apps like cut & paste • context dial tone or infrastructure • Global file system • easier to share context & not have to transfer it • just use pointers • How to search / browse • computers are good at searching large spaces • humans good at making associations • Why not search with Google instead of browser history? • Google easier to get at & seems to work well

  6. Context Toolkits / APIs / Refs • Bill Schilit’s Columbia / PARC Ph.D. • GA Tech GVU (Anind Dey) • IBM (Maria Ebling) • MIT (?) • ESPIRIT projects have looked at context • German? project according to Elin Pedersen

  7. Interface Between Context & Multimodal UIs • “Context is just another kind of input” • different to the user, but similar to the system • user input caused by EXPLICIT user action • context is IMPLICT or DERRIVED • Context and multimodal UIs have similar privacy problems • natural inputs are human “readable” • may not want to share context or my input

  8. Interface Between Context & Multimodal UIs • Context to choose output modality • e.g., user is in a meeting, don’t use speech • Context to disambiguate input(s) • help fusion: there is noise, don’t rely on speech • “the clutching problem” – infer user intent • Modality used to help inter context • e.g., talking to device -> user is alone?

  9. Model Initial Design for Multimodal UI Design Tool • Create “rough cuts” • informal (sketching / “Wizard of Oz”) • iterative design (user testing/fast mods) • Infer models from design • designer can augment model over time • Generate initial prototypes • UIs for multiple devices • designer adds detail / improve UI • or even removes detail

More Related