1 / 27

Generating Remote Control Interfaces for Complex Appliances

Generating Remote Control Interfaces for Complex Appliances. Jeffrey Nichols and Brad A. Myers Carnegie Mellon University October 30, 2002. The Fifteenth Annual Symposium on User Interface Software & Technology (UIST) 2002 Paris, France. The Problem. Appliances are too complex.

doane
Download Presentation

Generating Remote Control Interfaces for Complex Appliances

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Generating Remote Control Interfaces for Complex Appliances Jeffrey Nichols and Brad A. Myers Carnegie Mellon University October 30, 2002 The Fifteenth Annual Symposium on User Interface Software & Technology (UIST) 2002 Paris, France

  2. The Problem Appliances are too complex

  3. The Problem, cont. Each complex appliances has its own idiosyncratic interface! • Stereo systems • Telephones • VCRs • Alarm Clocks • … Increasingly Computerized Low Usability

  4. Feedback Specifications Control Our Solution Separate the interface from the appliance! Handheld becomes personal universal controller (PUC) Key Features • Interface-independent appliance specification • Automatic generation of GUI and speech interfaces

  5. Automatic Generation of UIs Benefits • All interfaces consistent for the user • With conventions of handheld • Even from multiple manufacturers Addresses hotel alarm clock problem! • Multiple modalities (GUI + Speech UI) • Can take into account user preferences • Will work on special purpose devices (for disabled)

  6. Important Work By Others • INCITS V2 Standardization Effort Alternative Interface Access Protocol (AIAP) [Zimmermann, CHI 2002] • User Interface Modeling Language (UIML) http:/www.uiml.org/ • Xweb (now ICE) [Olsen Jr., UIST 2000] • Stanford iRoom, iCrafter [Ponnekanti, Ubicomp 2001] • Speakeasy [Newman, UIST 2002]

  7. Architecture - Comm. Protocol - Interface Generators - Specification Lang. - Appliance Adaptors XML-based

  8. Language Design Approach • Create reference interfaces • AIWA Shelf Stereo • AT&T Telephone/Answering Machine • Test interfaces with subjects • Users twice as fast and made half the errors with reference interfaces as compared to manufacturers’ interfaces • Analyze interfaces for functional information

  9. Language Elements State Variables and Commands • Represent functions of appliance • State variables have types • Boolean, Enumeration, Integer, String, etc. • Variables sufficient for most functions but not all • “seek” button on a Radio Label Information • One label not suitable everywhere • The optimal label length changes with screen size • Speech interfaces may benefit from pronunciation and text-to-speech information

  10. Language Elements, cont. Group Tree • Specify organization of functions • We use n-ary tree with variables or commands at leaves

  11. Language Elements, cont. Dependency Information • Formulas that specify when a variable or command is active in terms of other state variables • Equals, Greater Than, Less Than • Linked with logical operators (AND, OR) • For example, <and> <equals state=“PowerState”>true</equals> <equals state=“RadioBand”>AM</equals></and>

  12. Interface Generators Generators for Two Modalities • Graphical • Implemented for PocketPC in Java 1.1 • Uses dependency information to generate panel structure of interface • Speech • Implemented using Universal Speech Interface (USI) techniques [Rosenfeld 2001] • Uses dependency information to disambiguate shortcut words (e.g. “play”) and resolve pre-conditions for a requested function (e.g. “play CD”)

  13. Graphical Interface Generator Focuses on panel structure of user interface • Small groups of controls have basic layouts • Complexity comes from structure of groups • Structure can be inferred from dependency info!

  14. Inferring Structure Find sets of variables that are “mutually exclusive” • Every variable in a set will never be active at the same time as a variable in another set Create structure with sets, using overlapping panels

  15. Choosing Panel Types a) b) c) partial screen full screen tabbed

  16. Making the Interface Concrete Finish conceptual layout • Choose controls (decision tree) • Choose row layouts (one column, two column, etc.) Allocate space • Examine panel contents and choose sizes Instantiate and place controls

  17. Generating Speech Interfaces Automatically build USI tree from dependencies • Allows verbal navigation of functional groups Automatically generate grammar for parser • Phrases for query and control “What is playmode?” “Set playmode to play” “play” Automatically generate language model and pronunciation for recognizer

  18. Controlling Appliances We have built adaptors for many actual appliances • Sony Digital Camcorder • X10 Lighting • Audiophase Shelf Stereo • AudioReQuest MP3 player • Software Media Players (WinAmp, WMP) Written specifications for others • Elevator • Telephone/Answering Machine

  19. Demonstration

  20. Future Work • Improve quality of generated user interfaces • Back-tracking, etc. • Specification and generation of list interfaces • Necessary for many appliances • Adaptive interface generation • New interfaces modeled on past interfaces • General purpose appliance adaptor for HAVi and UPnP protocols

  21. PUC Project Members Brad A. Myers Michael Higgins Joseph Hughes Thomas K. Harris Roni Rosenfeld Mathilde Pignol Stefanie Shriver Peter Lucas Kevin Litwack Funding National Science Foundation Pittsburgh Digital Greenhouse Microsoft Equipment Grants Mitsubishi (MERL) VividLogic Symbol Technologies Hewlett-Packard Lucent Acknowledgements

  22. Thanks! User Interface Software & Technology (UIST) October 30, 2002 http://www.cs.cmu.edu/~pebbles/puc/ http://www.cs.cmu.edu/~jeffreyn/

  23. Determining Panel Structure • Find states that others depend upon • Search for the group that contains this state • Determine mutually exclusive sets within group children • Assign sets to panels • Based upon types, tree properties FMStation Mode == Any FMPresets Power AMStation Mode AMPresets Band Radio Mode == Radio PlayState CD CDTrack Mode == CD

  24. Choosing Panel Types Rules for Choosing Panels • If… • state has boolean type • all controls are one set • Create… • Two full-screen overlapping panels.

  25. Choosing Panel Types, cont. Rules for Choosing Panels • If… • state has enumerated type • there is a set for every value of the state • Create… • a tabbed panel

  26. Choosing Panel Types, cont. Rules for Choosing Panels • If… • there are mutually exclusive sets of controls • Create… • overlapping panels • a control for the state not on the overlapping panel

More Related