1 / 73

Interaction Techniques for Ambiguity Resolution in Recognition-based Interfaces

Interaction Techniques for Ambiguity Resolution in Recognition-based Interfaces. Jennifer Mankoff CoC & GVU Center Georgia Tech. Acknowledgements. Gregory Abowd & Scott Hudson FCE Group & GVU NSF. Outline. Motivation Definitions & Illustration Broad Solution: OOPS Specific Solutions

selah
Download Presentation

Interaction Techniques for Ambiguity Resolution in Recognition-based Interfaces

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Interaction Techniques for Ambiguity Resolution in Recognition-based Interfaces Jennifer Mankoff CoC & GVU Center Georgia Tech

  2. Acknowledgements • Gregory Abowd & Scott Hudson • FCE Group & GVU • NSF

  3. Outline • Motivation • Definitions & Illustration • Broad Solution: OOPS • Specific Solutions • Conclusion & Future Work

  4. Ambiguity • am·big·u·ous1 a: doubtful or uncertain especially from obscurity or indistinctness <eyes of an ambiguous color> b: INEXPLICABLE2: capable of being understood in two or more possible senses or ways • An anathema to computers • Normal for humans

  5. Where does ambiguity arise? • Web search • user doesn’t know correct answer • multiple correct answers • Implicit input • user not involved with application • Multiple users • System states may not agree • Recognition

  6. Focus: Recognition • Recognition is becoming ubiquitous • Recognition is difficult to use • A range of interface problems result • OOPS toolkit helps solve them

  7. Research Methodology • Motivated by real world, non-CS problems • Evaluators • Study individual problems/solutions • Design space of possible solutions • Builders • Facilitate solutions to sets of problems (Architectural / toolkit solutions) • Design space exploration

  8. Outline • Motivation • Definitions & Illustration • Broad Solution: OOPS • Specific Solutions • Conclusions & Future Work

  9. Definitions • Mediation • dialogue between user and computer • used for resolving ambiguity • Recognizer • interprets user input • creates ambiguity • Error • mistake from user’s perspective • represented with ambiguity

  10. SILK (Landay, 1996)

  11. Burlap

  12. Outline • Motivation • Definitions & Illustration • Broad Solution: OOPS • Specific Solutions • Conclusions & Future Work

  13. OOPS Toolkit (CHI’00) • Toolkit-level support for handling ambiguity in recognition • Library of mediators • Architectural support • Based on subArctic

  14. Library of mediators • Design space based on survey • Generic and re-usable • Three major classes • Repetition • Choice • Automatic

  15. Library of mediators • Design space based on survey • Generic and re-usable • Three major classes • Repetition • Choice • Automatic

  16. Library of mediators • Design space based on survey • Generic and re-usable • Three major classes • Repetition • Choice • Automatic if (result is “W.”) reject it else do nothing

  17. Architectural Support • INDEPENDENT of any specific toolkit • Separation of mediators, recognizers, and application • Communication by a common internal model (ambiguous hierarchical events) • Maintains ambiguity indefinitely

  18. Three key pieces • Ambiguous hierarchical events • Changes to event dispatch • Mediation subsystem

  19. down down drag drag up up • • • • • • • • • • • • stroke stroke s c Ambiguous Hierarchical Events s

  20. Event Dispatch • A sensed event arrives • It is dispatched to all recognizers It is mediated rec1 rec2 Input handler event rec3 rec4 recn med2 med1 med4 med3 medn

  21. Mediation Subsystem • Ambiguity is identified automatically • Presence of multiple interactor leaf nodes • Hierarchy is passed to mediators • Recognizers, recipients informed of accept/reject decisions • Accept/reject modifies hierarchy • Application selects mediators from library

  22. Outline • Motivation • Definitions & Illustration • Broad Solution: OOPS • Specific Solutions • Conclusions & Future Work

  23. Problem Areas • Errors & Ambiguity • rejection errors • target ambiguity • Mediation • adding alternatives • occlusion

  24. Problem: The user’s input is completely missed Rejection Errors

  25. Problem: The user’s input is completely missed Solution: Allow the user to tell the system Rejection Errors

  26. Problem: The user’s input is completely missed Solution: Allow the user to tell the system Other applications: Substitution errors Rejection Errors

  27. Problem: The user’s input is completely missed Solution: Allow the user to tell the system Other applications: Substitution errors Any spatial recognition Requires extended recognizer API Rejection Errors

  28. Problem: There may be multiple targets of a user action Example: clicking Target Ambiguity

  29. Problem: There may be multiple targets of a user action Example: Clicking Solution: Give the user a choice of all of the targets Target Ambiguity

  30. Problem: There may be multiple targets of a user action Example: Clicking Solution: Give the user a choice of all of the targets Other applications: Any interface involving mouse press/release Requires separation of concerns Works with all interactors Target Ambiguity

  31. Problem: A mediator may obscure important information Occlusion

  32. Problem: A mediator may obscure important information Solution: Move that information into a more visible location Occlusion

  33. Problem: A mediator may obscure important information Solution: Move that information into a more visible location Other applications: Any crowded interface that uses an n-best list Requires extensible mediators Requires separation of concerns Occlusion

  34. Problem: The correct choice isn’t always present Adding alternatives

  35. Problem: The correct choice isn’t always present Example: word-prediction Adding alternatives

  36. Problem: The correct choice isn’t always present Example: word-prediction Solution: Allow the user to add choices Adding alternatives

  37. Problem: The correct choice isn’t always present Example: word-prediction Solution: Allow the user to add choices Other applications: Closely related choices (e.g. URL prediction) Requires extensible mediators Benefits from recognizer API Adding alternatives

  38. Outline • Motivation • Definitions & Illustration • Broad Solution: OOPS • Specific Solutions • Conclusions & Future Work

  39. Conclusions • Resolution of ambiguity in recognition through mediation • General toolkit architecture (CHI 00) • flexible, re-usable support for mediation • separates recognition, mediation, and applications • allows exploration of design space

  40. Next Steps • Implicit input • Sensed information about environment • Ambiguous output • Ambient displays • Brain-computer interface • Ambiguous, limited, error-prone input

  41. Locked-In Syndrome • Disease, stroke, accident survivors • Completely paraylzed • Unable to speak • Cognitively intact 500,000 people worldwide

  42. Brain-computer interface • Problem: locked-in syndrome • No alternative modalities available • Need for efficient error handling • Challenge: interpret brain signal in as rich a form as possible

  43. A new hope • Brain signals can be intercepted • Implanted electrodes (Schwartz, Chapin et al.) • External sensors (Junker, Wolpaw, Middendorf, Birbaumer et al., Spencer et al.) • Signals can be produced and controlled by imagined movements • Signals can be interpreted by a computer

  44. A Neurotrophic Electrode • Cone electrode invented in 1987 • Animal studies showed it to be stable • FDA permission given for human implantation in 1996

  45. Project Goals • High level: • Recreate movement • Restore communication • Turn disability into ability • Low level: • Intelligent Interpretation

  46. Recreating movement • Haptic feedback • Muscle stimulation • Eventually re-connect nerves

  47. Restore Communication • A basic need • From virtual keyboards to word-prediction

  48. Turning disability into ability:Supporting Creativity • Converting a the brain signal to music • Example • Two possible experiences • Trying to play a piano with ones feet • Having an artistic voice no one else can reproduce

  49. Intelligent Interpretation • Signals are difficult to control • Daily variability in signal • Patient Endurance • Adaption necessary

  50. Intelligent Interpretation • Raw signal-> mouse • Logical control • Neural gestures

More Related