1 / 31

Enhancing Pen-Based User Interfaces: Modes and Selection Mechanisms

Explore pen modes, selection mechanisms, gestures, handwriting recognition, and more in pen-based interfaces. Discover how different operations are managed with a pen, including ink vs. erasing, gestures, and control. Uncover the challenges and solutions in pen mode study and selection mechanisms. Dive into recognition scenarios, diagram recognition, and text recognition, including handwriting recognition techniques. Join the discussion on gestures, their ambiguity, robustness, and issues in recognition interfaces. Learn about UIs based on attaching meaning to ink strokes and improving gesture accuracy.

duffeyj
Download Presentation

Enhancing Pen-Based User Interfaces: Modes and Selection Mechanisms

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Pen Based User Interface II CSE 481b January 25, 2005

  2. Today • Pen Modes • Pen Input • Pointing • Discrete Selection • Control • Text Input • Content selection • Recognized input • Glyphs • Gestures • Diagrams • Handwriting

  3. Announcements • 1/24, 1/26: HCI for Pen Computing • 1/31: Real Time Stylus (Arin Goldberg) • 2/2: Topic TBA (Valentin) • 2/7: No class (probably) • 2/9: Prototype presentations (teams)

  4. Pen mode solutions • Problem: How do you allow different operations with a pen • Ink vs. erasing • Explicit modes • Ink vs. gesture • Recognition of gesture overrides ink • Ink vs. recognition vs. control • Area based modes

  5. Pen mode studyYang Li et al., CHI 2005 • Barrel Button • Hold • Non-preferential hand button • Pressure • Eraser

  6. Quick poll • Barrel Button • Hold • Non-preferential hand button • Pressure • Eraser Student Quick Poll

  7. Classify input activities with a pen What type of input activities are easy with a pen? What type of input activities are difficult with a pen? Student Submission

  8. Activity classification Difficult Easy Moderate

  9. Discrete selection • Choose from a finite set • Command from a menu • Character from an alphabet • Repeated selection from finite sets • Hierarchical menus • Commands with arguments • Sequences of characters • Words • Multi-digit numbers

  10. How many distinct mechanisms can you construct to select between four choices with a pen? The point of this exercise is that there are a small number of mechanisms that are distinct Student Submission Assume a pen without a button

  11. Selection mechanisms • Crossing • Pointing • Writing • Tapping • Pressure

  12. Basic pen operation • Crossing • Operation triggered by a stroke crossing a line segment

  13. CrossY: Crossing based UI • Specify operations by drawing through

  14. Hierarchical crossing • Principle – multiple commands without lifting the pen

  15. Flow Menu • Use movement through octants for control information Move Item 100% 100% Text Item Highlight Zoom 66.6% 200% 66.6% 200% Shape 50% 75.0 400% 50% 100.0 400% 25% 800% 25% 800% Custom Custom

  16. Abstract writing • Enter text with specialized, stroke based recognition • Optimized for automatic recognition • Not human readable • Character based or word based

  17. Cirrus (Georgia Tech) interaction education

  18. Quikwrite [Perlin, NYU] c a r p

  19. Writehelloworld Student Submission

  20. Selection problem • Identify one or more graphical elements from a domain • Mechanisms • Bounding Region • Geometric defined by stroke • Distance from cursor

  21. If the red circle is a selection tool, what is selected? Student Submission

  22. Bubble cursor • Selection radius depends on object proximity

  23. Recognition UIs • UIs based on attaching meaning to ink • Gestures • Diagram recognition • Handwriting recognition • Free form • Constrained recognition

  24. Gestures • Commands issued with a single stroke • May be drawn or invisible • Support from SDK • Register gestures to be recognized • UI Issues • Similar to keyboard short cuts • Speed up for experts • Hard to learn / remember

  25. Gestures • Ambiguity • Distinction between gestures • Distinction between gesture and other ink • Robustness • Handling misrecognized gestures • False positive • False negative • Gesture initiated actions should be undoable

  26. Diagram recognition • Challenges to recognition • Even simple shapes are hard! • Variation in drawing • Ink artifacts

  27. Text recognition • We will have a great lecture later in the course! • Basic approach • Collect a huge amount of data • Use data to train neural net

  28. Handwriting Recognition:Identify the following words Student Submission

  29. Recognition results

  30. Recognition scenarios • What level of error is tolerable • How is feedback provided to the user on recognition • How does the user specify corrections?

More Related