1 / 24

Pen-based stuff

Pen-based stuff Ink as data Agenda Questions Project selection Natural data types Pen, Audio, Video Pen-based topics Technology Ink as data Recognition Natural Data Types As we move off the desktop, means of communication mimic “natural” human forms of communication

Olivia
Download Presentation

Pen-based stuff

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Pen-based stuff Ink as data

  2. Agenda • Questions • Project selection • Natural data types • Pen, Audio, Video • Pen-based topics • Technology • Ink as data • Recognition CS 4470/6456 - Fall 2002

  3. Natural Data Types • As we move off the desktop, means of communication mimic “natural” human forms of communication • Writing, Speaking, Seeing • Ink, Audio, Video (today and Wed) • Understanding these data types leads to interesting applications • e.g., capture applications (Friday) CS 4470/6456 - Fall 2002

  4. Pen Computing • Use of pens has been around a long time • Light pen was used by Sutherland before Engelbart introduced the mouse • Resurgence in 90’s • Types of “pens” • Passive (same as using a finger) • Active (pen provides some signal) CS 4470/6456 - Fall 2002

  5. Example Pen Technology • Passive • Touchscreen (e.g., PDA, some tablets) • Contact closure • Vision techniques • Active • Pen emits signal(s) • e.g. IR + ultrasonic • Where is sensing? Surface or pen CS 4470/6456 - Fall 2002

  6. Questions about Pens • What operations detectable • Contact – up/down • Drawing/Writing • Hover? • Modifiers? (like mouse buttons) • Which pen used? • Eraser? • Difference between pen and mouse. CS 4470/6456 - Fall 2002

  7. Example: Expansys Chatpen • Reads dot pattern on paper • Transmits via Bluetooth • http://www.expansys.com/product.asp?code=ERIC_CHATPEN CS 4470/6456 - Fall 2002

  8. Example: mimio • Active pens • IR + ultrasonic • Portable sensor • Converts any surfaceto input surface • We have chained theseto create big surface • http://www.mimio.com CS 4470/6456 - Fall 2002

  9. Pen input • Free-form ink • Soft keyboards • Recognition systems • generalize to gesture-based systems CS 4470/6456 - Fall 2002

  10. Free-form ink • ink as data • humans can interpret • time-stamping • implicit object detection • special-purpose “domain” objects CS 4470/6456 - Fall 2002

  11. Free-form ink examples • Ink-Audio integration • Tivoli (Xerox PARC) • eClass (GT) • FlatLand (Xerox PARC) • Dynomite (FX-PAL) • The Audio Notebook (MIT) CS 4470/6456 - Fall 2002

  12. Soft Keyboards • common on small mobile devices • many varieties • tapping interfaces • Key layout (QWERTY, alphabetical, … ) • learnability vs. efficiency CS 4470/6456 - Fall 2002

  13. T9 (Tegic Communications) • Alternative tapping interface • phone layout plus dictionary • Soft keyboard or mobile phone CS 4470/6456 - Fall 2002

  14. Quickwrite (Perlin) • “Unistroke” recognizer CS 4470/6456 - Fall 2002

  15. Cirrin (Mankoff) • Word-level unistroke recognizer CS 4470/6456 - Fall 2002

  16. Recognizing pen input • Graffiti • unistroke alphabet • Other pen gesture recognizers • for commands • Stanford flow menus; PARC Tivoli implicit objects • measure features of strokes • Rubine, Long • usually no good for “complex” strokes CS 4470/6456 - Fall 2002

  17. Handwriting recognition • Lots of resources • see Web • good commercial systems • Two major techniques: • on-line • off-line CS 4470/6456 - Fall 2002

  18. Mixing modes of pen use • Users want free-form and commands • or commands vs. text • How to switch between them? • (1 mode) recognize which applies • (2 modes) visible mode switch • (1.5 modes) special pen action switches CS 4470/6456 - Fall 2002

  19. Error correction • Really slows effective input • word-prediction can prevent errors • Various strategies • repetition (erase and write again) • n-best list • other multiple alternative displays CS 4470/6456 - Fall 2002

  20. Other interesting applications • Signature verification • Note-taking • group (NotePals by Landay @ Berkeley) • student (StuPad by Truong @ GT) • meetings (Tivoli and other commercial) • Sketching systems • early storyboard support (SILK, Cocktail Napkin) CS 4470/6456 - Fall 2002

  21. Toolkits for Pen-Based Interfaces • SATIN (Landay and Hong) – Java toolkit • MS Windows for Pen Computing • MS Pocket PC, CE.net • Apple Newton OS • GO PenPoint • Palm Developer environments • GDT (Long, Berkeley) Java-based trainable unistroke gesture recognizer • OOPS (Mankoff, GT) error correction CS 4470/6456 - Fall 2002

  22. SATIN (UIST 2000) • Pen input for informal input • Sketching (others have investigated this) • Common toolkit story • Gee, “X” sure is a neat class of apps! • Golly, making “X” apps is tough! • Here’s a toolkit to build “X” things easily! CS 4470/6456 - Fall 2002

  23. The SATIN Toolkit • The application space • Informal ink apps • Beyond just recognition • Pen “look-and-feel” • Abstractions • Recognizers • Interpreters • multi-interpreters CS 4470/6456 - Fall 2002

  24. Critiquing the Paper/SATIN • Strong points • Weak points CS 4470/6456 - Fall 2002

More Related