1 / 38

In Search of Text Writing Methods for Off the Desktop Computing ― ATOMIK and SHARK

In Search of Text Writing Methods for Off the Desktop Computing ― ATOMIK and SHARK Shumin Zhai In collaboration with Barton Smith, Per-Ola Kristensson ( Linkoping U ), Alison Sue, Clemens Drews, Paul Lee ( Stanford ), Johnny Accot, Michael Hunter ( BYU ), Jingtao Wang ( Berkeley )

johana
Download Presentation

In Search of Text Writing Methods for Off the Desktop Computing ― ATOMIK and SHARK

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. In Search of Text Writing Methods for Off the Desktop Computing ― ATOMIK and SHARK Shumin Zhai In collaboration with Barton Smith, Per-Ola Kristensson (LinkopingU), Alison Sue, Clemens Drews, Paul Lee (Stanford), Johnny Accot, Michael Hunter (BYU), Jingtao Wang (Berkeley) IBM Almaden Research Center San Jose, CA

  2. Computing off the desktop • Desktop computing “workstation” interface foundation • Large and personal display • Input device (mouse) • Typewriter keyboard • HCI Frontier – beyond the desktop • Interfaces without display-mouse-keyboard tripod • Numerous difficult challenges

  3. The text input challenge • Indispensable user task • Efficiency • Learning • Size / portability • Visual cognitive attention • “History” of writing technology

  4. Text Entry Methods • Reduced keyboard • T9, miniature keyboard • Hand writing • English, Unistroke, Graffiti • Speech • Human factors limitation • Stylus (graphical) keyboards

  5. The QWERTY Keyboard • Invented by Sholes, Glidden, and Soule in1868 ― minimizing mechanical jamming • QWERTYnomics (P. David vs. Liebowitz & Margolis) • Touch typing ― low visual attention demand • Happen to be good for two hands alternation ― Dvorak did not prevail

  6. Wj Key ii Key j Dij Fitts’ law For stylus keyboard — a = 0.08 sec, b = 0.127 sec/bit (Zhai, Su, Accot, CHI 2002)

  7. Letter Transition Frequency (Digraph) • Mayzner and Tresselt (1965) • British National Corpus (BNC) • 2 new modern corpora • News - NY Time, SJ Mercury, LA Times • Chat room logs

  8. 34.2 WPM Movement Efficiency Model of Stylus Keyboards (Soukoreff & MacKenzie,1995; Zhai, Sue & Accot 2002)

  9. Manual explorations OPTI, MacKenzie & Zhang (42.8 wpm) FITALY keyboard (41.2 wpm)

  10. Zhai, Hunter, Smith, UIST2000 Algorithmic design - dynamic simulation Hooke’s Keyboard (45.1 wpm)

  11. Fitts-digraph “energy” • “Random walk” Zhai, Hunter & Smith, HCI 2002 Metropolis Method • UI physics - Keyboard as a “molecule” • Annealing – varying T

  12. 46.6 wpm – 36% more efficient than QWERTY

  13. 30% smaller search area by Hick’s law analysis Smith & Zhai INTERACT2001 Alphabetical “tuning” for novice users Novice user taping speed (wpm)

  14. Word connectivity • Zipf’s law Pi ~ 1/ia • connectivity Index

  15. 18000 16000 14000 Word connectivity Human Movement Study: Fitts’ law MT = a + b Log2(Dsi/Wi + 1) 12000 10000 8000 6000 4000 2000 0 sp E T A H O N S R I D L U W M C G Y F B P K V J X Q Z English Letter Corpus(News, chat etc) “Fitts-digraph energy” Metropolis “random walk” optimization Alphabetical tuning Alphabetically Tuned and Optimized Mobile Interface Keyboard (ATOMIK)

  16. Limitations and hints from ATOMIK • Tapping one key at a time – tedious. The stylus can be more expressive and dexterous. • Does not utilize language redundancy/statistical intelligence. • People tend to remember the pattern of a whole word, not individual letters.

  17. “word” Zhai, Kristensson, CHI 2003 The new phase - SHARK The basic idea: gesturing the word pattern defined by the keyboard

  18. Shorthand Aided Rapid Keyboarding ― SHARK Sample “sokgraphs” (Shorthand On Keyboard)

  19. Principle 1 - efficiency “Writing” one word at a time (not letters)

  20. A form of shorthand

  21. Principle 2: Scale and location relaxation • Sokgraph patterns, not individual letters crossed, are recognized and entered • Lower visual attention demand from tapping

  22. Principle 3: Duality tapping/tracing to gesturing • (Novice) User’s choice • Tapping and tracing as a bridge to shorthand gesturing. • Same trajectory pattern.

  23. Principle 4: Zipf’s law and common word components • A small number of words make disproportional percent of text • Common components e.g. -tion, -ing • Benefits early

  24. Principle 5 – Skill transition • Consistent movement patterns between tapping/tracing and gesturing • Visually guided action to recall based action • Gradual shift: closed-loop to open-loop • Falling back and relearning

  25. Related Work • Artificial alphabets • Unistrokes (Goldberg & Richardson 1993) • Graffiti (Blickenstorfer 1995) • Quikwriting (Perlin 1998) • Cirrin (Mankoff & Abowd 1998) • Dasher (Ward, Blackwell, Mackay 2000) • Marking menus (Kurtenbach & Buxton 1993) • T-Cube (Venolia & Neiberg 1994)

  26. Shark Gesture Recognition • Gesture recognition • sampling • filtering • normalization • matching against prototypes • Many shape matching algorithms • complexity – scalability • accuracy • cognitive, perceptive, motoric factors • Currently elastic matching

  27. Elastic Matching (Tappert 1982) • Measuring curve to curve distance • Minimizing average distance by finding closest corresponding points • Dynamic programming

  28. Live demo

  29. Many issues • Most compelling • Can people learn, remember, produce recognizable SHARK gestures at all? • Are SHARK gesture too arbitrary? • Is SHARK really feasible?

  30. Zhai, Kristensson, CHI 2003 A “Feasibility” Experiment

  31. Results: number of words learned per session

  32. Study conclusions • SHARK gestures can be learned • About 15 words per hour • About 60 words learned in 4 hours – already very useful (40% BNC)

  33. More research questions • Robust sokgraph recognition algorithms are being developed • Intimate human-machine interaction • Visual attention • Learning, skill acquisition • How people perceive, remember, produce gestures (e.g. topological vs. proportional)? • Speed accuracy trade-off • How fast people can do gestures? • How “sloppy” people get? • What is “reasonable”? • How do user computer “negotiate”? • Information quantification and modeling • Theory!

  34. D D W W Snapshot of other research programs ― Laws of action • Law of Pointing (Fitts’ law) • t = f (D/W) (Fitts, 1954) • Pointing with amplitude and directional constraint (Accot & Zhai, CHI 2003) • Two types of speed-accuracy tradeoff (Zhai 2004) • Law of Crossing • More than dotting the i’s (Accot & Zhai, CHI’02) • Law of Steering • Beyond Fitts’ law (Drury 1975, Accot& Zhai CHI’97) • VR locomotion (Zhai, Waltjer, IEEE VR 2003 best paper) • More “laws” needed

  35. Snapshot of other research programs ― eye gaze sensing based interaction • Hand-Eye coordinated action ― MAGIC pointing (Zhai, Morimoto, Ihde CHI’99; Zhai CACM 2003) • EASE Chinese input (Wang, Zhai, Su, CHI’01)

  36. Thank you and questions

  37. xW W W Varying Key Sizes • Fitts’ law • log(D/W + 1) • Central location effect • Asymmetry • Packing • Varying control precision Combined time Time from left to right key Time from right to left key

  38. 35 30 25 WPM 20 15 10 test 0 test 2 test 4 test 6 test 8 test 10 test 1 test 3 test 5 test 7 test 9 Zhai, Sue, Accot, CHI 2002 Learning • ERI (Expanding rehearsal interval)

More Related