Pen-Centric Shorthand Handwriting Recognition Interfaces

1. Pen-Centric Shorthand Handwriting Recognition Interfaces Charles C. Tappert1 and Jean R. Ward2 1 School of CSIS, Pace University, New York, USA 2 Pen Computing Consultant, Massachusetts, USA

2. Famous writings throughout history were effectively written in a style of shorthand Cicero’s orations Martin Luther’s sermons Shakespeare’s and George Bernard Shaw’s plays Background

3. Can provide critical infrastructure for natural pen-centric interactions, enhancing many pen-centric learning applications Can provide faster text input for pen-centric teaching, studying, and learning applications Will have greatest impact on the utility of applications running on small mobile devices Thesis: Pen-Centric Shorthand Interfaces

4. Fundamental Property of Writing Handwriting Recognition Difficulties Online(Pen-Centric) Handwriting Recognition Onlinemore accurate than Offline Recognition Online Info Can Complicate Recognition Process Design Tradeoffs/Decisions Historical Shorthand Alphabets Pen-Centric Shorthand Alphabets Pen-Centric Word/Phrase Shorthand Allegro/Chatroom Shorthand System Agenda

5. Differences between different characters are more significant than differences between different drawings of the same character This makes handwritten communication possible Fundamental Property of Writing

6. Fundamental Property of Writing • Property holds within subalphabets of uppercase, lowercase, and digits, but not across them • “I”, “l”, and “1” written with single vertical stroke • “O” and “0” written similarly with an oval

7. Shape, size, and slant variation Similarly shaped characters – U and V Careless writing in the extreme, almost illegible writing Resolving difficult ambiguities requires sophisticated recognition algorithms, syntax/semantics Handwriting Recognition Difficulties

8. Machine recognizes the writing while the user writes Digitizer equipment captures the dynamic information of the writing Stroke number,order,direction,speed A stroke is the writing from pen down to pen up Online(Pen-Centric) Handwriting Recognition

9. Can use both dynamic and static information Can often distinguish between similarly shaped characters E.g., 5 versus S where the 5 is usually written with two strokes and the S with one stroke Online(Pen-Centric) more accurate than Offline (Static) Recognition

10. Online Information Can Complicate Recognition Process • Large number of possible variations • E can be written with one, two, three, or four strokes, and with various stroke orders and directions • Four-stroke E has 384 variations (4! stroke orders x 24 stroke directions)

11. Online Information Can Complicate Recognition Process • Segmentation ambiguities • character-within-character problem • lowercase d might be recognized as a cl if drawn with two strokes that are somewhat separated from one another

12. No constraints on the user Machine recognizes user's normal writing User severely constrained Must write in particular style such as handprint Must write strokes in particular order, direction, and graphical specification Design Tradeoffs/Decisions

13. Small alphabet one case rather than both upper and lowercase Small number of writing variations per letter preferably only one One stroke per character (character = stroke) allows machine to recognize each character upon pen lift Separate writing areas for letters and digits avoids confusion of similarly shaped letters and digits Simplified Design Tradeoffs/Decisions for Graffiti and Allegro PDA Alphabets

14. We first review the history of shorthand systems prior to pen computing Shorthand is “a method of writing rapidly by substituting characters, abbreviations, or symbols for letters, words, or phrases” Shorthand can be traced back to the Greeks Historical Shorthand Alphabets

15. We focus on shorthand alphabets that might be appropriate for PDAs We review two types of shorthand Geometric shorthand Small number of basic shapes Shapes reused in multiple orientations Non-geometric shorthand Historical Shorthand Alphabets

16. Tironian Alphabet, 63 B.C.

17. Stenographie Alphabet, 1602

18. Stenographie Alphabet, 1602 Geometric shorthand – basic shapes/orientations

19. Moon Alphabet, 1894 • Geometric shorthand – basic shapes/orientations

20. Phonetic alphabets Pitman (1837) Gregg (1885) Systems for the blind Braille (1824) Cursive shorthands Gabelsberger (1834) Other Historical Shorthand Systems

21. Some of the earliest were for CAD/CAM symbols represent graphical items and commands Others developed for text input on small consumer devices like PDAs that have limited computing power We review geometric and non-geometric shorthands appropriate for small devices Pen-Centric Shorthand Alphabets

22. Historical alphabets presented above could be used for machine recognition symbols drawn with a single stroke (except K in Tironian and + in Stenographie) In addition to shape and orientation, online systems can use stroke direction to differentiate among symbols Pen-Centric Shorthand Alphabets

23. Allen Alphabet

24. Allen AlphabetBasic Shapes and Orientations

25. Goldberg Alphabet

26. Goldberg AlphabetBasic Shapes and Orientations

27. Graffiti and Allegro Alphabets • Not geometric alphabets • High correspondence to Roman alphabet • Might not qualify as shorthand but included here

28. Graffiti Alphabet

29. Allegro Alphabet

30. Commercially Successful Shorthands • Similar to the Roman alphabet • Easier to learn • Graffiti used in Palm OS devices • notably the Palm Pilot and Handspring models • Allegro used in Microsoft Windows devices • Geometric alphabets not successful

31. Pen-Centric Word/Phrase Shorthande.g., Chatroom Shorthand • Further increase speed of text entry • Potential applications • Where input speed important • Where word/phrase abbreviations occur frequently – e.g., email

32. Allegro/Chatroom Shorthand System • Developed for M.S. dissertation • Student was hearing impaired • Developed as output component of communication system • Handwriting to text to speech • Two input writing areas • One for Allegro (all-purpose) • One for chatroom-like words/phrases

33. Allegro/Chatroom Shorthand System

34. Allegro/Chatroom Shorthand System

35. Pen-centric learning interfaces should use shorthand, and word/phrase shorthand where appropriate, for fast text input Benefit of shorthand interfaces Provides critical infrastructure for many pen-centric learning applications Enhances natural pen-centric interactions for teaching, studying, and learning applications Has greatest impact on the utility of applications running on small mobile devices Conclusions