1 / 40

Ink Parsing in Tablet PC

Ink Parsing in Tablet PC. Sashi Raghupathy Software Design Engineer TabletPC Microsoft Corporation. Revolution by Evolution – the Tablet PC. Practical – a full PC with no loss of existing capabilities Natural Input Device – Pen and Speech Pointing device is now on par with keyboard

roary-diaz
Download Presentation

Ink Parsing in Tablet PC

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Ink Parsing in Tablet PC Sashi Raghupathy Software Design Engineer TabletPC Microsoft Corporation

  2. Revolution by Evolution – the Tablet PC • Practical – a full PC with no loss of existing capabilities • Natural Input Device – Pen and Speech • Pointing device is now on par with keyboard • Highly mobile and connected • Wireless scenarios

  3. Tablet PC Form Factors • Designed by OEMs • Broad Range • Pure tablets, pen input only, and external keyboards • Convertibles with attached keyboards

  4. Extends PC Usage • Taking notes • Natural ‘pen and paper’ style note-taking • Annotating documents • Reading • Electronic books, magazines • Working away from desk • Greater social acceptance: meetings • New PC usage scenarios

  5. Note-taking application that ships with Tablet PC

  6. Features of a good note-taking application • Support freeform thought capture

  7. Sample notes

  8. Sample notes

  9. Sample notes

  10. Features of a good note- taking application • Support freeform thought capture • Add value to captured ink by allowing the users to • Edit / manipulate ink in a smart way • Search hand-written notes • Cleanup and export ink

  11. Text Recognition Engine • Provided by TabletPC Platform Services • High accuracy – best in the market • But, not designed for recognition of freeform notes • Assumes all input is text • Assumes horizontal baseline

  12. Deciphering document structure is key -- Parsing

  13. Parsing: OCR vs. Ink

  14. Ink Parsing • The ink parser analyzes a page of ink and decides: • which ink strokes belong to words • which ink strokes belong to drawings • which words belong to the same “line of words” • It does this in real-time, independently of ink size, orientation, or lines on the page

  15. How does this help? • Improves recognition results • users can write all over the page • users can write at an angle • users can write out-of-order • Makes Selection and manipulation of ink strokes easier/more intuitive • grouping is fundamental to selection, which is fundamental to manipulation • Enables advanced ink cleanup and export

  16. High-Level Approach • Layout analysis • Classification

  17. Layout Analysis Identify collinear ink of similar size & orientation, and group it

  18. Layout Analysis Steps Temporal Line Grouping Spatial Block Grouping Spatial Line Grouping List Detection Word Grouping Demo…

  19. Classification Drawing Writing Classify writing/drawing based on shape and context

  20. More Ink Parsing Examples…

  21. Live Demo of Windows Journal

  22. Dealing with Parsing errors • Allow user correction of parsing errors • Group/Ungroup • Alternate recognition text • Build advanced features such as reflow only after we reach higher levels of accuracy

  23. Future Work • Improve accuracy of Ink Parsing • Recognize more semantic structures • Paragraphs • Tables • Bulleted Lists • Annotations

  24. Acknowledgements • Tablet PC • David Jones • UC Berkeley • Michael Shilman • MSR Redmond • Patrice Simard • MSR Asia • Zile Wei • Jian Wang

More Related