1 / 20

Introducing of handwritten icons

August 20th, 2008 ICFHR, Montreal. Ralph Niels , Don Willems and Louis Vuurpijl. Introducing of handwritten icons. Introducing the NicIcon database of handwritten icons. Ralph Niels Don Willems Louis Vuurpijl. NicI. Icon. Introducing the NicIcon database of handwritten icons.

sophie
Download Presentation

Introducing of handwritten icons

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. August 20th, 2008 ICFHR, Montreal Ralph Niels, Don Willems and Louis Vuurpijl Introducingof handwritten icons

  2. Introducing the NicIcon databaseof handwrittenicons Ralph Niels Don Willems Louis Vuurpijl NicI Icon

  3. Introducing the NicIcon databaseof handwritten icons Ralph Niels Don Willems Louis Vuurpijl Overview • Introduction • Domain: crisis management • Data collection • Icon design • Method • Data • Classification experiment • Method • Results • Conclusion and URL = NEW!

  4. Introducing the NicIcon databaseof handwritten icons Ralph Niels Don Willems Louis Vuurpijl Crisis management (CM) • Scenario: tunnel disaster • Distributed computer systems to support CM • Multiple modalities: speech, gestures, and pen The car is here

  5. Introducing the NicIcon databaseof handwritten icons Ralph Niels Don Willems Louis Vuurpijl Iconic pen gestures • Faster than handwriting • Easy to learn & remember • Visual meaningful shape

  6. Introducing the NicIcon databaseof handwritten icons Ralph Niels Don Willems Louis Vuurpijl Database • Public databases available for several handwriting applications • Not for iconic pen gestures • Based on symbology reference by USA Homeland Security Workgroup • Used in e.g., USA, Australia, New-Zealand

  7. Introducing the NicIcon databaseof handwritten icons Ralph Niels Don Willems Louis Vuurpijl Icons Flood Accident Car Bomb Roadblock Electricity Casualty Fire Fire brigade Police Injury Gas Paramedics Person

  8. Introducing the NicIcon databaseof handwritten icons Ralph Niels Don Willems Louis Vuurpijl Data collection • 35 participants • Online and offline • Variation in size • Per person: • 22 pages • 55 instances / icon

  9. Introducing the NicIcon databaseof handwritten icons Ralph Niels Don Willems Louis Vuurpijl Database content • Online: 26,163 iconic gestures(24,144 reported in paper) Tablet: Wacom Intuos2 A4 oversize

  10. Introducing the NicIcon databaseof handwritten icons Ralph Niels Don Willems Louis Vuurpijl Database content • Offline: 770 scanned pages Scanner: HP Scanjet 7400C flatbed (@ 300dpi, 24 bit colour)

  11. Introducing the NicIcon databaseof handwritten icons Ralph Niels Don Willems Louis Vuurpijl Classification experiment Data sub sets • Stratified • Writer dependent (WD), writer independent (WI) Complete data set (26,163 icons) Train set(36%) Test set (24%) Evaluation set(40%) WD Train set(36%) WI Train set(36%) WD Test set (24%) WI Test set (24%) WD Evaluation set(40%) WI Evaluation set(40%)

  12. Introducing the NicIcon databaseof handwritten icons Ralph Niels Don Willems Louis Vuurpijl Classification experiment on online data Multi classifier system • Feature sets • 28 geometric features (Willems & Vuurpijl) • 30/60 coordinates running features (Schomaker & Vuurpijl) • 1185 features (Willems, Niels, Van Gerven, Vuurpijl) • Feature classifiers • Support Vector Machine • Multi-Layered Perceptron • Template matching • Dynamic Time Warping (Niels & Vuurpijl) NEW!

  13. Introducing the NicIcon databaseof handwritten icons Ralph Niels Don Willems Louis Vuurpijl New feature set (‘m-fs’) • Features from literature • Geometrical, temporal, pressure D. Rubine, Specifyinggesturesbyexample, Computer Graphics. 25 (4) (1991) 329–337. J. LaViola Jr. & R. Zeleznik, A practical approachforwriter-dependentsymbolrecognitionusing a writer-independentsymbolrecognizer, IEEE Transactionsonpatternanalysis and machine intelligence. 29 (11) (2007) 1917–1926. L. Zhang & Z. Sun, Anexperimentalcomparison of machine learningforadaptive sketch recognition, AppliedMathematics and Computation. 185 (2) (2007) 1138–1148. and many others…

  14. Introducing the NicIcon databaseof handwrittenicons Ralph Niels Don Willems Louis Vuurpijl New feature set (‘m-fs’) • Over complete icon, but also… • Mean over strokes • Std. dev. over strokes Feature definitions in technical report at website

  15. Introducing the NicIcon databaseof handwritten icons Ralph Niels Don Willems Louis Vuurpijl New feature set (‘m-fs’) • Feature selection • 1185 features, which are the best? • Sort them on best individual performance • Add 1 by 1 to classifier • Performance maximizes at: • 545 features for WI • 660 features for WD • Selected features: • ± 1/3 full icon • ± 1/3 mean • ± 1/3 standard deviation The best features

  16. Introducing the NicIcon databaseof handwrittenicons Ralph Niels Don Willems Louis Vuurpijl Breaking news: the best features Averagecentroidal radius Sine first /last sample Length ofdiagonal Verticaloffsets Area

  17. Introducing the NicIcon databaseof handwritten icons Ralph Niels Don Willems Louis Vuurpijl Classificationresults NEW! g-28: 28 geometric features(Willems & Vuurpijl) m-fs: 1185 features(Willems, Niels, Van Gerven, Vuurpijl) af-30/af-60: 30/60 running features (Schomaker & Vuurpijl)DTW: Dynamic Time Warping(Niels & Vuurpijl) MCS: Multiple classifier system (majority voting)

  18. Introducing the NicIcon databaseof handwritten icons Ralph Niels Don Willems Louis Vuurpijl Misclassifications • Wrong box • Sloppy drawing • Retracing

  19. Introducing the NicIcon databaseof handwritten icons Ralph Niels Don Willems Louis Vuurpijl Discussion • Results already quite good (WD: 99.5%, WI: 97.8%),but gain is still possible • Which features are important? • For different domains • Performance in interactive experiment • Offline data still open • Mapping online -> offline

  20. Introducing the NicIcon databaseof handwritten icons Ralph Niels Don Willems Louis Vuurpijl http://unipen.nici.ru.nl/NicIcon • Freely available: • Online (Unipen) • Offline (PNG) • Technical report about new feature set

More Related