Hashigo: A Next-Generation Sketch Interactive System for Japanese Kanji

1. Hashigo: A Next-Generation Sketch Interactive System for Japanese Kanji Innovative Applications of Artificial Intelligence 2009 – Pasadena, CA, USA

2. (40 years ago) In 1969… [1963]

3. (20 years ago) In 1989…

4. (Present day) In 2009…

5. Our Solution – Computer aided language instruction using AI

6. About the Japanese Language

7. About Japanese Instruction Reading Writing Culture Speaking Listening

8. Comparing English to Kanji

9. Difficulties of Learning Kanji Many strokes Many Symbols Similarities in Kanji Writing Variations

10. About Stroke Order/Direction

11. Analogous Example to English w w w a a a t t t e e e r r r

12. Concrete Benefit for Order & Dir. 癴

13. Current Ways of Teaching Kanji Home Repetition Flashcard Memorization Workbook Self-Quizzes In-class Learning

15. 月 月 月 月 月 月 月 月 月 月 月 月 月 月 月 月 月 月 月 月 月 月 月 月 月

17. Kanji Assessment (on paper) What was drawn How it was drawn

18. Real-Time Kanji Assessment?(1-on-1 vs. 1-to-many) What was drawn How it was drawn

19. Labeling the stroke numbers • not natural • trying to teach natural writing 2 3 1 4

20. A “Dynamic” Workbook Solution is practical, robust, and cost-effective $ $ $ ★ ★ ★

21. “Dynamic” Workbook Features • pen input • no drawing location constraints • size independence • visual assessment (what it looks like) • writing assessment (how it was drawn)

22. Related Systems & Capabilities

23. Related Systems & Capabilities

24. Our Hashigo System digital kanji teacher paper

25. Background on System’s Name Hashigo LAnguage for Describing Drawing, 梯子 Display, and Editing in LADDER Recognition

26. Utilized Resources 力 Sezgin primitive shape recognizer LADDER sketching language

27. Addressing the Features

28. About LADDER components constraints aliases 十 古 口

29. About LADDER components constraints aliases

30. About LADDER 口 components constraints aliases 十 kanji048 horzLine line1 mouth

31. Shape Description Convention line orientation point orientation shape interaction

32. Shape Description Convention line orientation point orientation shape interaction center p1 p1 center p2 p2

33. Shape Description Convention line orientation point orientation shape interaction

34. Visual Recognition of Kanji 十 • Line • horzLine • Line • vertLine components

35. Visual Recognition of Kanji • Horizontal • horzLine • Vertical • vertLine line orientation

36. Visual Recognition of Kanji • LeftOf • horzLine.p1 • horzLine.p2 • Above • vertLine.p1 • vertLine.p2 point orientation

37. Visual Recognition of Kanji • SameSize • vertLine • horzLine • SameX/Y • horzLine.center • vertLine.center shape interaction

38. Technique Recognition of Kanji • Line line1 • horzLine • Line line2 • vertLine • Point start1 • vertLine.p1 • Point end1 • vertLine.p2 • Point start2 • horzLine.p1 • Point end2 • vertLine.p2 start2 line1 end1 start1 line2 end2 aliases

39. Technique Recognition of Kanji start2 Stroke Order? Stroke Direction? line1 line2 start1 start2 line2 start1 end1 line1 end2

40. Technique Recognition of Kanji start2 Stroke Order? Stroke Direction? line2 line1 start2 start1 line2 start1 end1 line1 end2

41. Technique Recognition of Kanji start2 Stroke Order? Stroke Direction? line1 line2 end1 start2 line2 line1 start1 end1 end2

42. The Hashigo CALI Application

43. The Hashigo CALI Application

44. The Hashigo CALI Application

45. The Hashigo CALI Application

46. Preliminary Studies • Sketch Recognition of Expert Writer’s Handwritten Kanji • Assessment Recognition of Novice Students’ Handwritten Kanji • Novice Students’ Kanji Learning Progress

47. Preliminary Study #1 古吾冒朋明 唱晶品呂昌 早旭世胃旦 胆亘凹凸

48. Preliminary Study #1 85% 93% 98% * OCCR: Online Chinese Character Recognizer

49. Preliminary Study #2 古吾冒朋明 唱晶品呂昌 早旭世胃旦 胆亘凹凸

50. Preliminary Study #2 100% 99%