1 / 24

Improving Chinese handwriting Recognition by Fusing speech recognition

This study by Zhang Xi-Wen from CSE, CUHK, and HCI Lab at ISCAS 2005 explores improving Chinese handwriting recognition by fusing speech recognition, detailing the challenges, segmentation methods, character recognition, speech recognition, and text fusion. The experimental results and the fusion process using dynamic programming are discussed, showing enhanced recognition results through information fusion.

rosador
Download Presentation

Improving Chinese handwriting Recognition by Fusing speech recognition

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Improving Chinese handwriting Recognition by Fusing speech recognition Zhang Xi-Wen CSE, CUHK and HCI Lab., ISCAS 2005.4.12

  2. Outline • 1 Chinese handwriting recognition • 2 Chinese speech recognition • 3 Information fusion • 4 Experimental results

  3. Handwriting Recognition • Handwriting segmentation • Character recognition

  4. 1.1 Handwriting segmentation • It is more difficult for Chinese handwriting segmentation

  5. Character extraction using histogram • A histogram of between-stroke gaps. • The dimidiate threshold of the histogram is to extract lines of strokes. • The dimidiate threshold of the histogram of a line of strokes is to extract characters.

  6. Figure 1. Handwriting segmentation

  7. Problems remained • A Chinese character may be mis-segmented into many characters. • Many Chinese characters may be mis-grouped as a character. • The segmentation error will inevitably result in handwriting recognition errors.

  8. 1.2 Character recognition • Isolated character recognizer from HW • Many candidates

  9. Handwriting. Text recognized from the handwriting. The ground-truth text. Figure 2. Handwriting recognition

  10. 2 Speech recognition • Chinese speech. • On-line, microphone. • Continuous speech recognizer from MS.

  11. Text recognized from the speech corresponding to the handwriting. The ground-truth text. Figure 3. Speech recognition

  12. 3 Text fusion • An optimization problem • Dynamic Programming

  13. 3.1 Principles • The fused text should contain more semantic information. • Construct a text with the least characters and the most semantic information.

  14. 3.2 Four ways Text recognized from the handwriting. Text recognized from the speech corresponding to the handwriting. Figure 4. Texts to be fused

  15. 3.3 Dynamic Programming • A directed graph. • Optimal paths.

  16. Figure 5. A directed graph with N levels.

  17. (a) Text recognized from the handwriting. (b) Text recognized from the speech corresponding to the handwriting. (c) The optimal fused text corresponding to the optimal path. (d) The ground-truth text. Figure 6. Text fusion using DP.

  18. 3.4 A language model • Lexicon • Syntax • Semantic

  19. Lexicon

  20. 4 Experimental results

  21. Thank you very much for • your criticism, comments and suggestions! • Email: xwzhang@cse.cuhk.edu.hk • Tel: 3163-4260

More Related