1 / 39

A mathematical formula recognition method and its performance evaluation

A mathematical formula recognition method and its performance evaluation. Masayuki Okamoto Shinshu University JAPAN. Goal of our study Character and symbol recognition Structure analysis and recognition Performance evaluation method Experimental results Future works.

finola
Download Presentation

A mathematical formula recognition method and its performance evaluation

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. A mathematical formula recognition method and its performance evaluation Masayuki Okamoto Shinshu University JAPAN

  2. Goal of our study Character and symbol recognition Structure analysis and recognition Performance evaluation method Experimental results Future works Overview of presentation

  3. High performance formula recognition system for “Archiv der Mathematik” Goal of our study

  4. Overview of Recognition System Labeling Character or symbol recognition Touching character separation Structure recognition

  5. Font type (1/2) • Alphabet • Roman • Italic • Bold • Calligraphy • German • Greek

  6. Font type (2/2) • Digits • Mathematical symbols • Characters Normal size Small size Number of characters or symbols: 650

  7. Dictionary data • Following three features are calculated from each sample image features feature calculation dictionary • Mesh features • Peripheral features • PDC features

  8. features result comparison Dictionary data Character recognition process Given image Feature calculation

  9. Majority vote Character recognition process • We classify the given image with each feature and we use the majority vote Result from mesh features Result from peripheral features Result from PDC features

  10. Touching characters • We assume a character which has a low score of similarity as a touching character ‘(’ /0.980 ‘y’ /0.990 ‘O’/0.847 Result/Score

  11. Blurring the image Calculate minimal points Estimate cutting lines Classification Comparison Touching character segmentation(1)

  12. Make projection profile Projection profile |hi– hi+1 | > θ Recognize Touching character segmentation (2) Image

  13. Segmentation experiment • 47 touching characters found in our experimental data

  14. Touch with fraction bar Correct result • Correct examples

  15. Three touching characters Other types Errors • Errors

  16. Recognition experiment • Number of symbols : 12659 • We excluded touching characters • We distinguished following similar shape characters

  17. Recognition rate

  18. Recognition rate • Similar shaped characters

  19. Examples of recognition errors • Most errors occurred at small characters such as scripts

  20. Our previous methods（１） • Projection profile cutting

  21. Core symbol in subexpression Our previous methods（２） • Specific structure processing(Bottom-up) • Script • Root • Matrix • Fundamental structure processing(Top-down) • Vertical division by symbols • Horizontal division by symbols • Horizontal division by blank space

  22. Outline of structure recognition Image Target symbol Top to bottom Character recognition [symbol = fraction,root,matrix] [symbol = script,limit] Structure recognition* Group A processing Group B processing Output Recursion Horizontal connection Output in LaTeX/mathML

  23. Matrix Recognition Target symbol Target symbol Structure Recognition (1/2) • Fractions • Roots • Matrices

  24. Target symbol Target symbol Adjacent symbol Adjacent symbol Structure Recognition (2/2) Scripts Limits

  25. Horizontal Overlap Vertical Overlap Matrix Recognition

  26. Left Parenthesis Right Edge Horizontal Overlap Vertical Overlap Case-distinction

  27. <msubsup rect="1,1,209,210"> Positional Information Answer Database Format <mrow> <msubsup rect="1,1,209,210"> <mrow> <mo>(</mo> <mfrac rect="43,11,87,187"> <mrow> <mi>&beta;</mi> </mrow> <mrow> <mi>&alpha;</mi> </mrow> </mfrac rect="43,11,87,187"> <mo>)</mo> </mrow> <mrow> </mrow> <mrow> <mo>(</mo> <msubsup rect="152,24,189,56"> <mi>e</mi> <mrow> <mi>i</mi> </mrow> <mrow> </mrow> </msubsup rect="152,24,189,56"> <mo>)</mo> </mrow> </msubsup rect="1,1,209,210"> <mo>=</mo> . . . Original expression

  28. Correct Recognition Count found Number in original expression (N) Number correctly recognized (C) Scripts 2 1 Fractions 1 1 found Recognition rate = C / N not found Comparison between Results and Answers (a) Original expression (b) Recognition result <mrow> <msubsup rect="1,1,209,210"> <mrow> <mo>(</mo> <mfrac rect="43,11,87,187"> <mrow> <mi>&beta;</mi> </mrow> <mrow> <mi>&alpha;</mi> </mrow> </mfrac rect="43,11,87,187"> <mo>)</mo> </mrow> <mrow> </mrow> <mrow> <mo>(</mo> <msubsup rect="152,24,189,56"> <mi>e</mi> <mrow> <mi>i</mi> </mrow> <mrow> </mrow> </msubsup rect="152,24,189,56"> <mo>)</mo> </mrow> </msubsup rect="1,1,209,210"> <mo>=</mo> . . . <mrow> <mrow> <mo>(</mo> <mfrac rect="43,11,87,187"> <mrow> <mi>&beta;</mi> </mrow> <mrow> <mi>&alpha;</mi> </mrow> </mfrac rect="43,11,87,187"> <mo>)</mo> </mrow> <mrow> <mo>(</mo> <msubsup rect="152,24,189,56"> <mi>e</mi> <mrow> <mi>i</mi> </mrow> <mrow> </mrow> </msubsup rect="152,24,189,56"> <mo>)</mo> </mrow> <mo>=</mo> . . .

  29. Correct Results (1/4) • limit Arch.Math., Page 44, Vol. 64

  30. Correct Results (2/4) • Multi-fraction Arch.Math., Page 272, Vol. 65

  31. Correct Results (3/4) • Sparse Matrix Original expression Arch.Math., Page 277, Vol. 64 Recognition result

  32. Correct Results (4/4) • Nested case-distinction Original expression Arch.Math., Page 108, Vol. 64 Recognition result

  33. Errors (1/2) • Matrix Original expression Arch.Math., Page 65, Vol. 24 Recognition result

  34. Errors (2/2) • Case-distinction Original expression Arch.Math., Page 104, Vol. 64 Recognition result

  35. Inapplicable expressions (1)

  36. Inapplicable expressions (2)

  37. Inapplicable expressions (3)

  38. Structure Recognition Rate

  39. Summary of structure recognition • Extension of recognition method • Matrix and case-distinction • Performance evaluation • Quantitative evaluation for a large number of expressions • Automatic calculation of recognition rate for each typical structure

More Related