Segmentation for Handwritten Documents

1. Segmentation for Handwritten Documents Omar Alaql Fab. 20, 2014

2. Outline • Optical Character Recognition (OCR). • OCR for the Historical Documents. • Text Lines Segmentation Approaches. • Profile Projection. • Hough Transform. • Level Set Method. • Affinity Propagation. • Steerable Directional Technique.

3. Optical Character Recognition (OCR) • The electronic translation of images into machine-editable text. Input Image Text

4. Optical Character Recognition (OCR) • There are four major stages which must be done in any optical characters recognition: • Preprocessing. • Segmentation. • Feature extraction. • Recognition.

5. Optical Character Recognition (OCR) • Preprocessing: • Noise reduction. • Binarizationor Gray scale image. • Compression in the amount of data to be analyzed.

6. Optical Character Recognition (OCR) • Segmentation: • The isolation of various writing units, such as paragraphs, sentences, words, or letters. Text Lines Segmentation

7. Optical Character Recognition (OCR) • Representation: • Extracts the most relevant information from the text image which helps the recognition stage to recognize the text. • This information is the features of each symbol that is needed to distinguish it from other symbols.

8. Optical Character Recognition (OCR) • Recognition: • Recognition stage is the last and the main decision making stage. • It is a classification process that identifies each unknown symbol and assigns it into a predefined class. • This classification is based on the extracted features which are the output of the previous stage.

9. OCR for the Historical Documents • Historical documents processing is a challenging task for various reasons: 1) Lack of standard alphabets and presence of unknown fonts. 2) Low quality.

10. OCR for the Historical Documents 3) The lack of constraints on page layout.

11. OCR for the Historical Documents 4) The complexity of handwriting. 5) The variability of skew between the different text-lines and within the same text-line.

12. OCR for the Historical Documents 6) Spaces between lines are narrow and variable. 7) The existence of small components. 8) Distinguishing noise from text. Small Components Narrow lines Noise

13. Text Lines Segmentation Approaches • There are many techniques for text lines segmentation: • Profile Projection. • Hough Transform. • Level Set Method. • Affinity Propagation. • Steerable Directional Technique.

14. Projection Profile • Summing pixel values along the horizontal axis for each y value. Horizontal Projection

15. Projection Profile • Example: • Input image.

16. Projection Profile • Example: • Skew Correction.

17. Projection Profile • Example: • Horizontal Projection.

18. Projection Profile • Example: • Peaks detection

19. Projection Profile • Example: • Positions for segmentation.

20. Projection Profile • Example: • Image for each text line.

21. Projection Profile • For skewed or fluctuating text lines, the image may be divided into vertical strips . • Subdivision the page into columns. • Determination of the minimal values of the histograms resulting from horizontal projections for all the columns. • Drawing horizontal stroke by means of each minimal value inside a column. • The link between these strokes allows the separation of two adjacent lines.

22. Projection Profile Partial Projection Method

23. Hough Transform • The Hough transform is used for locating straight lines in images. • Text line is best align matches the black pixels. • Any black pixel has an infinite number of lines that could pass through this pixel.

24. Hough Transform • There are two ways to represent the lines : • y = mx + c • x cos θ + y sin θ = ρ • Each line has a unique value (m , c) or (ρ, θ) which is called accumulator. • There is a vote for the accumulator when the line passes through a black pixel. • The text line is the line that has the maximum accumulator.

25. Level-set Method • Instead of directly segmenting on a binary image, it is converted to a probability map, where each element represents the probability of this pixel belonging to a text line. Input image Probability Map

26. Level-set Method • The probability map is analyzed using the level set method to segment text lines by determining the boundary of neighboring text lines. • The zero value for the boundary, automatically grows, merges, and stops to the final text line boundary. By Level Set, text lines are horizontally elongated Result after 10 iterations Initial estimate of text lines

27. Connected Components Clustering • Grouping many connected components in a cluster by using grouping algorithms, each cluster represents a separate text line. Connected Components

28. Affinity Propagation • The algorithm first estimates local orientation at each primary component of a word to build a sparse similarity graph. • At each point, the region is divided into five regions. • The Breadth-First Search algorithm is applied to find disjoint sets in the similarity graph. • There exist a path from each element to every other element in the set.

29. Steerable Directional Local Profile Technique • One of the connected components based approaches is steerable directional technique. • Adaptive local connectivity map (ALCM) is generated using a steerable directional filter.

30. Steerable Directional Local Profile Technique • Firstly, a steerable filter is used to determine foreground intensity along multiple directions at each pixel while generating the ALCM. Steerable Filter Text image ALCM

31. Steerable Directional Local Profile Technique • The ALCM is then binarized using an adaptive thresholding algorithm to get a rough estimate of the location of the text lines. Binarization ALCM Text Line Location

32. Steerable Directional Technique • This approach has difficulties and limitations when it comes to the binarization of the ALCM images. • Especially when text lines in the document are very close to each other.

33. Steerable Directional Technique • To solve the problem: 1) Steerable dynamic directional filter is applied. Angle value is taken instead of the density value. Input image Text Direction Map

34. Steerable Directional Technique 2) apply a mode filter to extract each paragraph in the document and its orientation. Paragraph Map

35. Steerable Directional Technique Input Image Paragraph Map

36. Steerable Directional Technique 3) a steerable static directional filter is applied. - the direction of the kernel is taken from the paragraph map.

37. Steerable Directional Technique 4) Thresholding Text lines patterns

38. Horizontal Projection Technique • To use Projection Technique: • First : paragraph segmentation. Paragraph Map Paragraphs Segmented

39. Horizontal Projection Technique • To use Projection Technique: • Second: Skew Correction. Paragraphs Segmented After skew correction

40. Horizontal Projection Technique • To use Projection Technique: • Third: Horizontal Projection.

41. Horizontal Projection Technique • To use Projection Technique: • Fourth: Profile Analysis. • There are some drawbacks makes finding he maximum and the minimum in the profile more complicated. • Short line will provide low peak that might be ignored • very narrow lines, or the lines that including many overlapping components will not produce significant peaks

42. Horizontal Projection Technique • To use Projection Technique: • Fourth: Profile Analysis. • To solve this problem, the profile should be smoothed.