1 / 0

Autonomous Cleaning of Corrupted Scanned Documents A Generative Modeling Approach

Autonomous Cleaning of Corrupted Scanned Documents A Generative Modeling Approach. Zhenwen Dai J ӧ rg Lücke Frankfurt Institute for Advanced Studies, Dept. of Physics, Goethe-University Frankfurt. A document cleaning problem. What method can save us?.

skip
Download Presentation

Autonomous Cleaning of Corrupted Scanned Documents A Generative Modeling Approach

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Autonomous Cleaning of Corrupted Scanned DocumentsA Generative Modeling Approach

    Zhenwen Dai JӧrgLücke Frankfurt Institute for Advanced Studies, Dept. of Physics, Goethe-University Frankfurt
  2. A document cleaning problem
  3. What method can save us? Optical Character Recognition (OCR)
  4. OCR Software input OCR (FineReader 11) ? ? vs. Character Segmentation Character Classification
  5. What method can save us? Optical Character Recognition (OCR) Automatic Image Inpainting
  6. Automatic Image Inpainting
  7. Automatic Image Inpainting Unable to identify the defects because corruption and characters consist of same features solution requires knowledge of explicit character representations
  8. What else? Optical Character Recognition (OCR) Automatic Image Inpainting Image Denoising? … Problem requires a new solution!
  9. Our Approach training data is only the page of corrupted document no label information a limited alphabet (currently) input our approach
  10. How does it work without supervision? Characters are salient self-repeating patterns. Corruptions are more irregular. Related to Sparse Coding input our approach
  11. The Flow of Our Approach b s Learning a e A Character Model on Image Patches y Cut into Image Patches Character Detection & Recognition
  12. A Probabilistic Generative Model Show a character generation process. A character representation (parameters) Feature Vectors (RGB color) mask param.
  13. A Tour of Generation 0.2 0.2 0.2 0.2 Prior Prob. 0.2 Select a character. Translate to the position. Generate a background. Overlap character with background according to mask. masks features Pixel-wise Background Distribution Translation by [12,10]T Learning
  14. Maximum Likelihood Iterative Parameter Update Rules from EM: prior prob. posterior tn t2 t1 t0 parameter set std A posterior distribution is needed for every image patch in the update rules.
  15. Posterior Computation Problem A posterior distribution is needed for every image patch in the update rules. Similar to template matching A pre-selection approximation Which character? A ? B ? C ? D ? E ? inference Where? ? ? ? hidden space (truncated variational EM) pre-selection (Lücke & Eggert, JMLR 2010) (Yuille & Kersten, TiCS 2006)
  16. An Intuitive Illustration of Pre-selection Select some local features according to parameters. Very few features A number of good guesses A B C D E C A C E E B B D D A Features in image patches B (Lücke & Eggert, JMLR 2010) B D (Yuille & Kersten, TiCS 2006)
  17. Learn the Character Representations Input: image patches (Gabor wavelets) A learning course: (about 25 mins) chars mask feature std chars mask feature std feature std 1 4 2 5 3 6 (heat map) (heat map)
  18. Learn the Character Representations Input: image patches (Gabor wavelets) A learning course: (about 25 mins) chars mask feature std chars mask feature std feature std 1 4 2 5 3 6 (heat map) (heat map)
  19. Document Cleaning How to recognize characters against noise? Character segmentation fails. Our model – one char per patch It is a non-trivial task. Try to explore from the model as much as possible.
  20. Document Cleaning Procedure Inference of every patch with the learned model Paint a clean character at the detected position. Erase the character from the original document. Accept original Fully visible=1 Clean Characters from the Corrupted Document reconstructed reconstructed
  21. Document Cleaning Procedure Inference of every patch with the learned model Iterate until no more reconstruction. more than one character per patch Accept Reject Fully visible=0 Fully visible=1 original Accept Reject Fully visible=0 Fully visible=1 Reject Accept Fully visible=0 Fully visible=1 reconstructed reconstructed reconstructed Accept Accept Fully visible=1 Fully visible=1 Reject Accept Fully visible=1 Fully visible=1 iteration 1 iteration 2 (about 1 min per iteration)
  22. Before Cleaning
  23. After Iteration 1
  24. After Iteration 2
  25. After Iteration 3
  26. More Experiments Rotated, random placed More characters (9 chars) Unusual character set (Klingon) Irregular placement (randomly placed, rotated) Occluded by spilled ink 9 chars Klingon Occluded original reconstructed
  27. Recognition Rates
  28. False Positives
  29. Not only a Character Model Detect and count cells on microscopic image data in collaboration with ThiloFigge and Carl Svensson
  30. Summary Addressed the corrupted document cleaning problem. Followed a probabilistic generative approach. Autonomous cleaning of a document is possible. Demonstrated efficiency and robustness. The dataset will be available online soon. Future directions: Extended to large alphabet by incorporating prior knowledge of documents. Extended to various different applications.
  31. Acknowledgement http://fias.uni-frankfurt.de/cnml
  32. Thanks for your attention!
  33. Performance
  34. Document Cleaning Procedure Character vs. Noise? MAP inference can only choose among learned characters. Define a novel quality measure. y a Threshold: 0.5 MAP mask param. mask posterior difference
More Related