1 / 62

Talk workplan

Pattern Recognition and Image Analysis Group (RFAI) Document (Image) Analysis related work Laboratory of Computer Science (LI) François Rabelais University Tours city, France. Talk workplan. Tours city François-Rabelais University, les deux lions / Portalis

lyndon
Download Presentation

Talk workplan

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Pattern Recognition and Image Analysis Group (RFAI)Document (Image) Analysis related workLaboratory of Computer Science (LI)François Rabelais UniversityTours city, France

  2. Talk workplan • Tours city • François-Rabelais University, les deux lions / Portalis • School of Engineering Polytech’Tours 4. Laboratory of Computer Science 5. RFAI group 6. DIA related work 6.1. Projects & partners outline 6.2. Layout analysis and document recognition 6.3. OCR, word spotting and signature verification 6.4. Symbol recognition & spotting 6.5. Content Based Image Retrieval 6.6. Camera based recognition 6.7. Graph matching and embedding

  3. Tours city Paris Tours - 137 046 people (2009) - 204 km southwest of Paris - Region «  centre - Indre et loire » - 1h20 from Paris by high speed train - Direct train connection to Charles de Gaulle -Orlyairport in 2h00

  4. Talk workplan • Tours city • François-Rabelais University, les deux lions / Portalis • School of Engineering Polytech’Tours 4. Laboratory of Computer Science 5. RFAI group 6. DIA related work 6.1. Projects & partners outline 6.2. Layout analysis and document recognition 6.3. OCR, word spotting and signature verification 6.4. Symbol recognition & spotting 6.5. Content Based Image Retrieval 6.6. Camera based recognition 6.7. Graph matching and embedding

  5. François-Rabelais University,les deux lions / Portalis François Rabelais University François Rabelais i.e. a famous French writer of XV° Century

  6. Talk workplan • Tours city • François-Rabelais University, les deux lions / Portalis • School of Engineering Polytech’Tours 4. Laboratory of Computer Science 5. RFAI group 6. DIA related work 6.1. Projects & partners outline 6.2. Layout analysis and document recognition 6.3. OCR, word spotting and signature verification 6.4. Symbol recognition & spotting 6.5. Content Based Image Retrieval 6.6. Camera based recognition 6.7. Graph matching and embedding

  7. School of Engineering Polytech - 12 schools in France (Grenoble, Lille, Marseille, Montpellier, Nantes, Nice-Sophia, Paris-UPMC, Paris ORSAY, Savoie, Orléans, Tours, Clermont-Ferrand) - 12 000 students • 720 students • - 5 departments (with Labs) Urban Planning CITERES Mechanics LMR Electronics LMP Computer Science LI Embedded computing

  8. Talk workplan • Tours city • François-Rabelais University, les deux lions / Portalis • School of Engineering Polytech’Tours 4. Laboratory of Computer Science 5. RFAI group 6. DIA related work 6.1. Projects & partners outline 6.2. Layout analysis and document recognition 6.3. OCR, word spotting and signature verification 6.4. Symbol recognition & spotting 6.5. Content Based Image Retrieval 6.6. Camera based recognition

  9. Laboratory of Computer Science 77 people, 5 research groups (2009) Data Bases and Natural Language Processing Pattern Recognition and Image Analysis Visual Data Mining and Biomimetic Algorithms Handicap and New Technologies Scheduling and Control

  10. Talk workplan • Tours city • François-Rabelais University, les deux lions / Portalis • School of Engineering Polytech’Tours 4. Laboratory of Computer Science 5. RFAI group 6. DIA related work 6.1. Projects & partners outline 6.2. Layout analysis and document recognition 6.3. OCR, word spotting and signature verification 6.4. Symbol recognition & spotting 6.5. Content Based Image Retrieval 6.6. Camera based recognition

  11. Pattern Recognition and Image Analysis (RFAI) (1) Medical Imaging - Image segmentation (ultrasound, MRI) - Video analysis, 3D reconstruction Document Image Analysis - Layout analysis & document recognition - OCR, word spotting & signature verification - Symbol recognition & spotting - Content based Image Retrieval - Camera based Recognition - Graph matching and embedding Machine learning for time series prediction

  12. Pattern Recognition and Image Analysis (RFAI) (2) Professors Hubert Cardot Jean-Yves Ramel Romuald Boné Alireza Alaei Thierry Brouard Sabine Barrat Muzzamil Luqman Mathieu Delalandre Romain Raveaux PhD Partha Roy Gilles Verley Nicolas Ragot Julien Olivier Nicolas Sidere Pascal Makris 12

  13. Pattern Recognition and Image Analysis (RFAI) (3) Aymen Cherif Fareed Ahmed Ahmed Ben Salah Cyrille Faucheux PhD Students & engineers The Anh Pham Frédéric Rayar Anh Khoi Ngo ho 13

  14. Talk workplan • Tours city • François-Rabelais University, les deux lions / Portalis • School of Engineering Polytech’Tours 4. Laboratory of Computer Science 5. RFAI group 6. DIA related work 6.1. Projects & partners outline 6.2. Layout analysis and document recognition 6.3. OCR, word spotting and signature verification 6.4. Symbol recognition & spotting 6.5. Content Based Image Retrieval 6.6. Camera based recognition

  15. Projects & partners outline (1)

  16. Projects & partners outline (2) National projects Centre de Recherche en Informatique de Paris 5 (Paris) Institut de Recherche en Informatique et Systèmes Aléatoires (Rennes) Centre d’Etude Supérieures de la Renaissance (Tours) Laboratoire Lorrain de Recherche en Informatique et ses Applications (Nancy) Laboratoire d'Informatique de Traitement de l'Information (Rouen) Laboratoire d'InfoRmatique en Image et Systèmes d'information (Lyon) Laboratoire d’informatique image et interaction (La Rochelle) Laboratoire Bordelais de Recherche en Informatique (Bordeaux) Laboratoire Informatique (Tours)

  17. Projects & partners outline (3) Partnership contracts Local government projects (i.e. projets region centre) So famous ! Centre des études supérieures de la renaissance – bibliothèque virtuelle humaniste Maison des Sciences de l'Homme PhD Scholarships international high-technology group in aerospace, defense and security Bibliothèque Nationale de France - portail Gallica Bilateral program Digitalisation company capturing, automatically processing, and managing all company’s incoming documents Atos Origin is a leading international IT services provider for business solutions

  18. Projects & partners outline (4) Computer Vision Center Document Analysis Group Barcelona - Spain “J. Llados, E. Valveny” Dept. of Computer Science and IS Osaka Prefecture University Osaka - Japan “K. Kise” Indian Statistical Institute Kolkata - India “U. Pal” Computational Intelligence Laboratory Athens - Greece “B. Gatos”

  19. Projects & partners outline (5)

  20. Talk workplan • Tours city • François-Rabelais University, les deux lions / Portalis • School of Engineering Polytech’Tours 4. Laboratory of Computer Science 5. RFAI group 6. DIA related work 6.1. Projects & partners outline 6.2. Layout analysis and document recognition 6.3. OCR, word spotting and signature verification 6.4. Symbol recognition & spotting 6.5. Content Based Image Retrieval 6.6. Camera based recognition

  21. Layout analysis & document recognition“AGORA (1)” • (1) Text/graphics separation • Foreground map: adaptive binarization [Saul2000] with connected component labeling, text/graphics separation is done in terms of size of connected components • (2) Line/word segmentation • 1. Background map: statistical distribution of white and black pixel on horizontal and vertical scanline • 2. Fusion: word segmentation (i.e. connected components grouping) is done in terms of thresholding on the background map.

  22. Layout analysis & document recognition“AGORA (2)” • (3) Interactive system (i.e. user driven analysis) Vertical position Horizontal position average= 0,46 stddeviation= 0,41 • (4) Results, since 2005: 300 books (50 000 pages) • http://www.bvh.univ-tours.fr/ average= 0,51 stddeviation= 0,07 22

  23. Layout analysis & document recognition“Document image characterization (1)” (1) Descriptor based on five features:

  24. Layout analysis & document recognition“Document image characterization (2)” • (2) Segmentation: • features are extracted at four different resolution (45 = 20 features) • features are then processed with the clustering algorithm CLARA (Clustering LARge Applications) [Kaufman1990] to achieve automatic segmentation in text/graphics/background

  25. Layout analysis & document recognition“Document image characterization (3)” • (3) Indexing applied on two different problems • Layout retrieval, distance is based on a contingency table [Younes2004] • Graphics retrieval, distance based on a dissimilarity function Handmade dataset I (400 images) Handmade dataset II (400 images)

  26. Layout analysis & document recognition “Cognitive digitalization” Topic: Incremental and interactive learning for document image, application for intelligent cognitive scanning of old documents. Problematic: - Estimate the scan parameters according to usage, past experience. - Improve the scan parameters for a document during the scanning. - Detect the default settings for a document, a collection, a work.

  27. Layout analysis & document recognition “Document classification” Form Publicity • Topic: Recognition of administrative forms for companies • Problematic: • - high variability “600 to 800 classes” • binary images at 300 dpi • time constraint:  to 1,5 s per image • commercial systems can’t outperform • a 60% recognition rate • Goals: • 1. To gain in robustness (set of adapted • and robust specialists) • 2. To gain in flexibility (self • learning, content adaptation) Free letter Acknowledge reply

  28. Talk workplan • Tours city • François-Rabelais University, les deux lions / Portalis • School of Engineering Polytech’Tours 4. Laboratory of Computer Science 5. RFAI group 6. DIA related work 6.1. Projects & partners outline 6.2. Layout analysis and document recognition 6.3. OCR, word spotting and signature verification 6.4. Symbol recognition & spotting 6.5. Content Based Image Retrieval 6.6. Camera based recognition

  29. OCR, word spotting and signature verification “Robust OCR-I (1)” Key idea : improving OCR robustness by using similar technics as those used for handwriting recognition: - Hidden Markov Models without explicit segmentation - Adapting a polyfont OCR to specificities of pages (fonts/noise) (1) Feature extraction is based on a sliding window and HoG features (no word/character segmentation) Sliding window initial model • (2) HMM classification and training • HMM characters models are learnt on a synthetic dataset (numerous fonts, degradations possible, no limits in the number of samples per character) = > polyfont OCR system • Each character model can be adapted to a specific font/book using only few lines of transcriptions. The HMM model is adapted at the structure level (number of states) and at the parameter level (Gaussian MAP adaptation). Structure adaptation New model training

  30. OCR, word spotting and signature verification “Robust OCR-I (2)” Experiments done using 100 fonts with the degradation model of Baird blurred thresholding sparse pixels

  31. OCR, word spotting and signature verification “Robust OCR-II” • Topic: digitalization and indexing of a military document database for retired pay • Problematic: • large amount of data (800 000 applications every 3 years) • large heterogeneity: • from XIX° century “middle” to today, • handwritten and typographic documents, • different languages, • no common layout, • different colors, • etc.

  32. OCR, word spotting and signature verification “Performances prediction/control of OCR” • Problematic: control and cost reduction of the digitization service to know which collection/document/part of document is OCRisable and at which quality • Select only adequate documents to be sent to the private service provider in charge of the digitization and OCRisation • Studies of relationships between meta-data information (date, format, …) and OCR results => difficult without deep analysis of the pages • Characterization of image content with SIFT+LBP; regression towards OCR results • Control of OCR quality assessed by the service provider • Detection of text zones forgotten by OCR using correct detection performed (contextual information) => in progress • Verification of OCR result by matching with image (=> in a near futur)

  33. OCR, word spotting and signature verification“Semi-automatic transcription (1)” Topic: user driven transcription of character in historical books • (1) Segmentation process based on Agora Standardized output (e.g. Alto) • (2) Clustering process • Finer description of shapes • Features extraction and selection (3) Transcription & Typography studies

  34. OCR, word spotting and signature verification“Semi-automatic transcription (2)” Experiments, The Vésalesbook Reasons is noise spot touching characters character on verso split character

  35. OCR, word spotting and signature verification“Word Spotting (1)” Text Extraction Topic: Word Retrieval in Historical Documents AGORA Transcriptionbycodebook Manuscripts Codebook of Primitives Query word Sequence of primitives PrimitiveString Matching Word Detection

  36. OCR, word spotting and signature verification“Word Spotting (2)” Topic: Word Retrieval in Historical Documents The codebook is created using a clustering algorithm by template matching of similarity Overcoming of segmentation problems are solved by the Water reservoir method. Query word is thus converted into a string of primitives. Approximate string matching algorithm is used for string matching • Tests done from • - 24 pages • corresponding to 57324 primitives • clustered in 183 representative primitives • P/R computed with 20 query word images

  37. OCR, word spotting and signature verification “MultlingualWord Spotting” • Topic:Robustmultilingualwordspotting: • Problematic: • Query by text/image • Partial matchingallowed (for occlusion, specialcharacters) • Matching in twosteps : global (shapecontext) / local (HMM)

  38. OCR, word spotting and signature verification “Online signature verification (1)” Problematic: to evaluate impact of temporality (i.e. time evolution) on signature, for performance evaluation of signature verification algorithms. (1) Database acquisition Training loop if rough differences (based on length, duration, speed) Enrollment “5 signatures” Final acquisition “5 signatures”

  39. OCR, word spotting and signature verification “Online signature verification (2)” (2) Statistical analysis (2.1) Global i.e. without temporal variability (2.1) With temporal variability Total duration per signer/ session Total length per signer/ session (3) Performance evaluation Authentication (i.e. recognition) algorithm based on a Coarse to fine approach - Coarse step on “basic” features (length, duration) - Fine step based on DTW Proposed dataset Dataset without temporality

  40. Talk workplan • Tours city • François-Rabelais University, les deux lions / Portalis • School of Engineering Polytech’Tours 4. Laboratory of Computer Science 5. RFAI group 6. DIA related work 6.1. Projects & partners outline 6.2. Layout analysis and document recognition 6.3. OCR, word spotting and signature verification 6.4. Symbol recognition & spotting 6.5. Content Based Image Retrieval 6.6. Camera based recognition

  41. Symbol recognition & spotting“Vectorization and GbR (1)” (1) Contour detection, chaining and polygonalisation [Wall1984] (2) Quadrilateral building (2.1.) Matching (2.2.) Sorting (2.3.) Merging 41

  42. Symbol recognition & spotting“Vectorization and GbR (2)” (3) Graph based representation (3) Pros and cons Cons - lost of connectivity Pro - better representation of filled & crossed areas Cons - parasite quadrilaterals

  43. Symbol recognition & spotting“Generation of synthetic documents (1)” C1 M1 c1 C2 M2 M3 c2 C3 M4 Key idea C4 To use a same background layer with different symbol layers Graphical documents are composed of two layers p L  (1) Constraint model p1 loaded symbol symbol model bounding box and control point p2 L1 L2 θ2 θ1 alignment 43

  44. Symbol recognition & spotting“Generation of synthetic documents (2)” Symbol Models (2) run (2) Building engine and user interaction Background Image Building Engine (1) edit (3) display 44

  45. Symbol recognition & spotting“Generation of synthetic documents (3)” (3) Datasets Mean localization results (4) Performance evaluation - Goal is to evaluate variability impact of produced datasets on spotting system(s) - Experiments have been done from the spotting system of R. Qureshi Background sets

  46. Symbol recognition & spotting“Graph scoring for symbol spotting (1)” (1) Graph based representation: based on the Jean-Yves Ramel’s work (2) Seeds detection in graph: a set of scoring functions is computed from all nodes and edges (3) Score propagation: based on a shortest path algorithm, a global score is normalized from individual score of edge/node 46

  47. Symbol recognition & spotting “Graph scoring for symbol spotting (2)” (4) Results & performance evaluation 1 0 SESYD dataset Precision 47 Recall

  48. Symbol recognition & spotting“Bayesian based system for symbol spotting (1)” (1) Representation phase: used the graph based representation of Jean-Yves Ramel (2) Description phase: approach based on attributes (of nodes and edges) (3) Learning and classification phases base on Bayesian network (3.1.) Discretization step: based on the Akaike Information Criterion (3.2.) Learning step: - network topology is done from a genetic algorithm - parameters conditional probabilities is done from a maximum likelihood estimation (3.3.) Classification step: 48

  49. Symbol recognition & spotting“Bayesian based system for symbol spotting (2)” (4) Performance evaluation at recognition level ISRC 2003 dataset clean Hand drawn Binary degrade SESYD dataset (5) Improvements of Rashid Qureshi’s results Precision 49 Recall

  50. Symbol recognition & spotting“Graph Embedding” Topic: Topological Graph Embedding (1) A lexiconisgeneratedfromthe network of non-isomorphicgraphs (2) The embedding is based on occurrences of the patterns

More Related