1 / 30

Pierre-Yves Burgi IT Division, University of Geneva, Switzerland

Content-based image retrieval integrated into Fedora. Pierre-Yves Burgi IT Division, University of Geneva, Switzerland Patrick Monbaron & Nastaran Fatemi University of Applied Sciences, Yverdon, Switzerland. Presentation outline. Context MPEG-7 Data migration Indexing Image retrieval

lorand
Download Presentation

Pierre-Yves Burgi IT Division, University of Geneva, Switzerland

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Content-based image retrievalintegrated into Fedora Pierre-Yves Burgi IT Division, University of Geneva, Switzerland Patrick Monbaron & Nastaran Fatemi University of Applied Sciences, Yverdon, Switzerland

  2. Presentation outline • Context • MPEG-7 • Data migration • Indexing • Image retrieval • Demo • Conclusions • Perspectives Context MPEG7 Migration Indexing Retrieval Demo Conclusions Perspectives

  3. Context of the project • Paradigm shift • Migration of image collections from Oracle DB to Fedora • First step: data synchronization • Second step: user interface targeting data retrieval • Third step: user interface for ingesting images (through Valet) • Why Fedora? • Conceptually rich (objects, datastreams, SOA, etc.) • Based on open standards (e.g. XML) • Convenient for adding datastreams such as MPEG-7 Context MPEG7 Migration Indexing Retrieval Demo Conclusions Perspectives

  4. Old (relational) object model Oracle Object Model 4 Objectifs Fedora Migration Indexing Retrieval Demo Conclusions Perspectives Context MPEG7 Migration

  5. Fedora Object Model Fedora’s view New object model 5 Objectifs Fedora Migration Indexing Retrieval Demo Conclusions Perspectives Context MPEG7 Migration

  6. Why MPEG-7 ? (and not DC) • Makes easier migration from the DB • Match from DB’s 21 fields to MPEG-7 possible • Fits more image description • dc:creator versus DS Creator with <Role> & <Agent> • <VisualDescriptor>, <MediaFormat>, etc. • Exif metadata • Etc. 6 Context MPEG-7 MPEG7 Migration Indexing Retrieval Demo Conclusions Perspectives

  7. What is MPEG-7 ? 7 Context MPEG-7 MPEG7 Migration Indexing Retrieval Demo Conclusions Perspectives

  8. Scalable color descriptor Edge histogram descriptor Color layout descriptor Caliph and Emir http://sourceforge.net/projects/caliph-emir/ MPEG-7: feature extraction 8 MPEG-7 Context MPEG7 Migration Indexing Retrieval Demo Conclusions Perspectives

  9. MPEG-7: encoding 9 Context MPEG-7 MPEG7 Migration Indexing Retrieval Demo Conclusions Perspectives

  10. MPEG-7: retrieving Match is expressed as a number 10 Context MPEG-7 MPEG7 Migration Indexing Retrieval Demo Conclusions Perspectives

  11. XML XSLT FOXML Image Data migration Oracle DB Record reading Application server Fedora MPEG7 + FOXML API-M gSearch XSLT FOXML Messaging Datamigration indexing Lucene File system Context MPEG7 Migration Indexing Retrieval Demo Conclusions Perspectives 11

  12. Data synchronisation • 3 Phases • Delete obsolete objects • « Delete » in Fedora those recordings which do not exist anymore in Oracle DB • Update objects • Update in Fedora objects corresponding to recordings modified in Oracle DB • Create new objects • Create new objects in Fedora corresponding to the new recordings present in Oracle DB • Algorithm based on the date of the last batch • Date saved within a configuration file Context MPEG7 Migration Indexing Retrieval Demo Conclusions Perspectives 12

  13. Migration: delete objects Compare both lists and « delete » from Fedora missing elements in Oracle Reading of the id of all Fedora objects. Reading of the id of all Oracle recordings 13 Context MPEG7 Migration Indexing Retrieval Demo Conclusions Perspectives

  14. Migration: create new objects For new objects: Reading of metadata and image analysis Data migration into XML format and into MPEG-7 and FOXML through XSLT Read date of last object creation Search id of those elements created since last update Ingest FOXML and images into Fedora 14 Context Fedora Migration Indexing Retrieval Demo Conclusions Perspectives

  15. Migration: update objects Read date of last object creation Search id of those elementsupdated since last update Ingest MPEG-7 into Fedora Reading of metadata and image analysis Data migration into XML format and into MPEG-7 15 Context Fedora Migration Indexing Retrieval Demo Conclusions Perspectives

  16. Indexing • Indexing of • DC • Textual MPEG-7 metadata • Not indexed but saved in the index • Technical MPEG-7 metadata • Image attributes 16 Context Fedora Migration Indexing Retrieval Demo Conclusions Perspectives

  17. XSLT FOXML Indexing • Indexing process within Lucene Oracle DB Record reading Application server XML XSLT Fedora FOXML MPEG7 + FOXML gSearch API-M Messaging Indexing Image Data migration File system Lucene 17 Context Fedora Migration Indexing Retrieval Demo Conclusions Perspectives

  18. Fedora API-A gSearch XSLT Resolver Indexing Image retrieval Application server User interface HTML page generation Request parsing XSLT Search by image matching gSearch connector File system Lucene 18 Objectifs MPEG-7 Migration Indexing Retrieval Demo Conclusions Perspectives

  19. Retrieval by image matching Image reference id Retrieval of the attributes of this image from Lucene XML output for display Reading of all data from Lucene and conversion into XML Selection (10 first) Match of each image attribute with that reference and computation of a matching score 19 Context Fedora Migration Indexation Retrieval Demo Conclusions Perspectives

  20. Demo: Photothèque 20 Context Fedora Migration Indexation Retrieval Demo Conclusions Perspectives

  21. Conclusions • Applications of the MPEG-7 standard to image matching retrieval give satisfactory results … but this is not terribly semantic! • For large image data bank indexation method might not be optimal (now about 1’000 images) • Parameter tuning of image attributes not easy • Image corpus too small to establish benchmarks • Migration procedure from a DB to Fedora tested • Data synchronization remains a difficulty • Integration within Fedora otherwise is a success • Culture change (paradigm shift) in progress 21 Context Fedora Migration Indexation Retrieval Demo Conclusions Perspectives

  22. Perspectives • Exploiting more Fedora’s features • Using disseminator for watermarking images on the fly • Applying XACML policies • Improving user interface • Play with search parameters (shape versus color) • Applying the methodology to other image data banks • Medicine, architecture, etc. • Other media (audio, video) 22 Context Fedora Migration Indexation Retrieval Demo Conclusions Perspectives

  23. Questions Complete report available in French at: http://www.unige.ch/dinf/ntice/accueil/MembresProjet/PMonbaronRapportFinal.pdf 23 Context Fedora Migration Indexation Retrieval Demo Conclusions Perspectives

  24. « building » (key word) 24 Context Fedora Migration Indexation Retrieval Demo Conclusions Perspectives

  25. Retrieval results 25 Context Fedora Migration Indexation Retrieval Demo Conclusions Perspectives

  26. Display of chosen image 26 Context Fedora Migration Indexation Retrieval Demo Conclusions Perspectives

  27. Retrieval by image matching 27 Context Fedora Migration Indexation Retrieval Demo Conclusions Perspectives

  28. Retrieval results (by contour matching) 28 Context Fedora Migration Indexation Retrieval Demo Conclusions Perspectives

  29. Display of an other chosen image 29 Context Fedora Migration Indexation Retrieval Demo Conclusions Perspectives

  30. Retrieval results (by color matching) 30 Context Fedora Migration Indexation Retrieval Demo Conclusions Perspectives

More Related