Enhancing Digital Libraries through EGEE-DILIGENT Integration

1. Pasquale Pagano, CNR – ISTI on behalf of the DILIGENT Consortium Geneva, CERN, 16th of December 2004

2. DLstoday The demand for DLs has recently grown considerably • DLs are perceived as a necessary instrument to support communication and collaboration among the members of communities of interest • Many application domainsrequire DL services, e.g. e-Health, e-Learning, e- Government • Many of the organizations that demand aDL are small, distributed, and dynamic; they use the DL to support temporary activities such as courses, exhibitions, projects, etc. EGEE - DILIGENT Interaction Meeting

3. The current DL development model • The construction and management of a DL requires high investments and specialized personnel, content production is very expensive, multimedia handling requires high computational resources • Years are spent in designing and setting up a DL • The systems lack interoperability and the services provided are difficult to reuse • This development model is not suitable to satisfy the demand of many organisations EGEE - DILIGENT Interaction Meeting

4. Expectation from the Grid • can supporta cost-effective DL operational model based on transient, flexible, coordinated “sharing of resources” • addresses the main DL architecture requirements: distribution, openness,interoperability, scalability, controlled sharing, availability, security, quality, …. • provides a basic common infrastructure for serving several different application domains • offers high storage and computing capabilities that enable the provision of powerful functionality on multimedia content e.g. images and videos EGEE - DILIGENT Interaction Meeting

5. Diligent Objective Create an advanced Digital Library Infrastructure that will allow members of dynamic virtual e-Science organizationsto access shared knowledge and to collaborate in a secure, coordinated, dynamic and cost-effective way. shared knowledge secure, coordinated, dynamic and cost-effective EGEE - DILIGENT Interaction Meeting

6. DILIGENTPartners Co-ordinators • European Research Consortium for Informatics and Mathematics (ERCIM) • Institute of Information Science and Technology of the Italian National Research Council (CNR-ISTI) Core partners • University of Athens (UoA) • Swiss Federal Institute of Technology Zurich (ETH Zurich) • Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. (FhG/IPSI) • University for Health Informatics and Technology Tyrol (UMIT) • European Organization for Nuclear Research (CERN) • Engineering Ingegneria Informatica SpA (ENG) • University of Strathclyde (USG) • Fast Search & Transfer ASA (FAST) • 4D SOFT Software Development Ltd. (4D SOFT Ltd.) EGEE - DILIGENT Interaction Meeting

7. TechnicalSolution • DILIGENT will be built by integrating Digital Library services on the infrastructure developed by the Enabling Grids for E-science in Europe project (EGEE) [IST-2003-508833] • TheDILIGENT design will be service oriented in order to provide as many reusable components as possiblefor other e-applications that could be created on top of the basic DILIGENT infrastructure • DILIGENT will enhance existing Grid services with the functionalityneeded to support the complex services interactions required to build, operate and maintain transient virtual digital libraries EGEE - DILIGENT Interaction Meeting

8. Technical Solution (cont.) • DILIGENT will integrate and enhance a set of services, developed in DL projects: • submission, indexing and discovery of mixed-media objects; • management and processing of these objects through annotation, composition, cooperative editing, etc. • service composition, optimization, personalisation, etc. • By taking full advantage of the scalable, secure, and reliable Grid infrastructure each DL service of this area will provide an enhanced functionality with respect the equivalent non-Grid-aware service EGEE - DILIGENT Interaction Meeting

9. Exploitation • The DILIGENT service infrastructure will dramatically change the DL development model:many distributed and dynamic organisations that do not have the financial and human resources for managing and maintaining a proprietary DL system will be able to set up their own DL when and for the time they need it by specifying a number of requirements on the information space and on the services • The DILIGENT Grid services will be make available for other Grid-based knowledge infrastructures(e.g. eLearning, eHealth, eGovernment) EGEE - DILIGENT Interaction Meeting

10. Validation Strategy DILIGENT will be demonstrated and validated by two complementary real-life application scenarios: • one from the culture heritage domain • one from the environmental e-Science domain EGEE - DILIGENT Interaction Meeting

11. Scenario Selection Two complementary scenarios • Earth Science Domain • well-established tradition in exploitation of new technologies • wide variety of content types (maps, satellite images, etc.) • very large, dynamic data sets • support of community events, report generation, support in case of disasters • Cultural Heritage Domain • IT technology exploitation still in infancy • multidisciplinary collaborative research • image based retrieval/semantic analysis of images • support for research and teaching EGEE - DILIGENT Interaction Meeting

12. The ImpECt Scenario (1)Implementation of Environmental Conventions • Goal: Improve accessibility, interoperability and usability of environmental data, models, tools, algorithms and instruments • community is represented by European Space Agency (ESA) • observers: e.g. selected European cost-guard offices, REMPEC, UNESCO IOC, ITOPF, ICRAM • use cases for DILIGENT VDLs • organization of conferences • preparation of reports • handling of information in case of environmental accidents EGEE - DILIGENT Interaction Meeting

13. The ARTE Scenario • Goal: stimulate collaborative multidisciplinary scientific research • Community is represented by researchers from project ARTE (funded by MIUR, Italian Ministry for Higher Education and Research), especially SNS and RAI Educational (as content provider) • Observers: e.g. Brown University (Department of Italian studies), Centre de Recherche en Histoire des Sciences et des Techniques Universidade da Coruña (Research Team on Hispanic Emblematic Literature), University of Glasgow (HATII), Università di Pisa (Facoltà di Lettere e Filosofia) • Use cases for DILIGENT VDLs • multidisciplinary research revolving around images and videos • organization of virtual exhibitions EGEE - DILIGENT Interaction Meeting

14. DILIGENT Observers Other organizations that have expressed their interest in participating to this experimentation and to exploit its results are: • NSF-funded National Science Digital Library NSDLproject; • DELOS Network of Excellence on Digital Libraries; • CoreGridEuropean Research Network on Foundations, Software Infrastructures and Applications for large scale distributed, GRID and Peer-to-Peer Technologies • Kettering University; • University of Glasgow – Humanities Advanced Technology & Information Institure; • University of Cyprus – Department of Computer Science; • Università di Pisa, Facoltà di Lettere e Filosofia; • Universidade da Caruña, Grupo de Investigación sobre Literatura Emblemática Hispánica; • Bielefeld University Library; • Hanser & Co GmbH, CPO Hanser Service; • Tiroler Landeskrankenanstalten (TILAK) – Tyrolean Hospital Consortium; • ENEA, Ente per le Nuove tecnologie, l’Energia e l’Ambiente; EGEE - DILIGENT Interaction Meeting

15. DILIGENTDetails • Person/Months: 1024 • Total Costs: 9.546.561 € • EU Grant: 6.300.000 € • Duration: 36 Months EGEE - DILIGENT Interaction Meeting

16. DL Creation and Management Services EGEE - DILIGENT Interaction Meeting

17. Digital Library Creation and Management Services • To create virtual digital libraries, i.e. transientDLs based on shared computational, multimedia and multi-type content and application resources. • To enable virtual research organizations to dynamically create and modify DLs by specifying a set of definition criteria that fits their needs and that includes requirements on the • information space (e.g. publishing institutions, subject of the content, document type, level of replication) • on the services (e.g. service type, configuration, lifetime, availability, response time, others). EGEE - DILIGENT Interaction Meeting

18. Services for Content and Metadata Management • part of the DILIGENT Digital Library Layer • Five interacting service components • Content Management • Metadata Management • Metadata Broker • Annotation Service • Content Security Support EGEE - DILIGENT Interaction Meeting

19. Component Interaction Content Existing Content and MetadataCollections Metadata EGEE - DILIGENT Interaction Meeting

20. Index & Search Management Design and implement six functional components as part of the Digital Library Layer Search Index Data Fusion Content Selection and Description Personalization Feature extraction EGEE - DILIGENT Interaction Meeting

21. Index & Search Operational ViewPhase A Content Management Services Feature Extraction Service DL Object DL Object Index Service DL Object Resource Description & Selection Service Metadata Application Specific Feature Extraction Components Metadata Metadata DL Object Index Update Description Update Metadata Management Resource Descriptions Indexes Meta- Metadata Metadata Objects/Links VDL Storage Services EGEE - DILIGENT Interaction Meeting

22. Index & Search Operational View Phase B Process Management End-User query End-User results Search Service Query Process Query Execution Plan Query Parser Query Optimizer Extended DIR Services Data Fusion Service VDL Management Grid & DL Information Services DL Application Specific Processing Components Resource Description & Selection Service Index Service Metadata Management Personalization Service Objects/Links Resource Descriptions Indexes Meta- Metadata Metadata Personalization Data VDL Storage Services EGEE - DILIGENT Interaction Meeting

23. Process Management Design and implement tree functional components as part of the Digital Library Layer Process Design & Verification Process Execution & Reliability Process Optimization EGEE - DILIGENT Interaction Meeting

24. VDL Services in the Grid W A K ContentMgmnt Index Service Personalization Extract Term Extract Color F C Z Separate MTypes Analyze Context ContentSecurity V D B Extract Color Extract Shape ContentMgmnt EGEE - DILIGENT Interaction Meeting

25. Colour FeaturesExtr. Service Content ManagementService (CMS) Media TypeSeparation Service Shape FeaturesExtr. Service Index Service CMS … Context AnalysisService Store Object Identify Parts Extract Characteristic Information Store Features Prepare Search … Combination of Services • All these services need to be combined into a coherent whole in order to provide value-added services • This is done by processes • Example: advanced image similarity search EGEE - DILIGENT Interaction Meeting

26. Processes in the Grid … Colour Features Index Service CMS Media Type Sep. … Shape Features CMS Context Analysis W A K ContentMgmnt Index Service Personalization Extract Term Extract Color F C Z Separate MTypes Analyze Context ContentSecurity V D B Extract Color Extract Shape ContentMgmnt EGEE - DILIGENT Interaction Meeting