1 / 26

Long-term preservation aspects in the eSciDoc project

Long-term preservation aspects in the eSciDoc project. Natasa Bulatovic Max-Planck Digital Library bulatovic@mpdl.mpg.de. eSciDoc: Background information. Joint Project between Max Planck Society and FIZ Karlsruhe

isolde
Download Presentation

Long-term preservation aspects in the eSciDoc project

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Long-term preservation aspects in the eSciDoc project Natasa Bulatovic Max-Planck Digital Library bulatovic@mpdl.mpg.de

  2. eSciDoc: Background information • Joint Project between Max Planck Society and FIZ Karlsruhe • Funded by German Federal Ministry for Education and Research (BMBF) until Mid 2009 • Additional substantial own efforts by both partners • Both Partners committed themselves to ongoing efforts until 2011

  3. eSciDoc project landscape

  4. Diversity of MPG supported by a service infrastructure • People • Librarians, researchers, developers, general public • Data • Publications, research data, supplementary material etc. • Processes • Various institutes apply various workflows supporting their data management

  5. eSciDoc: what it addresses and provides? • Organizational aspects • Business processes: introspection into the “research information and research data life-cycle” • Modelling of the “research workflows” (usage, scenarios, use cases, process worfklows) • “SOA” is not only technical infrastructure - it is driven by organization-wide processes and requirements • Technical aspects • service infrastructure instead of silos applications • solutions to visualize, publish, relate and manage data • easy composition of services and data mashups and repurposing Both organizational and technical activities facilitate the growth of the organization and the dissemination, accessibility and easy reuse of the research results across disciplines

  6. eSciDoc.PubMan: Management of publications

  7. eSciDoc.Virr: Management of digitized textual resources

  8. eSciDoc.FACES: Management of image collections

  9. Challenges And facts for eSciDoc project related to the digital preservation and long-term archiving

  10. Which data are managed? Support for variety of data (publications, old manuscripts, microscopic images, patents, datasets)

  11. Abstraction: Data structure • Abstraction in data modelling and implemented data services (items, containers).xml • Specialization through content models gives contextual information on what data it is (publication, scanned book page, lexical resource, collection of “digital things”) Atomistic model http://colab.mpdl.mpg.de/mediawiki/ESciDoc_Logical_Data_Model

  12. Atomistic model - items Seite 11

  13. Atomistic model - containers Seite 12

  14. Support for variety of workflows for different user-communities Communities and workflows • Community specific workflow Core workflow

  15. Different communities decide what to identify and when in a different manner and at a different state of the completeness of their data Persistent identification • Each resource and resource component (metadata, files) must be persistently identified • Users may decide what to identify (resource, resource version) • Users may configure when to assign the persistent identifier on a repository level • Keep all persistent identifiers of the resource (if such existed before the resource has been created in the system)

  16. Persistent identification system • But which PID system to choose out of many ? • Handle System (CNRI - Corporation for National Research Initiatives) • Digital Object Identifiers - DOI (IDF – International DOI Foundation) • Archival Resource Key –ARK (California Digital Library) • the Uniform Resource Name - URN (IANA) • National Bibliography Numbers - NBNs • the persistent URL – PURL (OCLC, Online Computer Library Center) • the Open URL • Most important is the organizational commitment for PID and PID infrastructure, PIDs are rather organizational then technical effort • EPIC – European Persistent Identifier Consortium (GWDG Germany, SARA Netherlands, CSC Finland, http://www.pidconsortium.eu/ )

  17. Support for variety of data (publications, old manuscripts, microscopic images, patents, datasets) and their metadata Resource metadata • Standards and Frameworks – have to be used so data are known • Dublin Core, MODS, other • Where no standards exist, describe the profile in standard format • Dublin Core Singapore Framework • Services, metadata quality and metadata extraction • Metadata Validation service • CoNE – Control of Named Entities service • JHove – technical metadata extraction service • Supports and important aspect: quality of data, clear identification of resources, format migration • Interoperability – data redundancy • Ensures resources are disseminated via RSS, OAI-PMH

  18. Metadata standards and frameworks “The Singapore Framework for Dublin Core Application Profiles is a framework for designing metadata applications for maximum interoperability and for documenting such applications for maximum reusability” http://dublincore.org/architecturewiki/SingaporeFramework/ http://dublincore.org/documents/2007/06/04/abstract-model/

  19. Metadata profiles • DCAP based (Dublin Core Application Profile) • DC terms (identified URIs) • eSciDoc solution specific terms (identified by URIs) • METS/MODS • Publicly available • Functional description http://colab.mpdl.mpg.de/mediawiki/ESciDoc_Application_Profiles • Application profiles schemas (xsd) http://metadata.mpdl.mpg.de/escidoc/metadata/schemas/0.1/ • Vocabulary encoding schemas http://colab.mpdl.mpg.de/mediawiki/ESciDoc_Encoding_Schemes • Interoperability levels • Shared term definitions (done) • Semantic interoperability (done) • Description set syntactic interoperability (started) • Description set profile interoperability (started)

  20. CoNE – Control of named entities • Journals, Persons, Dewey Decimal Classification (3 public levels), Creative Commons Licenses (CC licenses), ISO 639-3 Languages, MIME Types, PACS classification, Custom classifications • “Vocabulary Encoding schemas for resources” • Validation service • Semantic metadata validation based on Shibboleth • If the publication is of type “Journal Article” then the “Journal” information has to be present • Rules are preserved in XML – and are not encoded in the business logic code • Jhove service • Technical metadata are extracted prior to binary content upload • Technical metadata are kept together with the binary content • Example • http://faces.mpdl.mpg.de/details/escidoc:47276 • http://r-coreservice.mpdl.mpg.de/ir/item/escidoc:47276 Services

  21. Data provenance • Support for event logs and version history (what happened with the content?) • PREMIS v1 created for each resource (events info) Example: http://pubman.mpdl.mpg.de/pubman/item/escidoc:66603 • Upcoming: upgrade to PREMIS v2

  22. Bitstream preservation  Software, standards, documentation • Software • all used software is an open source software • eSciDoc software available under CDDL license (open source) • Documentation • Accessible (still a lot of work to do) • Specifications and some design are available via MPDL Colab e.g. http://colab.mpdl.mpg.de/mediawiki/PubMan_About • Standards and formats • XML, XSD used for interface and service operation messages • Use of metadata standards • PREMIS • Open formats wherever possible e.g. metadata, lexical resources TEI-XML etc. • Containment: Fedora XML objects for content resources • OAIS reference model – compatible

  23. OAIS Archive: What we will put for LTA? • XML • XML/RDF • XSDs • Open source code • Binary content with technical metadata (XML) and calculated MD5 checksums

  24. Lessons learned • Have we covered all aspects? • They must be considered from the very start in infrastructural design and implementation • Things are getting more complicated in dynamic environment (data are changed often) • Selection process • What – all or only selected research data? • When – whenever data is created or after the finalization of curration activities? • Why - “institutional memory”, future research? • Data redundancy • As many copies as possible • Share and replicate as much as possible - redundancy does not hurt (it costs ) • Organizational commitment • Sustainable • Long-term • Technical • Collaboration and cooperation with partners necessary (its not a single unit effort)

  25. Thank you! • www.escidoc-project.de • http://colab.mpdl.mpg.de • bulatovic@mpdl.mpg.de

  26. Resources • eSciDoc project web pages, Infrastructure download • http://www.escidoc-project.de • MPDL collaboratory network • http://colab.mpdl.mpg.de • MPDL software download • http://escidoc1.escidoc.mpg.de/projects/common_services/ • http://escidoc1.escidoc.mpg.de/projects/pubman/ • http://escidoc1.escidoc.mpg.de/projects/faces/ • http://escidoc1.escidoc.mpg.de/projects/virr/ • http://colab.mpdl.mpg.de/mediawiki/ESciDoc_Admin

More Related