1 / 23

The Planets Interoperability Framework

The Planets Interoperability Framework. Integrated Access to Preservation Tools. Rainer Schmidt AIT Austrian Institute of Technology rainer.schmidt@ait.ac.at. 1st DPIF Symposium, April 21-23, 2010, Dresden, Germany. Outline. Overview of the Integrated Environment

ros
Download Presentation

The Planets Interoperability Framework

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. The Planets Interoperability Framework Integrated Access to Preservation Tools Rainer SchmidtAIT Austrian Institute of Technology rainer.schmidt@ait.ac.at 1st DPIF Symposium, April 21-23, 2010, Dresden, Germany.

  2. Outline • Overview of the Integrated Environment • Main Objectives and Architecture • Planets Preservation Services • Digital Objects and Metadata • Integrating Repositories • The Workflow Execution Engine (WEE) • Conclusions & Lessons Learned

  3. Planets Project • “Permanent Long-term Access through NETworked Services” • Addresses the problem of digital preservation • driven by National Libraries and Archives • Project instrument: FP6 Integrated Project • 5. IST Call • Consortium: 16 organisations from 7 countries • Duration: 48 months, June 2006 – May 2010 • Budget: 14 Million Euro • http://www.planets-project.eu/

  4. The Planets Interoperability Framework • An integrated System for the development and evaluation of preservation strategies. • Uniform access mechanisms to a broad range of “commodity” tools, e.g. for characterization, migration, emulation. • Integration of existing repositories, data/metadata formats. • Specification, execution, recording of preservation workflows. • Integration with end-user applications for preservation planning and the evaluation of tools/strategies. • PLANETS Preservation Planning Tool and Testbed

  5. Agents and Activities Export Digital Objects Service Registration Data Model Mapping <<migrate>> Experiment Repository Digital Library/Repository ApplicationProvisioning <<retrieve objects>> <<apply object>> <<characterize>> Deposit Result IF Gateway Server Data Transfer Service Orchestration <<create experiment>> Provenance <<compare>> Access Pres. Applications Preservation Expert Preservation Services User Management

  6. Service-Orientated Architecture • XML Web Services (SOAP, WSDL, WS-*) • Platform, Language, and Location Independence • Homogeneous interfaces for preservation activities, data management, workflow execution. • Remotely access repositories and data. • Discover and dynamically utilize tools in a workflow. • Supports distributed and cross-organizational deployments • Shared hardware, software, maintenance • Browser-based access to large number of resources

  7. Service Gateway Architecture Administration UI Preservation Planning Tool Experimentation Testbed Application Workflow Execution UI User Applications Workflow Execution and Monitoring Experiment Dataand Metadata Repository Service and Tool Registry Authentication and Authorization Notification and Logging System Portal Services Application Services ExecutionServices Data Access Services Application Execution and Data Services Physical Resources, Computers, Networks

  8. Preservation Interfaces • Define atomic preservation activities (level-one) • Concentrates on low-level concepts and actions • Bit-stream operations, no data management • Designed to be light-weight and easy to implement • Independent from a specific tool, language, or content type • E.g. Characterize, Migrate, Compare, CreateView • >50 Tools wrapped/provided as Planets Services • Provides the basic abstractions for assembling workflows.

  9. Preservation Interfaces (the Verbs) • Define atomic preservation activities (level-one) • Concentrates on low-level concepts and actions • Bit-stream operations, no data management • Designed to be light-weight and easy to implement • Independent from a specific tool, language, or content type • E.g. Characterize, Migrate, Compare, CreateView • >50 Tools wrapped/provided as Planets Services • Provides the basic abstractions for assembling workflows.

  10. Digital Objects • Generic data abstraction for modeling digital entities. • Encapsulates content and metadata • Consumed and/or produced by Planets preservation services • Provides minimal and generic model for data management • Stored in Object Repository • Does not prescribe serialization schema • May be created from DC/ORE RDF record and be • serialized using METS/PREMIS schemas.

  11. Digital Objects Type, Time, Agent, Service, Result, … Creator, Title,Description, Format, … Properties Events Digital Object fragment Metadata Content contains_object Embedded Data or Repository URL Tagged Uninterpreted Metadata Chunks Relationships (possibly associated with event)

  12. Digital Object Managers • Individual adapters for retrieving (& storing) Planets DOs • Provide access to existing repositories. • Map metadata records to Planets DOs • Ingest digital objects to Planets data repositories • Current implementation for • retrieving OAI-PMH records, BL digitized newspaper, Web resources, Amazon S3 buckets, … • Planets Data Registry services (ingesting DOs) based on Apache Jackrabbit and Fedora Commons.

  13. Data Registry • A service to deposit, access, and organize Planets digital objects based on bi-directional Digital Object Manager. • Accessible to Workflow Execution Engine • Records Experiment and Preservation Metadata • Supports Export of Experiment Results • A Repository that implements Planets Digital Object Model and naming schema (Planets URIs). • Supports asynchronous pass-by-reference and direct access to binary Content (Content Resolver)

  14. Data Registry • A service to deposit, access, and organize Planets digital objects based on bi-directional Digital Object Manager. • Accessible to Workflow Execution Engine • Records Experiment and Preservation Metadata • Supports Export of Experiment Results • A Repository that implements Planets Digital Object Model and naming schema (Planets URIs). • Supports asynchronous pass-by-reference and direct access to binary Content (Content Resolver)

  15. Workflow Orchestration • Separation of concerns: • Fragments of complex workflow logic (templates) are implemented by <<workflow developers>> • <<Experimenters>> selected from predefined templates, configure them, and execute individual processes. • Templates implement abstract and reusable processes definitions based on level-on operations (API) and decision logic. • Execute in trusted environment (level-two) • handle digital objects in metadata repository and • basis for recording provenance and preservation information

  16. Workflow Execution Engine (WEE) Service WEE Execution Service <<3: configure>> <<4: execute>> Template XML Cmp. Workflow Client Application Cmp. Workflow Developer Experimenter <<2: select>> <<1: register>> WEE Template Rep. Service

  17. Summary • Research infrastructure for • integrating variety of tools and repositories • executing defined preservation operations • recording provenance and preservation metadata • Not necessary an “out-of-the-box” solution • Extensible network of services, • Public deployment, • Allows sharing of resources and results. • Downloadable package available for local installation of selected preservation tools/services.

  18. Conclusions (1) - Preservation Actions • Defined interfaces for Preservation Actions required • Prerequisite for QA and other complex pres. strategies (workflows) • Preservation strategy often trivial (complexity within the tool) • Automation and Quality Control are key issues • Verifiability of technical interoperability is crucial • Depends much on communication method (native, DSL) • keep as simple as possible • Semantic interop. requires well defined properties and metrics • often domain dependent • defined tests and benchmarks required

  19. Conclusions (2) - Component Framework • The Planets IF provides an environment for preservation components to run and interact • Distributed system required for extensibility and integration • Service interfaces specified at exchange language level (HTTP, SOAP, WS* Specs.) • Interoperability often not a problem of specification but of inconsistencies in different implementations • 3rd party tools impose multiple levels of indirection • OS calls, different languages, different middleware stacks • Supporting (proprietary) tools may impact hosting environment and factors like performance, robustness, and fault tolerance.

  20. Conclusions (3) - Repository Integration • Planets provide a flexible approach for bridging access to heterogeneous repository systems. • Diverse APIs, metadata representation, data access • Stds. exist (OAI-ORE, RDF) but not yet adopted • Missing standards for integration of digital preservation actions with digital repository systems • (a) Defined Methods for Access, Re-Ingest, Versioning • (b) Entirely integrated with repository • Considerable efforts required to adapt data management systems in place

More Related