Interoperability Among Scholarly Repositories: Enabling Workflows Across Distributed Information

1. Interoperability Among Scholarly Repositories:Enabling Workflows Across Distributed Information Carl Lagoze Information Science Cornell University, USA Herbert Van de Sompel Research Library Los Alamos National Laboratory, USA

2. Acknowledgments This talk based on the following work: • NSF-funded Pathways project (IIS-0430906) • Cornell University (PIs: Carl Lagoze, Sandy Payette, Simeon Warner) • LANL Digital Library Research & Prototyping Team (PI Herbert Van de Sompel). • The LANL aDORe repository effort. • http://dx.doi.org/10.1093/comjnl/bxh114 • http://african.lanl.gov/aDORe/ • The PhD thesis by Jeroen Bekaert (Advisor Herbert Van de Sompel) regarding protocol-based interfaces for Open Archival Information Systems (OAIS). • http://hdl.handle.net/1854/4833

3. References • “Rethinking Scholarly Communication”, D-Lib September 2004 • “Interoperability for Distributed Scholarly Workflows”, D-Lib October 2006 • “Pathways: Augmenting Interoperability for Scholarly Repositories”, Upcoming Journal of Digital Libraries

4. Some Background • Digital transition of scholarly communication has been in formrather than nature • Try and build a scholarly communication system that is more natively digital, i.e. use the capabilities of digital, network technologies • Collaboration • Immediacy • Reuse • Dynamic • Exploit advances in institutional repositories and interest in open access • Frame scholarly communication as a workflow among distributed information units • Provide framework for new advanced services • Visualization • Usage analysis • …

5. Interoperability in a Heterogeneous World • Diversity of (repository) technology • DSpace • Fedora • aDORe • EPrints • Greenstone • Define an interoperability layer in which • Information can be modeled • Information can be shared • Information can be transfered • Information can be reused

6. Some Meta-Observations on Interoperability • Scholarly communication is a long-term endeavor: • Dependent on stability and integrity of participants • Need abstract definitions of models and interfaces that can be instantiated on the basis of various technologies as time goes by • Identificationis particularly important: • Scalable • Agnostic about existing identification schemes • Granular • Object decomposition • Repository origination • Value chains do not require transfer of all digital object content • The content that needs to be transferred depends on the nature of the value chain

7. Shared Data Model and Services Individual Data Models and Services Augmenting interoperability across Repositories DSpace Fedora aDORe ePrints arXiv Nature

8. Motivation 1 : Richer cross-Repository services • Distributed Repositories provide source materials for cross-Repository overlay services such as discovery services • Manner in which those materials are exposed must allow for the seamless emergence of rich and meaningful services

9. Richer cross-Repository services : Scenario Scenario 1: Chemical search engine • A search engine monitors scholarly repositories but is only interested in making machine-readable chemical structures contained in Digital Objects available from those repositories searchable. • This constitutes re-use of the (part of) the Digital Objects by a service overlaid upon the monitored repositories. • And, of course, a chemical compound discovered via the search engine can be cited in some new paper, i.e. the value chain does not stop here

10. Motivation 2 : Scholarly communication workflow • Distributed Repositories at the basis of a digital scholarly communication system • Scholarly communication as a global workflow (value chain) across those Repositories • Digital Objects from Repositories are the subject of the workflow; they are used and re-used in many contexts.

11. Scholarly communication workflow : Scenarios Scenario 2: Citation • An author writes a paper (to be Put into her institutional repository) and cites 10 papers available from other repositories. • A citation to a paper is a type of re-use of the cited paper in a new context. • And, of course, the new paper can be cited too, i.e. the value chain does not stop here.

12. Adding Value to Fundamental Units Paul Ginsparg

13. Scholarly communication workflow : Scenarios Scenario 3: Overlay journal • The editor of an overlay journal selects papers from 3 different repositories for inclusion in the next issue of the overlay journal. • Each of those articles is being re-used in a new context, with value being added. • And, the overlay journal can be mirrored for preservation purposes, i.e. the value chain does not stop here.

14. Building Block I - Repositories Networked system that provides services pertaining to a managed collection of digital objects. Institutional repositories, online journals, dataset stores, learning objects, etc.

15. Aim: Digital Object use and re-use • We must leverage the value of the materials that become available in those distributed Repositories. • Think about these Repositories as activenodes in a globalenvironment, not as passive localnodes • These Repositories are about facilitating the use and re-use of materials in many contexts • These Repositories are the starting point of value chains

16. id id Building Block II: Digital Objects Abstract units of scholarly communication Compound aggregations consisting of: • Multiple media types • Linkage to services Have a persistent identifier Can be recursive: digital objects within digital objects Instantiated in various implementations c.f. Kahn/Wilensky Model Digital Objects

17. id m Terminology Digital Object: A data structure whose principal components are digital data and key-metadata. Digital data can be a Datastream or a Digital Object, i.e. a Digital Object may have one or more other Digital Objects as nested components. Key-metadata must include an identifier for the Digital Object. Datastream: An ordered sequence of bytes. Data Model: An abstraction for Digital Objects such that each Digital Object can be seen as an instance of the class defined by a Data Model. Example Data Models include the Pathways Core model, the MPEG-21 Digital Item Declaration model, etc. Surrogate: A serialization of a Digital Object according to a Data Model.

18. Put Harvest Obtain Terminology Repository: a networked system that provides services pertaining to a collection of Digital Objects. Obtain interface: a Repository interface that supports the request of services pertaining to individual Digital Objects (including their component Datastreams). Harvest interface: a Repository interface that exposes Surrogates for incremental collecting/harvesting. Put interface: a Repository interface that supports submission of one or more Surrogates into the Repository, thereby facilitating the addition of Digital Objects to the collection of the Repository.

19. Put Harvest Obtain m Individual Data Models and Services Augmenting interoperability across Repositories DSpace Fedora aDORe ePrints arXiv Nature

20. m Common Data Model • Provides a common abstraction for describing digital objects despite their (repository, service)-specific implementation. • A common denominator: • Does not completely cover implementation-specific features • Features conform to requirements of interoperability fabric (e.g., identity, workflow support, etc.)

21. m Model Core Requirement • Recursion for n-levels of information containment • Identity independent of specific schemes • Lineage relationships among objects • evidence of workflow for evidential citation • Semantics associated with entities • facilitate service mapping • Link to concrete representation • Assertion of persistence levels

22. Data Model

23. m Augmenting interoperability across Repositories Pathways Core Surrogates (currently XML/RDF) • A Surrogate is available for every Digital Object • A Surrogate is a representation of the Digital Object according to the Pathways Core data model • The representation is uniform across repositories; not tied to identifier type, content type, application domain. • The Surrogate is what is used in the value chains; the Surrogate is used at Obtain, Harvest and Put interfaces. • Expresses properties and access points for the Digital Object (see later)

24. m Augmenting interoperability across Repositories Pathways Core Surrogates (currently XML/RDF) • The Surrogates provide By-Reference access to constituent datastreams of Digital Objects • Full asset transfer is only required for certain applications • Avoid IP issues at the level of the interoperability framework • The idea is that the Surrogate itself is not encumbered by IP issues; attach - by definition - a liberal Creative Commons license to Surrogates • Allow Surrogates to flow freely independent of business models of the underlying content

25. Put Harvest Obtain Augmenting interoperability across Repositories Obtain interface: a Repository interface that supports the request of services pertaining to individual Digital Objects (including their component Datastreams). The core service is the request of a Surrogate for a Digital Object. Harvest interface: a Repository interface that exposes Surrogates for incremental collecting/harvesting. Put interface: a Repository interface that supports submission of one or more Surrogates into the Repository, thereby facilitating the addition of Digital Objects to the collection of the Repository.

26. id id Put providerInfo recombine & add value Obtain Obtain Obtain providerInfo id Surrogate is at the core of the value chain Lineage Lineage

27. Harvest Harvest Put Put Obtain Obtain service Put2 Harvest2 Obtain2 Repo2 Put1 Harvest1 Obtain1 Repo1

28. Harvest Harvest Put Put Obtain Service Registry Obtain Put1 Harvest1 Obtain1 Repo1 Put2 Harvest2 Obtain2 Repo2

29. Meeting in NYC, April 20-21 2006 • Supported by Microsoft, Mellon Foundation, Coalition for Networked Information, Digital Library Federation, JISC • Representatives from institutional Repository projects, scholarly content Repositories, Registry projects, various projects that touch on interoperability • See http://msc.mellon.org/Meetings/Interop/ for Agenda, Participants, Topics & Goals, Terminology, Presentations, Prototype demonstration. • Report available since beginning of August 2006 • Very likely that an international interoperability effort will be started towards the end of 2006

30. Demonstration • Overlay journal Scenario combined with Search engine Scenario • Surrogates compliant with Pathways Core Data Model, expressed in RDF/XML. • Obtain interfaces (OpenURL Application) at: • an aDORe repository • arXiv • a DSpace repository • a Fedora repository • Harvest interfaces (OAI-PMH) at: • an aDORe repository • arXiv • a Fedora repository • Put interface at a Fedora repository • MS Live Clipboard functionality in user interfaces of arXiv, Fedora, and the overlay search engine

31. Demonstration • Acknowledgments: • Carl Lagoze, Sandy Payette, Simeon Warner, Chris Wilper at Cornell University • Rob Tansley at HP • Luda Balakireva, Xiaoming Liu, Herbert Van de Sompel, Zhiwu Xie at the Los Alamos National Laboratory

32. id id Put Obtain Demonstration Live Clipboard Copy Live Clipboard Paste Submit