NISO’s IOTA Working Group Improving OpenURLs Through Analytics

1. NISO’s IOTA Working GroupImproving OpenURLs Through Analytics UKSG Conference Harrogate, United Kingdom April 4 – 6, 2011 Rafal Kasprowski, Rice University

2. Agenda • In the Beginning: Full-text linking and Advent of OpenURL • IOTA: Created in response to OpenURL linking problems • IOTA’s analytical approach • Community-derived reports comparing quality of vendor OpenURLs • Concept of the OpenURL Quality Index • IOTA & KBART: relationship & joint initiative • Community involvement in IOTA: necessary for best outcomes

3. Before OpenURL: Proprietary Linking • A&I database providers offered option for full-text linking (e.g., CSA, PubMed, etc.) . • Libraries manually activated linking to full-text providers they had subscriptions with. • A&I --> Full Text

4. Proprietary Linking: Cons and Pro • Linking had to be activated manually by libraries for each full-text provider. • A&I providers offering this option were few. • Selection of full-text providers was limited. But... •  Once set up, the static links to full texts were accurate.

5. Advent of OpenURL • Objective: Deliver full texts unrestrained by proprietary silos. • Open standard generating link at time of request. • Library's holdings indicate provider of "appropriate copy". • A-Z list (e.g., e-journal, e-books): • Knowledge base (KB) with library's holdings. • Intermediary in linking. • A&I ("Source") --> A-Z list ("KB") --> Full Text ("Target")

6. OpenURL Syntax and Resolver Source Citation Target OpenURL (Source OpenURL structured similarly) http://ps4ps6lm2r.search.serialssolutions.com/?issn=0957-4484&volume=21&issue=44&date=20101105&spage=445201&title=Nanotechnology&atitle=A+ versatile+nanotechnology+to+connect+individual+nano-objects+for+the+ fabrication+of+hybrid+single-electron+devices.&aulast=A++Bernand A, Bernand, et al. "A versatile nanotechnology to connect individual nano-objects for the fabrication of hybrid single-electron devices." Nanotechnology 21, no. 44 (November 5, 2010): 445201. Academic Search Complete, EBSCOhost (accessed October 24, 2010).

7. Pros & Cons of OpenURL Pros: • KB/Resolver vendors took over most of the linking setup: Less work for libraries and providers. • Participation by A&I platforms and full-text providers exceeded proprietary linking: OpenURL scales better Cons: • Dynamic linking less predictable than static linking: more difficult to pinpoint cause of link failures • OpenURL linking not improved significantly last 10 years. • No systematic method exists to benchmark OpenURLs.

8. Problem Statement & Methodology "72% of respondents to the online survey either agreed or strongly agreed that a significant problem for link resolvers is the generation of incomplete or inaccurate OpenURLs by databases (for example, A&I products)." Culling, James. 2007. Link Resolvers and the Serials Supply Chain: Final Project Report for UKSG, p.33. http://www.uksg.org/sites/uksg.org/files/uksg_link_resolvers_final_report.pdf. • Recently, researchers have indicated the need for metadata quality metrics, including: • completeness; • accuracy; • conformance to expectations; • logical consistency and coherence. • Bruce, Thomas R. and Hillmann, Diane I. 2004. The Continuum of Metadata Quality: Defining, Expressing, Exploiting. In Metadata in Practice. Ed. Diane I. Hillmann and Elaine L. Westbrooks. Chicago: American Library Association, pp. 238-256.

9. Année philologique OpenURL Study 2008 Cornell study led by Adam Chandler* • Problem: Too often links sent from Aph did not successfully resolve to requested resource. • Objective: Examine quality of OpenURLs offered to users by Aph in order to improve the linking. Aph Study investigated: • Faulty citation metadata from source database. • Method to evaluate the OpenURLs. *Chandler, Adam. 2009. Results of L’Année philologique online OpenURL Quality Investigation: Mellon Planning Grant Final Report. http://metadata.library.cornell.edu/oq/files/200902%20lannee-mellonreport-openurlquality-final.pdf.

10. Scoring System & Aph Study Outcomes Concept of scoring in Aph study (based on B. Hughes study)* • establish a baseline for comparison; • results to be shared with data providers; • develop a best practice. Problem analysis in Aph study limited to: • source link • presence/absence of citation metadata elements Results: • OpenURL quality model: compares elements in Aph OpenURLs to those of other providers. • No scores, but model is first step towards scoring system. *Hughes, Baden. 2004. Metadata Quality Evaluation: Experience from the Open Language Archives Community. In Digital Libraries: International Collaboration and Cross-Fertilization. Ed. Zhaoneng Chen et al. Berlin: Springer-Verlag, 2004, pp. 320-329.

11. Creation of IOTA NISO accepts proposal to take Aph Study to wider community • Improving OpenURLs Through Analytics (IOTA): • Formed in January 2010. Basic Assumptions: • Results are achieved through an analytical investigation of how OpenURL links work. • Practical: Not OpenURL standard is addressed, but links (OpenURLs) generated by standard. • Selective changes to OpenURLs will lead to significant improvement in linking success rate. • Motto: "small changes. big improvements"

12. Desired Outcomes • Produce qualitative reports that will help OpenURL providers quickly compare their OpenURL quality to that of their peers. • Develop community-recognized index for measuring the quality of OpenURL links generated by content providers. • Method: • fair; • transparent; • scalable across all OpenURLs and their providers.

13. Why are OpenURL reports important? • Content providers can submit their OpenURL data: • Compare their OpenURL data with other vendors; • Institutions can submit their OpenURL data: • See how OpenURLs from providers work and make local adjustments to their OpenURL setup • Third parties can use IOTA’s OpenURL data: • E.g. link resolver vendors, web-scale discovery system vendors can use reports to adjust their OpenURL linking;

14. Running reports

15. Reports: Log file providers

16. Report: source (vendor or database)

17. Report: selecting source = vendor

18. Report: element, source = vendor

19. Report: pattern, source = vendor

20. Report: element, source = database

21. Report: pattern, source = database

22. Report: element and pattern frequency

23. Report: element & pattern frequency - selecting vendor

24. Report: element/pattern frequency: Choosing Metric

25. Report: element/pattern by vendor

26. Report: element & pattern frequency - selecting dbase

27. Report: element/pattern by database

28. Reporting System: current improvements • Consolidating variant instances of databases and vendors if the same; • Separating article-like requests from book-like requests • Either/Or situation: most resources do not offer both formats • Once separation is completed, users will be given corresponding options to select OpenURL data by format:ARTICLE or BOOK

29. OpenURL Quality Index: initial version 1. Core elements: • Any element contained in IOTA's OpenURL reporting system; • 10M OpenURLs already obtained from libraries content providers. 2. Scoring system based on assumption: • Correlation exists between • # of core elements ("OpenURL completeness") & • ability of OpenURLs to link to specific content. 3.  Weighting assigned to core elements: • Based on relative importance • spage vs atitle • issn vs jtitle • doi/pmid vs date, etc.

30. OpenURL Quality Index: vendor rating

31. Work in Progress • Element weighting still in progress: • E.g., importance of identifiers (doi, pmid) vs bibliographic data (issn, volume, spage). • Currently, IOTA focuses on OpenURLs from citation sources only. OpenURL quality is also influenced by: • knowledge base, • resolver, • full-text provider (target). • High "completeness" score of OpenURLs not always indicative of "success" in linking to full texts • Combination of multiple indexes along linking nodes may provide more complete picture.

32. IOTA & KBART: related through OpenURL • IOTA node: • analyzing data sent from OpenURL source to link resolver. • KBART node: • creating formatting best practices for data sent from content providers to knowledge base (and link resolver) vendors.

33. KBART/IOTA joint initiative: underway • KBART/IOTA node: • Exploring together the third source of failures: • link-to (or target) syntax and behavior which couples link resolvers to content providers • Collaboration begun in March 2011 is meant to address OpenURL quality in a broader context.

34. How can I get involved? If you are a content provider: • Contribute data to IOTA • Review the IOTA data • This data is meant to help make improvements in your OpenURL linking. If you are a librarian: • Contribute data to IOTA • Help spread the word to vendors

35. IOTA web presence

36. Questions? • http://openurlquality.niso.org • http://www.niso.org/workrooms/openurlquality • @nisoiota on twitter Rafal Kasprowski Electronic Resources Librarian Rice University Fondren Library MS 44 Houston, TX 77005 USA rk11@rice.edu