The Role of COUNTER David Sommer JIBS User Group 2009

1. The Role of COUNTER David SommerJIBS User Group 2009

2. Background • Understanding usage • Different approaches • Role of usage statistics • Usage statistics • Should enlighten rather than obscure • Should be practical • Should be reliable • Are only part of the story • Should be used in context • COUNTER • Current status • Future directions

3. Why COUNTER? • Goal: credible, compatible, consistent publisher/vendor-generated statistics for the global information community • Libraries and consortia need online usage statistics • To assess the value of different online products/services • To support collection development • To plan infrastructure • Publishers need online usage statistics • To experiment with new pricing models • To assess the relative importance of the different channels by which information reaches the market • To provide editorial support • To plan infrastructure

4. COUNTERCodes of Practice • Definitions of terms used • Specifications for Usage Reports • What they should include • What they should look like • How and when they should be delivered • Data processing guidelines • Auditing • Compliance • Maintenance and development of the Code of Practice • Governance of COUNTER

5. COUNTER: current Codes of Practice 1) Journals and databases • Release 1 Code of Practice launched January 2003 • Release 2 published April 2005 replacing Release 1 in January 2006 • Now a widely adopted standard by publishers and librarians • 100+ vendors now compliant • 15000+ journals now covered • Librarians use it in collection development decisions • Publishers use it in marketing to prove ‘value’ • Release 3 published August 2008, implemented by 31 August 2009 • 2 vendors already compliant: HighWire Press, MPS Technologies 2) Books and reference works • Release 1 Code of Practice launched March 2006 • 13 vendors now compliant • Relevant usage metrics less clear than for journals • Different issues than for journals • Direct comparisons between books less relevant • Understanding how different categories of book are used is more relevant

6. Code of Practice for Journals and Databases:Release 3 – main features • SUSHI protocol incorporated into R3 • Usage Reports must be provided in XML as well as in Excel • Vendors providing journal archives as a separate acquisition must report usage of these in a separate report • Search and Session activity generated by federated search engines and automated search agents must be reported separately • New library consortium usage reports, in XML only • Activity generated by internet robots, crawlers, etc. must be excluded form the COUNTER reports • A new optional additional report that enables usage of journals and books available on the same platform to be included in the same COUNTER report Release 3 is available at http://www.projectcounter.org/code_practice.html

7. Journal and Database Code of Practice: Release 3 Usage Reports • Consortium Report 1 (XML only) • Successful full-text journal article or book chapter requests by month • Consortium Report 2 (XML only) • Total searches by month and database • Journal Report 1 • Full text article requests by month and journal • Journal Report 1a • Full text article requests from an archive by month and journal • Journal Report 5 • Full-text article requests by year-of-publication and journal • Journal Report 2 • Turnaways by month and journal • Database Report 1 • Total searches and sessions by month and database • Database Report 2 • Turnaways by month and database • Database Report 3 • Searches and sessions by month and service

8. SUSHI • Standardized Usage Statistics Harvesting Initiative (SUSHI) • No mechanism existed for automatically retrieving, combining, and storing COUNTER usage data from different sources • NISO-sponsored XML-based SUSHI provides a means to do just this, via a standard model for machine to machine automation of statistics harvesting. • COUNTER and NISO have worked together to develop the SUSHI protocol. More details of SUSHI can be found at:- http://www.niso.org/committees/SUSHI/SUSHI_comm.html

9. SUSHI: Objectives • COUNTER statistics provides an excellent model and rules for usage statistics counting • Librarians needed: • A more efficient data exchange model • Current model is file-by-file spreadsheet download • Background query and response model is more efficient and scalable

10. What it is: A web-services model for requesting data Replaces the user’s need to download files from vendor’s website A request for data where the response includes COUNTER data Using COUNTER’s schema What it isn’t: A model for counting usage statistics A usage consolidation application SUSHI: What it is and Isn’t

11. Why COUNTER and SUSHI are important • For libraries and publishers • Usage statistics are being used to inform decisions • They need to be consistent, credible and comparable = • And, easy to obtain = SUSHI

12. Summary • COUNTER provides the consistency and credibility • ERM Systems/Usage Consolidation service provide the tools for more effective consolidation and reporting of usage data • SUSHI acts as an enabling technology by allowing Usage Consolidation modules to automate the harvesting of COUNTER reports • Release 3 of COUNTER Codes of Practice, will turn the theory and promise of these systems into reality by making make SUSHI support a requirement for compliance.

13. Future challenges • Improving/extending the Codes of Practice • Reliability ( audit, federated searches,) • Usability (number of compliant vendors, XML format, additional usage reports) • Database usage reports • Additional data (year of publication, individual article usage data) • Usage statistics for Institutional Repositories • Categories of content/activity • Online publications are more than collections of articles • How to reflect the value provided by other features • Deriving metrics from the Codes of Practice • Journals (cost per use, Usage Factor) • Databases? • Books?

14. Deriving metrics from COUNTER-compliant usage statistics • Local metrics • For libraries and library consortia • At journal, collection and publisher level • To compare the cost-effectiveness of journal subscriptions • To assess the value of Big Deals • Global metrics • For authors, funding agencies, libraries and publishers • At journal, collection and publisher level • To compare quality and value • Usage Factor project • PIRUS project

15. COUNTER: ‘global’ metrics • Impact Factor • Well-established, easily understood and accepted • Endorsed by funding agencies and researchers • Does not cover all fields of scholarship • Reflects value of journals to researchers • Over-emphasis on IF distorts the behaviour of authors • Over-used, mis-used and over-interpreted • Usage Factor • Usage-based alternative perspective • Would cover all online journals • Would reflect value of journals to all categories of user • Would be easy to understood

16. Basis for Usage Factor A possible basis for the calculation of Usage factor could be COUNTER Journal Report 1: Number of Successful Full-text Article Requests by Month and Journal Usage Factor = Total usage (COUNTER JR1 data for a specified period) Total articles published online (for a specified period)

17. Usage Factor Objective: Assess the feasibility of developing and implementing journal Usage Factors • Stage 1 • Level of support from author, librarian and publisher communities • Data from which UF would be derived • COUNTER Journal Report 1? • Article numbers • Process for consolidation, calculation and reporting of UFs • Factors in the calculation • Level of reporting • Total usage • Articles? • Report submitted in May 2007 • Find at http://www.uksg.org/usagefactors • Stage 2: • Test approach and methodology on real publisher usage data • RFP to be circulated in mid-2009

18. Usage Factor: Stage 1 feedback • Are the COUNTER usage statistics sufficiently robust? • Frustration at lack of comparable, quantitative data on journals • Should items covered by restricted to articles? • Many journals still have significant usage in print • Diversity of views on the factors in the calculation • Specified usage period • Specified publication period • Usage data is more susceptible to manipulation • Will the journal be a meaningful concept in the future? • Two measures with different limitations are better than one, and UF will be derived from a set of credible, understandable data • Usage data will be used as a measure of value, whether publishers like it or not

19. Stage 2:Preparing and harmonising data for analysis • Journal usage data from multiple publishers to be ingested and analysed by expert third party • 6 publishers • 1 aggregator • 1 hosting service • All interested in the insights to be gained from contributing data to the project

20. Key Data Issues • Consistency – numerator/denominator • Defining article usage year • Defining article publication date • Different usage patterns by subject

21. Report specification • With key data issues addressed, we have developed a specification for the report via which participating publishers will transfer their usage data to the appointed third party for analysis

22. Data Analysis and Modelling • Using the data provided, the appointed contractor will produce a report which will:- • Outline the various metrics assessed • Recommend which of them prove consistent and robust enough to be adopted for scaled up onward monitoring • Suggest any ways in which data providers might amend the way they capture, structure, label, and maintain their data which would make the measurement of Usage Factors:- • Easier • More reliable • Propose ways to audit Usage Factors for accuracy

23. Usage Factor Project • More information at:- • http://www.uksg.org/usagefactors • Or contact:- Richard Gedye richard.gedye@oxfordjournals.org

24. PIRUS Publisher and Institutional Repository Usage Statistics • A COUNTER Project • Sponsored by JISC through the PALS Metadata and Interoperability programme (phase 3) Further information and full report: http://www.jisc.ac.uk/whatwedo/programmes/pals3/pirus.aspx

25. Increasing interest in article level usage • More journal articles hosted by Institutional and other Repositories • Authors and funding agencies are increasingly interested in a reliable, global overview of usage of individual articles • Online usage becoming an alternative, accepted measure of article and journal value • Knowledge Exchange report recommends developing standards for usage reporting at the individual article level • Usage-based metrics being considered as a tool for use in the UK Research Excellence Framework and elsewhere.

26. Article usage metrics now more practical • Implementation by COUNTER of XML-based usage reports makes more granular reporting of usage a practical proposition • Implementation by COUNTER of the SUSHI protocol facilitates the automated consolidation of usage data from different sources.

27. The challenge • An article may be available from:- • The main journal web site • Ovid • ProQuest • PubMed Central • Authors’ local Institutional Repositories • If we want to assess article impact by counting usage, how can we maximise the actual usage that we capture?

28. PIRUS Project Mission • To develop a global standard to enable the recording, reporting and consolidation of online usage statistics for individual journal articles hosted by Institutional Repositories, Publishers and other entities

29. PIRUS Project Aims • Develop COUNTER-compliant usage reports at the individual article level • Create guidelines which, if implemented, would enable any entity that hosts online journal articles to produce these reports • Propose ways in which these reports might be consolidated at a global level in a standard way.

30. PIRUS Outputs • A proof-of-concept COUNTER-compliant XML prototype for an individual article usage report • Can be used by both repositories and publishers

31. PIRUS outputs 2. A tracker code, to be implemented by repositories that sends a message to either: • an external party that is responsible for creating and consolidating usage statistics and for forwarding them to the relevant publisher for consolidation • The local repository server

32. Step 1: a fulltext article is downloaded Step 2: tracker code invoked, generating an OpenURL log entry Scenario A Scenario B Scenario C Step A1: OpenURL log entries sent to external party responsible for creating and consolidating the usage statistics Step B1: OpenURL log entry sent to local server Step C1: OpenURL log entry sent to local server Step B2: OpenURL log entries harvested by external party responsible for creating and consolidating usage statistics Step C2: logs filtered by COUNTER rules Step A2: logs filtered by COUNTER rules Step C3: COUNTER-compliant usage statistics collected and collated per article (DOI) in XML format Step B3: logs filtered by COUNTER rules Step A3: COUNTER-compliant usage statistics collected and collated per article (DOI) in XML format Step B4: COUNTER-compliant usage statistics collected and collated per article (DOI) in XML format Step C4: COUNTER compliant usage statistics available from IR to authorized parties Step A4: COUNTER compliant usage statistics available from central organization to authorized parties Step B5: COUNTER compliant usage statistics available from central organization to authorized parties PIRUS outputs 3.

33. PIRUS outputs 4.Specification for the criteria for a a central facility that will create the usage statistics where required or collect and consolidate the usage statistics

34. COUNTER Membership • Member Categories and Annual Fees (2009) • Publishers/intermediaries: $850 • Library Consortia: $560 • Libraries: $425 • Industry organization: $425 • Library affiliate: $170 (non-voting member) • Benefits of full membership • Owner of COUNTER with voting rights at annual general meeting, etc. • Regular bulletins on progress • Opportunity to receive advice on implementation • Vendors: no compliance fee; reduced price audit fees

35. http://www.projectcounter.org Apply for COUNTER membership

36. For more information………. http://www.projectcounter.org Thank you! Peter Shepherd, COUNTER pshepherd@projectcounter.org