COUNTER: Enhancing Online Usage Statistics for Information Community

1. COUNTER: background, Codes of Practice, current activities, future developments Peter Shepherd Director COUNTER January 2008

2. Background • Understanding usage • Different approaches • Role of usage statistics • Usage statistics • Should enlighten rather than obscure • Should be practical • Should be reliable • Are only part of the story • Should be used in context • COUNTER • Achievements • Current status • Future challenges

3. So how are we getting there? • ICOLC Guidelines for statistical measurement of usage of web-based information resources • National Commission on Libraries and Information Science (NCLIS) Electronic access and use-related measures • NISO – Z39.7 (Library Statistics) • ISO – 2789 (library statistics) and 11563 (library performance measures) • MESUR – investigate metrics derived from the network-based usage of scholarly information • COUNTER (Counting Online Usage of NeTworked Electronic Resources)

4. Why COUNTER? • Goal: credible, compatible, consistent publisher/vendor-generated statistics for the global information community • Libraries and consortia need online usage statistics • To assess the value of different online products/services • To support collection development • To plan infrastructure • Publishers need online usage statistics • To experiment with new pricing models • To assess the relative importance of the different channels by which information reaches the market • To provide editorial support • To plan infrastructure

5. COUNTERCodes of Practice • Definitions of terms used • Specifications for Usage Reports • What they should include • What they should look like • How and when they should be delivered • Data processing guidelines • Auditing • Compliance • Maintenance and development of the Code of Practice • Governance of COUNTER

6. COUNTER: current Codes of Practice 1) Journals and databases • Release 1 Code of Practice launched January 2003 • Release 2 published April 2005 replacing Release 1 in January 2006 • Now a widely adopted standard by publishers and librarians • 80+ vendors now compliant • 10000+ journals now covered • Librarians use it in collection development decisions • Publishers use it in marketing to prove ‘value’ 2) Books and reference works • Release 1 Code of Practice launched March 2006 • 10 vendors now compliant • Relevant usage metrics less clear than for journals • Different issues than for journals • Direct comparisons between books less relevant • Understanding how different categories of book are used is more relevant

7. Journal and Database Code of Practice Usage Reports • Journal Report 1 • Full text article requests by month and journal • Journal Report 2 • Turnaways by month and journal • Database Report 1 • Total searches and sessions by month and database • Database Report 2 • Turnaways by month and database • Database Report 3 • Searches and sessions by month and service

8. Code of Practice for books • Book Report 1 • Number of successful requests by month and title • Book Report 2 • Number of successful section requests by month and title • Book Report 3 • Turnaways by month and title • Book Report 4 • Turnaways by month and service • Book Report 5 • Total searches and sessions by month and title • Book Report 6 • Total searches and sessions by month and service

9. Journal Report 1Full text article requests by journal Html and PDF totals reported separately

10. COUNTER Audit • Independent audit required within 18 months of compliance, and annually thereafter • Audit is online, using scripts provided in the Code of Practice • Auditor can be: • Any Chartered Accountant • Another COUNTER-approved auditor • ABCE is the first COUNTER-approved auditor • Industry-owned • Not-for-profit • Independent and impartial • Part of ABC (Audit Bureau of Circulations) • Providing website traffic audits for over 150 companies and certifying over 1400 domains • Have successfully completed test audits on COUNTER usage reports

11. ABCE Audit fees

12. COUNTER: deriving metrics from Journal Report 1 • Local metrics • For libraries and library consortia • At journal, collection and publisher level • To compare the cost-effectiveness of journal subscriptions • To assess the value of Big Deals • Global metrics • For authors, funding agencies, libraries and publishers • At journal, collection and publisher level • To compare quality and value

13. COUNTER: ‘local’ metrics • JISC (UK Joint Information Systems Committee) • Funded by UK higher education funding councils • Supports higher education in the use of information and communications technologies • Access to information and communication resources • Advice on creation and preservation of digital archives • Implications of using ICT • Network services and support • Research to develop innovative solutions • National overview of online journal usage • Develop a reliable, widely applicable methodology • Use COUNTER Journal Report 1 ‘article full-text requests’

14. Local metrics: an example • COUNTER data was analysed in relation to: • usage range • Price band • Subject category • Metrics derived from this analysis • Trend in number of full-text article downloads • Full text article requests per title • Full text article requests per publisher package • Full text article requests per FTE user • Most requested titles • Usage of subscribed vs.. unsubscribed titles • Cost per full-text article downloads • Cost per FTE user • Summary report available at: www.ebase.uce.ac.uk/projects/NESLi2.htm

15. Local metrics: an example • Growth in full-text article downloads • Publisher A: 12%- 208% • Publisher B: 12%- 59% • Publisher C: 23%- 154% • Publisher D: 22%- 81% • Cost per full-text article download • Publisher A: £0.97- £5.26 • Publisher B: £0.70 - £2.91 • Publisher C: £0.80 - £3.29 • Publisher D: £0.45 - £2.26

16. COUNTER: ‘global’ metrics • Impact Factor • Well-established, easily understood and accepted • Endorsed by funding agencies and researchers • Does not cover all fields of scholarship • Reflects value of journals to researchers • Over-emphasis on IF distorts the behaviour of authors • Over-used, mis-used and over-interpreted • Usage Factor • Usage-based alternative perspective • Would cover all online journals • Would reflect value of journals to all categories of user • Would be easy to understood

17. Global metrics: UKSG Project • Assess the feasibility of developing and implementing journal Usage Factors • Level of support from author, librarian and publisher communities • Data from which UF would be derived • COUNTER Journal Report 1? • Article numbers • Process for consolidation, calculation and reporting of UFs • Factors in the calculation • Level of reporting • Total usage • Articles • Report submitted in May 2007 • Find at http://www.uksg.org/usagefactors • Next steps • Test approach and methodology on real publisher usage data

18. UKSG Project: feedback • Are the COUNTER usage statistics sufficiently robust? • Frustration at lack of comparable, quantitative data on journals • Should items covered by restricted to articles? • Many journals still have significant usage in print • Diversity of views on the factors in the calculation • Specified usage period • Specified publication period • Usage data is more susceptible to manipulation • Will the journal be a meaningful concept in the future? • Two measures with different limitations are better than one, and UF will be derived from a set of credible, understandable data • Usage data will be used as a measure of value, whether publishers like it or not

19. Current issues • Metasearch • Potential inflationary effect on search counts • Pre-fetching • Potential inflationary effect on full-text download counts • Reporting separately purchasable digital archive usage • Currently all usage for a journal is usually reported together • Separately purchasable archives mean we need separate reports for archival content, or a year of publication breakdown of usage • Usage in Institutional Repositories • Growth in Institutional Repository (IR) content • Need for credible IR usage statistics • IR usage statistics already being collected, but no standards • SUSHI • Click-based searches • New report? • Usage reports for Consortia • Current usage reports inadequate • New reports in XML format

20. Reporting separately purchasable digital archive usage • Increasingly requested by librarians • Interim solution • Journal Report 1a:Number of Successful Full-Text Article Requests from an Archive by Month and Journal • Optional additional usage report • Longer-term solution • Journals Report 1a? • Include year-of-publication data in JR1?

21. SUSHI • Standardized Usage Statistics Harvesting Initiative (SUSHI) • No mechanism existed for automatically retrieving, combining, and storing COUNTER usage data from different sources • NISO-sponsored XML-based SUSHI provides a means to do just this, via a standard model for machine to machine automation of statistics harvesting. • COUNTER and NISO have worked together to develop the SUSHI protocol. More details of SUSHI can be found at:- http://www.niso.org/committees/SUSHI/SUSHI_comm.html

22. Future challenges • Improving/extending the Codes of Practice • Reliability ( audit, federated searches, prefetching) • Usability (number of compliant vendors, XML format, additional usage reports) • Additional data (year of publication, article level reports) • Categories of content (Institutional Repository content) • Deriving metrics from the Codes of Practice • Journals (cost per use, Usage Factor) • Databases? • Books?

23. Next steps….. • Release 3 of Code of Practice for Journals/Databases • Features: prioritisation on basis of demand and practicality • Process: consultation via focus groups,etc; publication of draft CoP • Timetable: focus groups November/December 2007; draft R3 in early 2008; final R3 in mid-2008; implementation of R3 in 2009 • Release 2 of Code of Practice for Books • Review R1 in practice • Other categories of content ( eg Institutional Repositories) • Metrics derived from the COUNTER usage statistics • Cost per use • Usage Factor

24. COUNTER Membership • Member Categories and Annual Fees (2008) • Publishers/intermediaries: $825 • Library Consortia: $545 • Libraries: $412 • Industry organization: $412 • Library affiliate: $165 (non-voting member) • Benefits of full membership • Owner of COUNTER with voting rights at annual general meeting, etc. • Regular bulletins on progress • Opportunity to receive advice on implementation • Vendors: no compliance fee; reduced price audit fees

25. http://www.projectcounter.org Apply for COUNTER membership

26. For more information………. http://www.projectcounter.org Thank you! Peter Shepherd, COUNTER pshepherd@projectcounter.org