Measuring the usage and impact of the Irish Research eLibrary ( IReL )

1. Measuring the usage and impact of the Irish Research eLibrary (IReL) An evolving project John Cox Deputy Librarian National University of Ireland, Galway

2. IReL in Brief • Established in 2004 • Government-funded • Focused on research community • Accessible at 7 Irish universities • Covers most disciplines • c. 90 “products” • 25000+ e-journals • 40000+ e-books

3. IReL Content: STM

4. IReL Content: AHSS

5. IReL Management • Part-time, voluntary groups staffed by member libraries • Steering Group • Monitoring Group • Licencing Group • Web Group • IT Group?

6. Monitoring Group: remit • Collate and monitor performance statistics in relation to the value for money of IReL titles. • Collate and monitor downtime of IReL titles. • Suggest retention or cancellation of IReL titles based on information gathered. • Provide summaries of changes of content in IReL major services i.e. deletions of titles or addition of new titles. • Note deficiencies of IReL information supply with regard to specific areas of research. • Suggest ways of continuing to promote the IReL service.

7. Monitoring Group: members • Rosarii Buttimer, University College Cork • John Cox (chair), National University of Ireland, Galway • Aoife Geraghty, University of Limerick • Arlene Healy, Trinity College Dublin • Jack Hyland, Dublin City University • Fiona McGoldrick, IRIS • Niall McSweeney, National University of Ireland, Galway • Claire Moran, University College Dublin • Val Payne, National University of Ireland, Maynooth

8. Activities • Downtime register • Usage statistics • User survey • Ongoing interaction with: • Steering Group • Users • Vendors

9. Downtime (known)

10. Usage Statistics • Excel templates for e-journals, databases • Basic quantitative indicators of uptake and value • Number of downloads/searches • Cost per download/search • Top 10 journals per resource according to downloads • Number and % of journals per download “band” • Turnaways • Annual frequency, with trend reports

11. E-Journal Example

12. Database Example

13. Annual Report of IReL Usage • Mix of figures and commentary • Summary table of download volumes and costs • Most downloaded journals overall • Analysis by download band • Usage by type of resource • Trends by discipline • Comparison with earlier year(s)

14. Value for Money

15. Consolidation of Usage

16. Trends of interest (STM only) • Strong uptake in nursing, chemistry • Journal of Advanced Nursing has most downloads • Cost per download compares very favourably to ILL • Usage tends to increase over time • Significant % of journals with <50 downloads • Lower usage, higher costs for non-journal resources

17. Compilation Difficulties • Labour-intensive • Mix of COUNTER/non-COUNTER data • Costs – need to factor in: • IReL /local payments initially • “maintained spend” • VAT • Total consortium figures • Some vendors slow to respond • Timing, eg synchronisation with subscription decisions • Unanswered questions, eg impact, quality, satisfaction?

18. 2007 IReL Impact Survey • Given priority over 2006 usage stats compilation • Essential complement to statistical data • Pre-consultation with researchers and funders • Focused distinctively on: • Value to researchers • Purpose of use • Impact on work • Satisfaction with coverage • (Recognised) use of IReL resources • Role of print • Access

19. Who Participated? • 2266 researchers in all disciplines • Staff • Research only, eg centres • Research and teaching • PhDs • Research Masters • 7 institutions

20. Findings of Note • IReL includes 75% of researchers’ “top 5” journals • But… gaps include journal backfiles, newspaper archives • Significant access (eg off-campus) and discovery issues • Lack of association with IReL • 55% don’t need print copies of IReL journals

21. Use of IReL

22. How IReL Benefits Research • Speed • Ease of online access • Coverage, including multidisciplinary • Currency • Stronger competitiveness • Easier collaboration • New areas of research now possible

23. How IReL Benefits Teaching • Faster transfer of ideas to lecture hall • Integration of online journals in Blackboard • Easier access to course readings • Wider choice of sources • Updated teaching materials

24. IReL is a Luxury, not a Necessity Discontinuation = “disaster”, “Dark Ages”, “would leave”

25. Blackwell Synergy: AHSS users

26. Survey experience • Labour-intensive • Seemed to engage senior stakeholders more than stats • Good on impact, quality of experience • Influential in likely continuation of IReL funding • Helpful in identifying specific gaps in coverage • But important to correlate findings with stats

27. Different Messages?

28. Usage Data: areas for development • Zero use titles? • Correlation with impact factors? • Cost per student/staff FTE? • E-book data • Outputs, eg researcher publications

29. In Conclusion • Usage data and user survey complementary • Stats • valuable indicators of activity • identifiers of uptake and value for money • guidance on subscription decisions • what and when? • Survey: • actual user experience • satisfaction levels • impact, return on investment • who, how and why?