E-Metrics: Lessons Learned from the ARL E-Metrics Project, Challenges and Opportunities

1. E-Metrics: Lessons Learned from the ARL E-Metrics Project, Challenges and Opportunities Martha Kyrillidou 226th American Chemical Society National Meeting CINF Section B, Jacob K. Javits Convention Center, Room 1E20 September 8, 2003 • New York, NY

2. Association of Research Libraries • Mission: Shaping and influencing factors affecting the future of Research Libraries in the process of scholarly communication • Members: 124 major research libraries in North America • Users: More than 3 million students and faculty served • Service measures: 75 million circulations 17 million reference transactions • 42,000 FTE staff employed • Members’ total expenditures are over $3.2 billion

6. LibQUAL+™ 2003 Colleges or Universities – Library Use

7. Expenditures for Electronic Resources 19.54% of the library materials budget

10. Developing measures and evaluation techniques for networked services will take time, effort, and on-going learning on everyone’s part – but we must begin now. (Carla Stoffle, University of Arizona) We not only need to measure things in new ways but we also need to measure new things. (Sherrie Schmidt, Arizona State University)

11. Data Collection • How/What to Collect – Sources of Data • Systems Based – OPAC, Web, Vendor Data (Canned Reports, Other Reports) • Electronic/Web Resource (costs & use) • Non-Systems (manual) Library Counts • Reference Activities • ILL Transactions • Document Delivery • Turnstile Counts • Other Manual Counts • Qualitative Data Sources (Surveys, Focus Groups, Process Analysis)

12. Data Examples Source: TUG (Tri-University Group of Libraries - University of Guelph, Wilfrid Laurier University & University of Waterloo) SilverPlatter ERL (Electronic Reference Library) Usage Statistics

13. Data Examples Example – Consortia CSA Database Use Data Source: TUG (Tri-University Group of Libraries - University of Guelph, Wilfrid Laurier University & University of Waterloo) CSA (Cambridge Scientific Abstracts) Usage Statistics

14. Recommended Statistics & Measures • Patron Accessible Electronic Resources (R1-3) • Use of Networked Resources & Services (U1-5) • Expenditures for Networked Resources & Related Infrastructure (C1-3) • Library Digitization Activities (D1-3)

15. Patron Accessible Electronic Resources • R1 – Number of electronic full-text journals • R2 – Number of electronic reference sources • R3 – Number of electronic books

16. Use of Networked Resources & Related Infrastructure • U1 – Number of electronic reference transactions • U2 – Number of logins (sessions) to electronic databases • U3 – Number of queries (searches) in electronic databases • U4 – Items requested in electronic databases • U5 – Virtual visits to library’s website and catalog

17. Expenditures for Networked Resources & Related Infrastructure • C1  Cost of electronic full-text journals • C2  Cost of electronic reference sources • C3  Cost of electronic books • C4  Library expenditures for bibliographic utilities, networks & consortia • C5  External expenditures for bibliographic utilities, networks & consortia

18. Library Digitization Activities • D1 – Size of library digital collection • D2 – Use of library digital collection • D3 – Cost of digital collection construction & management (Collecting these data requires staff familiar with the digital environment.)

19. The E-Metrics Assessment Implementation Loop Identify Tasks, Data & Needs Data Collection Mission/Purposes Objectives Managing Information Preparation Reporting, Learning & Implementing

20. Preparation for Assessing Network Services is as Important as the Assessment Itself !

21. Mind the Gap Mine the Gap

22. Thank you Martha@arl.org