Martha Kyrillidou Brinley Franklin Terry Plum

1. Assessing the Value of Networked Electronic Services: The MINES survey MINES for LibrariesTM ASSOCIATION OF RESEARCH LIBRARIES Martha Kyrillidou Brinley Franklin Terry Plum

2. ARL Statistics and Measurement …To describe and measure the performance of research libraries and their contribution to teaching, research, scholarship and community service …

3. ARL Statistics and Measurement …To describe and measure the performance of research libraries and their contribution to teaching, research, scholarship and community service …

4. Web Presence

5. StatsQUAL™

6. What is MINES? • A research methodology consisting of a web-based survey form and a sampling plan. • Measures who is using electronic resources, where users are located at the time of use, and their purpose of use • Adopted by the Association of Research Libraries (ARL) as a part of the “New Measures” toolkit in May, 2003. • Different from other electronic resource usage measures that quantify total usage (e.g., COUNTER, EQUINOX, E-Metrics, ICOLC guidelines, ISO and NISO standards) or measure how well a library makes electronic resources available (LibQUAL+TM, DigiQUALTM).

7. Questions Addressed • How extensively do sponsored researchers use the new digital information environment? • Are researchers more likely to use networked electronic resources from inside or outside the library? • Are there differences in usage of electronic information based on the user’s location (e.g., in the library; on-campus, but not in the library; or off-campus)? • What is a statistically valid methodology for capturing electronic services usage both in the library and remotely through web surveys? • Are particular network configurations more conducive to studies of digital libraries patron use?

8. MINES Methodological Considerations • A representative sampling plan, including sample size, is determined at the outset. Typically, there are 48 hours of surveying over 12 months at a medical library and 24 hours a year at a main library. • Random moment/web-based surveys are employed at each site. • Participation is usually mandatory, negating non-respondent bias, and is based on actual use in real-time. • Libraries with database-to-web gateways or proxy re-writers offer the most comprehensive networking solution for surveying all networked services users during survey periods.

9. Library web architecture

10. Web Survey Design Guidelines • Simple text for different browsers – no graphics • Different browsers render web pages differently • Few questions per screen or simply few questions • Easy to navigate • Short and plain • No scrolling • Clear and encouraging error or warning messages • Every question answered in a similar way - consistent • Radio buttons, drop downs • ADA compliant • Introduction page or paragraph • Easy to read • Must see definitions of sponsored research. • Can present questions in response to answers Dillman, D.A. 2000 (December). Mail and Internet Surveys, The Tailored Design Method. 2nd Ed. New York: John Wiley & Sons.

11. Library User Survey

12. Recent Data Collection Activities • Data was collected at seven main campus libraries and seven academic health science libraries in the U.S., between January, 2003 and January, 2005.

13. Recent Data Collection Activities • More than 45,000 networked electronic • services uses were surveyed. • At each library, the MINES survey was one component of a comprehensive • cost analysis study that assigned all library costs to sponsored research, instruction/education/non-sponsored research, patient care, other sponsored activities and other activities. • MINES is also being conducted by the Ontario Council of University Libraries (OCUL) in 2004-2005 in conjunction with ARL.

14. Purpose of Use By Location Medical Libraries 2003 – 2005 On-Campus, Not in the Library n = 15,423 In the Library n = 6,312 Overall Use n = 26,110 Off-Campus n = 4,375 *83% of sponsored research usage occurred outside the library. 92% of this use took place on-campus.

15. Purpose of Use By Location Main Campus Libraries2003 – 2005 On-Campus, not in the Library n = 6,641 In the Library n = 7,618 Off-Campus n = 5,012 All Usage n = 19,271 • 72% of sponsored research usage of electronic resources occurred outside the library; 83% of this took place on campus.

16. Location of Users2003-2005 Medical Library Main Library 76% 60% 24% 40% All Libraries 70% 30%

17. Demographics by Location of UserMain Libraries Inside the Libraryn = 7,064 On Campus, Not in the Libraryn = 6,391 Off-Campusn = 4,953 Total Usersn = 18,408

18. Demographics by Location of User Medical Libraries Inside the Library n = 6,635 On Campus, Not in the Library n = 19,244 Off-Campus n = 5,133 Total Usersn = 31,012

19. Electronic Services Sponsored Research Use Compared to Print Journal and Total Library UseMedical Libraries

20. Electronic Services Sponsored Research Use Compared to Print Journal and Total Library UseMain Libraries

21. martha@arl.org brinley.franklin@uconn.edu terry.plum@simmons.edu http://www.arl.org/stats/newmeas/mines.html