Web-Based Usage Surveys MINES for Libraries TM

1. Web-Based Usage SurveysMINES for LibrariesTM Brinley Franklin University of Connecticut Brinley.franklin@uconn.edu Terry Plum Simmons GSLIS Terry.plum@simmons.edu CNI Fall 2005 Task Force Meeting Phoenix, AZ December 5, 2005

2. Research problem Assessment of networked electronic services Open access – beyond vendor supplied data Issues with web-based surveys in assessing the use of networked electronic services Digital libraries – how to assess Project MINES for LibrariesTM Web based usage and user survey Assessment infrastructure and gateway Examples – data from: Medical libraries Large academic libraries Consortia Overview

3. Vendor supplied data - standards COUNTER - Counting Online Usage of Networked Electronic Resources http://www.projectcounter.org/ ICOLC – International Coalition of Library Consortia http://www.library.yale.edu/consortia/ ISO – International Standards Organization ISO 11620 Library Performance Indicators http://www.iso.org/ NISO – National Information Standards Organization NISO Z39.7 Library Statistics http://www.niso.org/ ARL E-Metrics http://www.arl.org/stats/newmeas/emetrics/index.html are helpful and trusted LibQUAL+TM Suite of services that libraries use to solicit, track, understand, and act upon users’ opinions of service quality. http://www.libqual.org/index.cfm DigiQUALTM Digital library service quality – set of 180 items tested against five collections Assessment of networked services

4. ISO standards for the electronic collection include e-books, electronic databases, ejournals, and digital documents. Free internet resources are counted separately. Yet, library liaison and subject web pages include free resources. Other electronic resources are important, including: Digital libraries – local, archival resources Pre-print and post-print servers Open access journals Open access repositories (e.g., institutional repositories) Electronic theses and dissertations servers Can we assess use of these resources as well as vendor supplied databases and ejournals? Open access --Beyond vendor supplied data

5. Advantages Can go beyond gross usage counts, using sampling User and use information E.g.; can query purpose of use, the discipline or affiliation, the status of the patron Opinion Service quality perception Can be tied to usage Point of use Can fold in other services (e.g., ILL, Ask A Librarian) Methodological issues Sample, not census Replicable? Truly random sample is difficult to obtain High non-response rate – are the responses representative of the population? Survey not seen Survey seen, but not filled out Users, not usage Remembered, predicted or critical incident reports – perceived to be not as reliable as vendor census based on usage Browsers can influence the display of the web survey And more. See for example: Holly Gunn, “Web-based Surveys: Changing the Survey Process”, First Monday, Issue 7, 2002. D.A. Dillman, Mail and Internet Surveys, The Tailored Design Method. 2nd ed., 2000. Web-based usage surveys

6. MINES is a transaction-based research methodology consisting of a web-based survey form and a random moments sampling plan. MINES typically measures who is using electronic resources, where users are located at the time of use, and their purpose of use. MINES measures usage as well as users. MINES is administered at the point of use, and surveys repeat usage during the sample period. MINES was adopted by the Association of Research Libraries (ARL) as part of the “New Measures” toolkit in May, 2003. MINES is different from other electronic resource usage measures that quantify total usage (e.g., COUNTER, E-Metrics) or measure how well a library makes electronic resources accessible (LibQUAL+TM). MINES for LibrariesTM

7. MINES for LibrariesTM

8. Assessment Infrastructure On Campus Patrons In Library Patrons ? Off campus patrons

9. Assessment Gateway online catalog federated searching

10. Demographics by Location of UserU.S. Main Libraries MINES for Libraries™ Inside the Libraryn = 9,172 On Campus, Not in the Libraryn = 8,838 Off-Campusn = 7,688

11. MINES for Libraries™ Demographics by Location of User U.S. Medical Libraries On Campus, Not in the Library n = 19,582 Off-Campus n = 5,133 Inside the Library n = 6,819

12. MINES for LibrariesTM Demographics by Location of User Ontario Council of University Libraries On Campus, Not in the Library n = 7,090 Inside the Library n = 4,047 Off-Campus n = 9,163

13. Purpose of Use By Location Medical Libraries 2003 – 2005 On-Campus, Not in the Library n = 15,423 In the Library n = 6,312 Overall Use n = 26,110 Off-Campus n = 4,375 *83% of sponsored research usage occurred outside the library. 92% of this use took place on-campus.

14. Purpose of Use By Location Main Campus Libraries2003 – 2005 On-Campus, not in the Library n = 6,641 In the Library n = 7,618 Off-Campus n = 5,012 All Usage n = 19,271 • 72% of sponsored research usage of electronic resources occurred outside the library; 83% of this took place on campus.

15. MINES for LibrariesTM Location of Electronic Resources Users U.S. Main Libraries Total Usersn = 25,698 U.S. Medical Libraries Total Users n = 31,883 OCUL (Canada) Libraries Total Usersn = 20,300

17. OCUL Data from MINES

18. brinley.franklin@uconn.edu terry.plum@simmons.edu http://www.arl.org/stats/newmeas/mines.html Web-Based Usage SurveysMINES for LibrariesTM