Online Northwest Twentieth Annual Conference Corvallis, Oregon • February 28, 2003

1. LibQUAL+TM: An Innovative, Technology- Based Assessment Tool for Libraries Online Northwest Twentieth Annual Conference Corvallis, Oregon • February 28, 2003 Julia Blixrud, ARL Director of Information Services

2. Why New Measures • Increased customer and stakeholder expectations • Greater demands for accountability • Exploding growth in use and applications of technology • Increasing competition for resources • Need for reliable and valid data • Benchmarking and best practice • Trends over time

3. ARL New Measures Initiative • Collaboration among member leaders with strong interest in this area • Specific projects developed with different models for exploration • Projects self-funded by interested members • Intent to make resulting tools and methodologies available to full membership and wider community • Freeze modifications to existing descriptive measures

4. Antecedents • Effective service delivery • “every unit … is valued in proportion to its contribution to the quality success of the campus” Danuta Nitecki

5. LibQUAL+™ Description LibQUAL+TM is a research and development project undertaken to define and measure library service quality across institutions and to create useful quality-assessment tools for local planning.

6. Project Resources • LibQUAL+TM is an ARL/Texas A&M University joint effort. The project is supported in part by a 3-year grant from the U.S. Department of Education’s Fund for the Improvement of Post-Secondary Education (FIPSE).

7. The Purpose of the Research • To fill a knowledge void in modeling the dimensions of library service quality from a user perspective • Based upon the model, to develop a web-delivered, effective total market survey instrument equivalent for service quality assessment in academic libraries • Using the derived instrument to recommend a process for an ongoing program of comparative outcome measurement for academic libraries

8. LibQUAL+™ Project Goals • Establishment of a library service quality assessment program at ARL • Development of web-based tools for assessing library service quality • Development of mechanisms and protocols for evaluating libraries • Identification of best practices in providing library service

9. Relationships: Perceptions, Service Quality and Satisfaction ….only customers judge quality; all other judgments are essentially irrelevant” Zeithaml, Parasuraman, Berry. (1999). Delivering quality service. NY: The Free Press.

10. Source: Parasuraman, ARL Symposium on Measuring Service Quality Washington, DC, October 2000

11. 70+ Interviews Conducted • York University • University of Arizona • University of Connecticut • University of Houston • University of Kansas • University of Minnesota • University of Pennsylvania • University of Washington • Smithsonian • Northwestern Medical

12. Content Analysis Software Atlas Ti

14. Dimensions of Library Service Quality

15. Spring 2000 (41-Item Survey) Spring 2001 (56-Item Survey) Spring 2002 (25-Item Survey) Affect of Service Affect of Service Service Affect Reliability Library as Place Library as Place Library as Place Reliability Personal Control Provision of Physical Collections Self Reliance Information Access Access to Information Access to Information Survey Dimensions

16. Affect of Service • Emerged as the dominant factor early in our work • Absorbed several of the original SERVQUAL questions measuring Responsiveness, Assurance and Empathy • In the current analysis also includes Reliability • All in all: the Human Dimension of Service Quality

17. Library as Place • Transcends the SERVQUAL dimension of Tangibles to include the idea of the library as the campus center of intellectual activity • As long as physical facilities are adequate, library as place may not be an issue

18. Personal Control • How users want to interact with the modern library • Personal control of the information universe in general and web navigation in particular

19. Access to Information • Ubiquity of access: information delivered in the format, location and time of choice • Comprehensive collections—either print or electronic

20. The Challenge of Analysis • There are few useful conclusions to be drawn from aggregate data of all institutions, because their missions and subsequent user expectations for service are too diverse • There are commonalities in service delivery profiles that merit further investigation • In the long run, information that may be derived from demographic responses of individuals may yield the richest data

21. LibQUAL+TM Participants Year 3 Year 2 164 Participants 43 Participants Year 1 12 Participants Spring 2000 Spring 2001 Spring 2002 For More Information about Participants: Visit the LibQUAL+ web site.

22. Process Timeline June/July 2002 August 2002 September 2002 October – December 2002 • Gather information about LibQUAL+TM survey • Determine if have appropriate resources • Identify budgetary requirements if any • Register for Spring 2003 survey • Subscribe to ARL-QUALITY listserv • Designate survey liaison/committee/project team • Register for LibQUAL+TM related workshops • Identify and initiate steps to obtain human subjects research approval from IRB • Identify sample groups • Identify best data source to obtain valid e-mail addresses for sample groups • Meet with person(s) who will be drawing e-mail addresses to determine process feasibility • Register for LibQUAL+TM related workshops

23. Process Timeline January 2003 February 2003 March-April 2003 May 2003 June 2003 July 2003 • Need to have IRB approval by mid-January • January 27-28, orientation session for participating libraries held during ALA Midwinter, Phildelphia, PA Attendance is required! • Complete online demographics questionnaire • Preview survey turned on • Draw final e-mail address samples • Spring 2003 survey open to public. • Send out survey announcements • Spring 2003 survey closed to public • Participants complete online post hoc survey • Survey results distributed • LibQUAL+TM evaluation questionnaire sent to participants

24. Survey Instrument • 3 scales: • Minimum • Desired • Perceived

25. Survey Instrument 9 point scale

26. Sample Survey

27. Sample Survey…continued

28. Sample Survey…continued

29. Sample Survey…continued

30. Sample Survey…continued

31. Sample Survey…continued

32. Sample Survey…continued

33. Technology Approach Software • Reduced HTML requirements • No client-side actions (no javascript, Java, Active-X, cookies) • ColdFusion (for website and load-balanced servers) • SQL Server 2000 (for database server)

34. Technology Approach Hardware • 4 servers at TAMU • 1 reserved for old.libqual.org site • 2 load-balanced web/application ColdFusion-only servers that serve up the web pages of the survey • Database server that collects all data • 1 development server at ARL

35. Hardware Diagram

36. Project Deliverables • Print and web-based results include: • Aggregate Summaries • Demographics by Library • Item Summaries • Dimension Summaries • A copy of the survey instrument • Dimensions measured for survey implementation

38. Surveys Completed Spring 2002

41. LibQUAL+ Assessment Survey Aggregate (All Ranks) (All) Aggregate (All Ranks) Texas A&M University

43. Aggregate Dimension Summary (n=70,445) Note: LibQUAL+ Spring 2002 Aggregate Survey Results. (2002). vol. 1, p. 24

44. Interpretation Framework Zone of Tolerance • The area between minimally acceptable and desired service quality ratings • Perception ratings ideally fall within the Zone of Tolerance

45. Interpretation Framework Score Norms • Norm Conversion Tables facilitate the interpretation of observed scores using norms created for a large and representative sample • LibQUAL+TM norms have been created at both the individual and institutional level

46. LibQUAL+™ Fundamental Contributions to the Measurement of Effective Delivery of Library Services • Shift the focus of assessment from mechanical expenditure-driven metrics to user-centered measures of quality • Re-ground gap theory for the library sector, especially academic libraries • Grounded questions yield data of sufficient granularity to be of value at the local level • Determine the degree to which information derived from local data can be generalized, providing much needed “best practices” information • Demonstrate the efficacy of large-scale administration of user-centered assessment transparently across the web • Makes little demand of local resources and expertise

47. LibQUAL+™ Resources • LibQUAL+Web Site http://www. libqual.org • LibQUAL+Bibliography http://www.coe.tamu.edu/~bthompson/servqbib • Survey Participants Policies and Procedures Manual http://www.libqual.org/documents/admin/procedures3.8.pdf

48. Julia C. Blixrud Director of Information Services Association of Research Libraries 21 Dupont Circle, Ste 800 Washington, DC 20036 jblix@arl.org 202-296-2296 ext. 133 202-872-0884 (fax) 202-251-4678 (mobile)