Surfing the Tsunami of Service Quality: AAHSL/ARL Partnership in Exploring LibQUAL+ Outcomes Assessment

1. Surfing the Tsunami of Service Quality: The AAHSL/ARL Partnership in Exploring Outcomes Assessment through LibQUAL+

2. Tamera Lee, Director, Greenblatt Library, Medical College of Georgia, Chair, AAHSL Task Force on Service Quality Assessment • James Shedlock, Director, Galter Health Sciences Library, Northwestern University, Editor of AAHSL Annual Statistics, Task Force Liaison • Rick Forsman, Director, Denison Memorial Library, University of Colorado Health Sciences Center, AAHSL President 2002/Board Liaison Surfing the Tsunami of Service Quality

3. What is LibQUAL + • A new measurement initiative of the Association of Research Libraries and researchers at TAMU (Colleen Cook, Fred Heath and Bruce Thompson) • Define quality to complement the traditional quantitative library measures (number of volumes, amount of space, size of staff, etc.) Surfing the Tsunami of Service Quality

4. What is LibQUAL+? • National, broad-scale project to define and measure service quality across institutions and identify best practices • Web-based survey based on research grounded in gap analysis from the service quality business literature Parasuraman, Zeithaml and Berry Surfing the Tsunami of Service Quality

5. What is LibQUAL+? • Specifically the survey indicates how user expectations compare with user perceptions of service quality • Measure gaps between what service dimensions users most desire, what they expect and perceive they are getting Surfing the Tsunami of Service Quality

6. Purpose/Task Force • Outline the qualitative efforts of the AAHSL Task Force on Service Quality Assessment: • Jane Blumenthal-Georgetown University • Diana Cunningham-NY Medical College • Rick Forsman-University of Colorado • Tamera Lee-Medical College of Georgia • Logan Ludwig-Loyola University-Chicago • James Shedlock-Northwestern University • Julie Sollenberger-University of Rochester • Susan Starr-University of California, San Diego • Laurie Thompson-State University of New York, Upstate Surfing the Tsunami of Service Quality

7. The Association of Academic Health Sciences Libraries (AAHSL) • One of two consortia’s in the 2002 pilot of LibQUAL+™; the other was the Ohio Library and Information Network (OhioLINK), a consortium of Ohio’s college and university libraries and the State Library of Ohio. • Focus: the AAHSL/ARL partnership, the AAHSL aggregate experience, results and collaborative outcomes. Surfing the Tsunami of Service Quality

8. Background • With a long history of compiling and analyzing comparative statistics to serve as useful benchmarks for member libraries, AAHSL recognizes the increasing importance of outcomes measures to be used as performance indicators. Surfing the Tsunami of Service Quality

9. Background • AAHSL partnered with ARL for the participation of a cross section of 36 AAHSL libraries in the spring 2002 pilot. • The AAHSL project was funded generously by the National Library of Medicine, with shared costs by AAHSL and participating libraries to support deliverables. Surfing the Tsunami of Service Quality

10. Background • The Task Force on Quality Assessment was created in July 2001 to lead AAHSL’s involvement in a service quality assessment process and evaluate LibQUAL+™ as a qualitative measurement tool for academic health sciences libraries. Surfing the Tsunami of Service Quality

11. Task Force Goals • Testing the usefulness of the instrument for benchmarking • Testing the feasibility of customizing the instrument for the health sciences • Determining the value of cross data analysis with AAHSL statistics • Gaining experience with an aggregate outcomes measurement process Surfing the Tsunami of Service Quality

12. The Process • Coordinated the cohort registration and established a listserv to facilitate efficient communication • Drafted a procedural manual to serve as a guide and created a survey administration tip sheet for participants Surfing the Tsunami of Service Quality

13. Task Force Challenges • Clarified the importance of a clean set of collective data for AAHSL and defined demographics for respondents Surfing the Tsunami of Service Quality

14. Task Force Challenges • Developed 5 unique AAHSL questions in collaboration with participants and in consultation with the National Library of Medicine. • The main objective was to add service performance issues for the health sciences, especially priorities not included or covered sufficiently by the LibQUAL+™ survey. Surfing the Tsunami of Service Quality

15. Unique AAHSL Queries • Providing health information when and where I need it • Employees teaching me how to access or manage information • Environment that facilitates group study and problem solving • Access to information resources that support patient care • Having comprehensive electronic resources Surfing the Tsunami of Service Quality

16. Library Service Quality SurveyLibQual+ Surfing the Tsunami of Service Quality

17. Task Force Challenges • Guided discussion of human subjects clearance and disparities of IRBs regarding issues of anonymity, age, etc. Surfing the Tsunami of Service Quality

18. The Respondents • The TF recommended that the AAHSL participants survey their entire user populations. Only those who completed the survey in its entirety and with measurable consistency were included. • Over half (50.97%) of the people who entered the survey completed it. With each of the 4 pages viewed the percentage dropped 8-20%, most significantly at the 2nd (demographics) and 3rd pages. Surfing the Tsunami of Service Quality

19. Results • Chart—percentage of AAHSL respondents by user group (N=13,976) Surfing the Tsunami of Service Quality

20. Results • Chart—percentage of respondents for disciplines reflecting 5% or more (N=13,976) Surfing the Tsunami of Service Quality

21. Results • Chart—average scores for AAHSL aggregate rating general satisfaction from 1-9 (with 1 being “strongly disagree” and 9 representing “strongly agree”) Surfing the Tsunami of Service Quality

22. Frequency of Library Use—AAHSL Aggregate • Most respondents reported using the Library weekly, both on the premises (35.93%) and electronically (34.84%). • Daily electronic use was reported by 31.83% and monthly use on premises was reported by 31.42%. Surfing the Tsunami of Service Quality

23. Service Dimension Gaps that fell outside “zone of tolerance” • AAHSL aggregate—electronic resources accessible from home or office • AAHSL students—convenient business hours • AAHSL faculty—complete runs of journal titles, electronic resources accessible from home or office, comprehensive electronic resources Surfing the Tsunami of Service Quality

24. Dimensions Most Desired byAAHSL Aggregate • Comprehensive electronic resources (8.44) • Electronic resources accessible from home or office (8.35) Surfing the Tsunami of Service Quality

25. Dimensions Most Desiredby AAHSL Aggregate • Students: (undergrad and grad): having comprehensive electronic resources (8.33-8.50), followed closely by convenient business hours (8.32-8.43) • Faculty: having comprehensive electronic resources (8.49), followed closely by making electronic resources accessible from home or office (8.46) Surfing the Tsunami of Service Quality

26. Service Affect Dimensions MostDesired by Staff • Employees who are courteous consistently (8.55), • Willingness to help users (8.52) • Employees who have the knowledge to answer users’ questions (8.46) • Readiness to respond to users’ questions (8.46) Surfing the Tsunami of Service Quality

27. The Results • ARL reliability results indicate higher AAHSL user expectations and levels of satisfaction compared to the ARL environment. • Responses from the general academic campus reflect a higher response rate from undergraduate students, whose expectations and needs differ from the user profile of AAHSL institutions, consisting primarily of graduate level and above. Surfing the Tsunami of Service Quality

28. The Results • Useful view of the data is through T-scores and norm tables. • T-scores enables examination of individual score norms for faculty and students and institutional scores in relation to scores of peer institutions. Surfing the Tsunami of Service Quality

29. The Results • The overall norm table for institutions of all types shows that one health sciences library clearly stands out as exceeding expectations--even desires. • Mercer University School of Medicine Library and LRC has the only positive average superiority score, indicating overall perceptions are higher than desired. Surfing the Tsunami of Service Quality

30. Comments • Frequent strong themes shared included 1) need for more journals, especially online 2) frustration with remote access problems 3) positive comments about staff Surfing the Tsunami of Service Quality

31. AAHSL Queries“Comprehensive ElectronicResources” • Reflected the highest desired measure by all user groups • Represented a significant gap in minimal expectations for faculty Surfing the Tsunami of Service Quality

32. AAHSL Queries“Comprehensive Electronic Resources” • The ARL research team advised the task force to reconsider this question as an unreasonable service demand. Seemingly the sense is that comprehensive print resources may be more achievable. • For the 2003 LibQUAL+™ cohort iteration, this question was revised to provide more focus, “Electronic resources matching my information needs.” Surfing the Tsunami of Service Quality

33. AAHSL Queries • Summative data for other four AAHSL queries indicates needs for slight improvements for all user groups. Surfing the Tsunami of Service Quality

34. Participating Libraries-Outcomes • Data used to complement other assessment measures and create additional ones, such as focus groups. • A correlative test to consider the integration of qualitative and quantitative measures conducted by Lee, Dennison and Joubert showed no correlation between service affect gap score and annual expenditures. Surfing the Tsunami of Service Quality

35. Participating Libraries-Outcomes • Many shared presentations with respective senior administrations, academic senates, student government associations, physical facilities planners and library staff. • Some made immediate changes and others want to repeat the survey in 2003 for a better view. A few incorporated LibQUAL+™ data into their annual performance metrics and other review processes. Surfing the Tsunami of Service Quality

36. Task ForceRecommendations for AAHSL: • Participate in the final phase of pilot (2003 survey administration) for a cross section of 23 new and repeat participating libraries to optimize our assessment and maximize our deliverables Surfing the Tsunami of Service Quality

37. Task ForceRecommendations forAAHSL • Consider if and how a qualitative data component (such as LibQUAL+ survey results) may be used in conjunction with the annual quantitative stats for member libraries • Continue to support LibQUAL+ assessment and development, other qualitative measurement activities and future initiatives in outcomes assessment Surfing the Tsunami of Service Quality

38. Benefits-ARL • Assisted in furthering the applicability of LibQUAL+™ as a qualitative assessment tool across different types of institutions. • The special cohort assisted ARL with promotion, training and increased participation. Surfing the Tsunami of Service Quality

39. Benefits-AAHSL • Partnership created a heightened awareness of the viability of outcomes assessment and a cost efficient protocol for listening to users. • The additional five questions allowed AAHSL to include a construct for issues perceived as unique or more significant to health sciences libraries. • Prompted further interest in testing the instrument's feasibility for a new cohort of hospital libraries. • Advanced a culture of assessment within AAHSL and for its members. Surfing the Tsunami of Service Quality