LibQUAL+ ™ Origins, Design, Interpretation

1. LibQUAL+™Origins, Design, Interpretation La Calidad en las Bibliotecas Conferencia Palma de Mallorca 13-14 January 2005 Fred Heath Vice Provost and Director, University of Texas Libraries

2. Why Assessment? “In an age of accountability, there is a pressing need for an effective…process to evaluate and compare research libraries.” • 124 Association of Research Libraries (ARL) alone, over $3.2 billion dollars were expended in 2000/2001 • 500 LibQUAL+ participants in Lib QUAL+ Note. M. Kyrillidou and M. Young. (2002). ARL Statistics 2000-01. Washington, D.C.: ARL, p.5.

3. The Challenge of Assessmentin Libraries • Traditional statistics emphasize inputs, expenditures, acquisitions, holdings, etc. • Help funding agencies understand success of their investments • No demonstrable relationship between expenditures and service quality—spending money is not enough…. • Lack of metrics describing outcomes: how can we measure success from the user’s point of view • Need to redesign library services to better meet changing patterns of use

4. Libraries Remain a Credible Resource in 21st Century 98% agree with statement, “My … library contains information from credible and known sources.” Note. Digital Library Federation and Council on Library and Information Resources. (2002). Dimensions and Use of the Scholarly Information Environment.

5. Changing Behaviors Recent Survey: Only 15.7% agreed with the statement “The Internet has not changed the way I use the library.” Note. Digital Library Federation and Council on Library and Information Resources. (2002). Dimensions and Use of the Scholarly Information Environment.

6. Library Use SummaryLibQUAL+ Spring 2003 Survey – UT Austin Aggregate

7. Library Use SummaryLibQUAL+ Spring 2003 Survey – UT Austin Aggregate “Googleization” Users and funders alike can begin to wonder about the relevance of libraries

8. “…everyone in class tried to get those articles on line and some people didn’t even bother to to to the stacks when they couldn’t Google them.” Graduate Student NYT Online 6/21/04 (Katie Hafner, “Old search engine in the the library tries to fit into a Google world”)

9. Facilities Usage: University of TexasEntrance Statistics - UT Austin Libraries 1991-2003

10. Printed Book Circulation: All ARL Libraries Note. M. Kyrillidou and M. Young. (2002). ARL Statistics 2000-01. Washington, D.C.: ARL, p.7.

11. Research Behavior: Personal Control When searching for print journals for research: • Only 13.9% ask a librarian for assistance • Only 3.2% consider consulting a librarian a preferred way of identifying information Note. Digital Library Federation and Council on Library and Information Resources. (2002). Dimensions and Use of the Scholarly Information Environment.

12. Reference Decrease: All ARL Libraries Web-savvy users wish to be able to negotiate the information labyrinth on their own terms Note. M. Kyrillidou and M. Young. (2002). ARL Statistics 2000-01. Washington, D.C.: ARL, p.7.

13. Searches for Online Journals: University of TexasUT Austin Libraries 2002-2004 Monthly

14. Web Usage: University of TexasTotal File Requests - UT Austin Libraries 2000-2003

15. Enter LibQUAL+: A response to • The necessity of assessment • Rapid shifts in information-seeking behavior • The reallocation of resources from traditional services into technology-enabled inquiry

16. LibQUAL+™ Goals • Improve mechanisms and protocols for evaluating libraries • Develop web-based tools for assessing library service quality • Identify best practices in providing library service • Support libraries seeking to understand changes in user behavior • Assist libraries seeking to re-position library services in the new environment

17. LibQUAL+™ Outcomes • Securing information that contributes meaningfully to planning and improvement efforts at a local level • Providing analytical frameworks that institutional staff can apply without extensive training or assistance • Helping decision-makers understand success of investments • Finding useful inter-institutional comparisons

19. 76 Interviews Conducted • University of Minnesota • University of Pennsylvania • University of Washington • Smithsonian Institution • Northwestern Medical • York University (Canada) • University of Arizona • Arizona State • University of Connecticut • University of Houston • University of Kansas

20. LoadedPT:P1:01xxxxxxxxxxxxxx.txt,S:\Admin\Colleen\ServQual Interviews\TEXT Only\01xxxxxxxxx.txt (redirected: c:\zz\atlasti\fred Atlas Ti

23. LibQUAL+™ Participants

25. Validity Correlations Validity Correlations Serv_Aff Info_Con LibPlace TOTALper Serv_Aff 1.0000 .7113 .5913 .9061 Info_Con .7113 1.0000 .6495 .9029 LibPlace .5913 .6495 1.0000 .8053 TOTALper .9061 .9029 .8053 1.0000 ESAT_TOT .7286 .6761 .5521 .7587 EOUT_TOT .5315 .6155 .4917 .6250

26. alpha By Language By Language Service Info. Lib as Group n Affect Control Place TOTAL American (all) 59,318 .95 .91 .88 .96 British (all) 6,773 .93 .87 .81 .94 French (all) 172 .95 .90 .89 .95

28. Survey Structure – Page 2

29. “And a Box” Why the Box is so Important • About 40% of participants provide open-ended comments, and these are linked to demographics and quantitative data. • Users elaborate the details of their concerns. • Users feel the need to be constructive in their criticisms, and offer specific suggestions for action.

30. Rapid Growth in Other Areas • Languages • American English • British English • French • Dutch • Swedish • Consortia • Each may create 5 local questions to add to their survey 3. Types of Institutions • Academic Health Sciences • Academic Law • Academic Military • College or University • Community College • European Business • Hospital • Public • State 4. Countries Canada, the Netherlands, South Africa, Sweden, France, Australia, New Zealand, Malaysia, U.K., U.S.

31. Understanding LibQUAL+ Results • Measures the distance between minimally acceptable and desired service quality ratings • Perception ratings ideally fall within the Zone of Tolerance

32. LibQUAL+ Survey Tool • Conducted at UT Austin in 2001, 2002 and 2003, 2004 • Web-based survey sent to 1200 faculty, 1200 graduate students and 1800 undergraduates • Participants selected randomly from University email databases • 22 questions measuring users’ perceptions of library service quality

33. LibQUAL+™ 2004 Summary Colleges or Universities –Faculty - American English Question view Dimension view (n = 11,755)

34. Key to Radar Charts

35. Key to Bar Charts

36. LibQUAL+™ 2004 Summary Colleges or Universities –Faculty - American English Negative gap Positive gap (n = 11,755)

37. Institutional Norms for PerceivedMeans on 22 Core Questions Note: Thompson, B. LibQUAL+ Spring 2002 Selected Norms, (2002).

38. Access to Information by StatusLibQUAL+ Spring 2003 Survey – UT Austin Faculty at Texas less approving of collection quality than students

39. Library as Place by StatusLibQUAL+ Spring 2003 Survey – UT Austin Students rate “Library as Place” more disapprovingly than Faculty (size of gap)

40. Four Dimensions – Social Science & PsychologyLibQUAL+ Spring 2003 Survey – UT Austin Our Psychology faculty do not rate their access to needed collections approvingly

41. Trends: Access to Information by Status Look for steadily improving trajectories

42. LibQUAL™ Interactive Institutional Statistics Your peer list of institutions Master List

43. The very act of administering LibQUAL+™ is beneficial

45. LibQUAL+™ Resources • LibQUAL+™ Website:http://www.libqual.org • Publications: http://www.libqual.org/publications • Events and Training: http://www.libqual.org/events • LibQUAL+™ Bibliography: http://www.coe.tamu.edu/~bthompson/servqbib • LibQUAL+™ Procedures Manual: http://www.libqual.org/Information/Manual/index.cfm

46. LibQUAL+™ Contact Information • Martha Kyrillidou • Senior Program for Office of Statistics and Measurement • martha@arl.org • Consuella Askew • LibQUAL+™ Program Specialist • consuella@arl.org • Amy Hoseth • LibQUAL+™ Project Assistant • amyh@arl.org • Jonathan D. Sousa • Technical Applications Development Manager • jonathan@arl.org

47. This presentation available at: http://webspace.utexas.edu/fh355/www

48. Core Questions SummaryLibQUAL+ Spring 2003 Survey – ARL Faculty

49. UT Austin vs. PeersLibQUAL+ Spring 2003 Survey – UT Austin