Bonnie Allen Associate University Librarian for Collection Development and Technical Services

1. What our users are telling us:service quality survey results for Oregon State University Libraries Bonnie Allen Associate University Librarian for Collection Development and Technical Services October 11, 2002

2. Background: • Associate Research Libraries (ARL)search for a survey instrument to establish service assessment • ARL obtained FIPSE grant funding for three year research and development project • Collaboration with Texan A&M University Libraries

3. Goals of the project: • Define and measure library service quality across institutions • Create useful quality assessment tool and protocol for library service quality • Identification of best practices • Evolve into a service quality assessment program at ARL

4. Why • Assessment is increasingly required by funding agencies and accrediting bodies • Library’s need to measure effectiveness of services • Need to define quality in libraries beyond the size of collections

5. Development of LibQual+TM • Project began 2000 • Based upon ServQual, a service measurement tool used in business • Based on Gap Theory of ServQual which measures difference in minimum and perceived services when perceived is below minimum requirement. • Dimensions

6. Grounded in Qualitative data • Grounded Theory focuses on general method of constant comparative analysis which generates or elaborates theories as data is gathered • 60 interviews (faculty, grad, undergrad students) at various ARL institutions • Interviews followed the dimensions established in ServQual: Tangibles, Reliability, Responsiveness, Assurance and Empathy

7. Qualitative Information: • Users have high need for self-reliance • Skills developed from trial and error • Instruction develops self-reliance when integrated with course when timing meets need • Users want libraries to be a caring and helpful place (Empathy dimension) “the way librarians handle these people is a big factor”

8. Qualitative Information • Undergraduates service perception is especially sensitive to uncaring responses • Aura of approachability is an issue • Knowledge and courtesy (assurance dimension) very important • Faculty have a generally positive view of individual librarians but see the profession as having a “circle the wagons” mentality

9. Qualitative information: Responsiveness • Comprehensive collections uniformly valued as key to academic success-especially for faculty • General understanding of budgetary issues but highly value institutional commitment to more than minimum resources • Access: the delivery of information when/where needed in the medium of choice

10. Qualitative Information: Responsiveness • Access to electronic resources raises expectation of service while introducing new reliability issues • ILL acceptable access but inefficient for researcher’s elimination of sources • Library web site role important to access

11. Qualitative Information: • Reliability issue from the lack of accuracy in records for material status • Library as place: facility condition only an issue when it impairs self-reliance • Library as symbolic of life of the mind • Self-reliance and quality service is a “chicken and egg” situation

12. Qualitative information: summary • Research library is expected to work simultaneously on several levels • Critically important were staff who were informed, courteous engage in their roles as they interacted with users. • Users expect dignified and solicitous understanding of their needs

13. Access to collections Assurance Empathy Library as place Reliability Responsiveness Tangibles Self-reliance Instruction Dimensions of LibQual+ TM

14. Definitions of Dimensions • Reliability: providing service dependably and accurately • Responsiveness: willingness to provide prompt service • Tangibles: physical facilities, equipment, and appearance • Self-reliance: services that enable the user to navigate on his/her own

15. Definitions of Dimensions • Instruction: teaching in classes and one on one • Access to collection: hours, stacks, collections • Assurance: knowledge and courtesy of employees and their ability to inspire trust and confidence

16. Definitions of Dimensions • Empathy: caring, individualized attention • Library as place: a calm, reflective place to do work

17. Scoring • Rating scale of 1-9, 9 being highest • Minimum required, perceived, desired • Mean scores are reported • Normative scores in development for inter-institutional comparison and to place scores in context

18. Validity testing • Alpha calculated testing revealed scores to be extremely reliable Alpha=.964 • Regression analysis

19. OSU Survey • Spring, 2001 • Second round of LibQual+ • Administered to 43 institutions (8 ARL and 35 non-ARL) • 20,000 respondents total

20. Protocol • Permission from IRB obtained • Respondents randomly selected from patron loads to our library system • Email extracted from the system and invitations to participate sent • Survey administered online, data collected and analyzed at Texas A&M

21. OSU Respondents • 329 respondents • 53% female, 47% male • Undergraduates: 20.7% • Graduates: 35.3% • Faculty: 26.7 % • Library Staff: 12.5% • Other: 4.9%

22. OSU Respondents by college • Science : 25.8% • Other: 18.2% • Agriculture/Environmental: 15.2% • Health Sciences: 9.7% • Social Sciences : 8.2% • Engineering: 7.9% • Education : 6.1%

23. OSU respondents by age • Younger than 22: 12.2% • 22-30 : 29.2% • 31-45: 28.3% • older than 45: 30.4%

24. Approach to the data • Broadest view: overall satisfaction scores • Cumulated scores for each dimension by subgroup • Examination of scores for each question to identify problem areas • Comparison of scores with the aggregate (41 institutions) to place in context

25. OSU service quality 7.21 63 percentile normalized service affect 7.3 responsiveness assurance empathy support for scholarly effort 6.91 Aggregate service quality 6.99 46 percentile for ARL normalized service affect 7.23 support for scholarly effort 6.81 Overall satisfaction scores

27. Access to collection Comprehensive print collection (27%ile) Complete run of journal titles (27%ile) Reliability Accuracy in the catalog, borrowing and overdue records (55%ile) *Gaps for Aggregate institutions as well Gaps in the all-ranks view

28. Areas where perceived service exceeds desire • Instruction • Library staff focusing on the here and now, not teaching search skills for the future. • Tailoring a response to meet a particular request rather than teaching me more than I need to know Tangibles • Visually appealing

29. Assurance Employees who have knowledge to answer user questions Responsiveness Willingness to help users. Areas where perceived service is less than minimum (Gaps) by Undergraduates

30. Access to collections full text delivered electronically comprehensive print collections complete runs of journal titles resources added on request Reliability accuracy in catalog, borrowing and overdue records providing services as promised Gaps reported by Graduate students

31. Responsiveness readiness to respond to user questions prompt service to users keeping users informed about when services will be performed Self-reliance enabling me to find information my self 24 hours a day making electronic resources accessible from my home or office Gaps reported graduates students

32. Graduate students most critical • In all areas pertaining to access to collections, OSU graduate students were more critical than the Aggregate scores for graduate students. • OSU collections do not meet graduate student need nor are we succeeding in basic service issues

33. Gaps reported by OSU Faculty • Access to Collection • comprehensive print collection • complete journal runs

34. Gaps suggested by Library Staff: Are we worried about the right things? • Access to collections • complete journal runs • comprehensive collections • Assurance • willingness to help users • Reliability • Accuracy in records • dependability in handling user problems • Self-reliance issues

35. In closing: • The LibQual+ research project presents libraries with a tool to obtain and evaluate the quality of the services from the user perspective. • The information to date points to greater acceptance of remote services but also greater expectations • The need for humaness in our service is strong • Librarians and libraries are held in high esteem • The complexities of libraries are recognized as users strive to be self-reliant

36. Next Steps • Participate in LibQual+ TM 2003 • Develop assessment tools to further describe the service issues • Communicate the results of this research project to university community • Communicate with our users our progress toward resolving access issues

37. Bibliography • www.coe.tamu.edu/~bthompson/servqbib.htm • updated bibliography Principal investigators: Colleen Cook, Texas A& M Bruce Thompson, Texas A&M Fred Heath,