“I am ecstatic about the whole darn thing”

1. “I am ecstatic about the whole darn thing” Customer Feedback and Lessons Learned at QandANJ, New Jersey’s Live Virtual Reference Service Karen Hyman & Peter Bromberg South Jersey Regional Library Cooperative Virtual Reference Desk Conference November 17, 2003

2. In today’s session, we’ll talk aboutQandANJ… • Brief overview • Data collection history/methodology • Data analysis • Customer comments • Conclusions/surprises

3. QandANJ…what is it? • An LSTA-funded project staffed by 37 libraries of all types in the NJ Library Network. • Free, interactive, search assistance on the Web. • Available 24/7. • Live since October 1, 2001. • 1500 live answers weekly and growing.

5. Libraries currently staffing QandANJ include: • Atlantic City Free Public Library • Burlington County College Library • Burlington County Library • Camden County Library • Centenary College • Cherry Hill Free Public Library • Cumberland County Library • East Brunswick Public Library • Gloucester County College Library • Gloucester County Library • Hunterdon County Library • Livingston Public Library • Mercer County Library • Middlesex County College • Middletown Public Library • Millburn Public Library • Monmouth County Library • Montclair Public Library • Mount Laurel Library • New Brunswick Free PL • New Jersey City University • New Jersey State Library • Newark Public Library • Ocean County Library • Old Bridge Public Library • Piscataway Public Library • Princeton Public Library • Somerset County Library • South Brunswick Public Library • UMDNJ – Stratford • UMDNJ - Newark • Vineland Public Library • Wayne Public Library • West Deptford Public Library • West Orange Public Library • William Paterson University • Woodbridge Public Library

6. QandANJ…Why are we doing it? • Market libraries and library services • Meet needs and expectations of 21st century customer • Immediacy • Shock value • Go where the people are

9. QandANJ…LSTA Funding history

10. QandANJ:rationale for data collection Find out what we want to know to support a fast moving, experimental culture • Who’s out there? • How are we doing? • What’s our overall impact on libraries?

11. Brief History of Data Collection Original customer Feedback form • 7 questions related to satisfaction • 4 questions related to demographics • 1 question related to marketing • Results sent to email • Results were eyeballed, and manually entered into spreadsheet.---ugh!

12. Brief History of Data Collection Questions relating to Satisfaction: • Use again? • Completely Answered? • Reasonable amount of time? • Satisfied? • What did you like? • Suggestions for improvement? • Other Comments Questions relating to Marketing: • How did you hear about us?

13. Keeping it simple! • Radio buttons for quick answers • Comment boxes for more detail

14. Brief History of Data Collection Revised Feedback form • Started writing results to a text file, and importing into a database—ahhh... • Revised wording; changed satisfaction rating to a 4 point scale for easy computation.

15. A few revisions! • 4 point, color-coded, scale. • Optional email for follow-up survey • Privacy statement

16. What did they tell us? • Results from “pop-up” customer feedback form, June 2002 – Oct 2003. • Total responses: 8745

17. What did they tell us? Gender Gender Number Percent Female: 4571 65.1% Male: 2451 34.9%

18. What did they tell us? Age Age Number Percent 18 and under: 3549 45.8% 19 – 35: 1357 17.5% 36 – 59: 2120 27.4% 60 and up: 724 9.3%

19. What did they tell us? Grade/Occupation Grade/Occupation Number Percent College: 1241 17.2% High School: 1488 20.7% Middle School: 1689 23.5% Elementary School 377 5.2% Professional: 2404 33.4%

20. What did they tell us? Was your question answered in a reasonable amount of time? Answer Number Percent Yes: 6029 73.7% No: 1144 14.0% Could be Quicker: 1013 12.4%

21. What did they tell us? Have you used Q and A NJ before? Answer Number Percent Yes: 2186 53.9% No: 1871 46.1%

22. What did they tell us? Would Use Service Again? Answer Number Percent Yes: 7548 89.8% No: 862 10.2%

23. What did they tell us? Was your question completely answered? Answer Number Percent Yes: 4999 60.1% No: 2102 25.3% Referred: 1214 14.6%

24. What did they tell us? Overall Satisfaction Satisfaction No. Percent 1-Very Satisfied 2398 58.4% 2-Fairly Satisfied: 895 21.8% 3-Fairly Unsatisfied 261 6.4% 4-Very Unsatisfied 550 13.4% Average Satisfaction Rating: 1.75 (approx “grade” equiv: 3.25) Combined: 80.2%

25. So our pop-up survey respondents were... • Often students • More female • Younger • Satisfied with service • Likely to use again

26. Follow-up Survey • We wanted more detail on customer demographics and satisfaction. • And to correlate with: • Library behaviors: past and future • Opinion of libraries • Demographics • 691 Customers provided emails • 251 Responded

27. Rating of Service: a solid B+ Overall Service: 3.62 Quality of Help: 3.69 Consistency: 3.55 Courteousness: 3.82 Convenience: 3.84 Ease of Use: 3.79 No tech. problems: 3.56 Accuracy: 3.46

28. Follow-up Survey Demographics • Female/male ratio (67/33) • Respondents tended to be older (only 13% under 18) • Ethnically diverse by heritage and languages spoken in the home (just like N.J.)

29. Follow-up Survey Demographics • Fairly educated (41% had attained more than a 4 year degree) • Computer literate. • 97%have a pc in their home • 88%use for word processing • 41%use for banking • 67%use for shopping • 43%used other chat-based services

30. Follow-up Survey Mostly Library Users, but not all... • 12% visit library yearly or less often • 4.2% NEVER seek help from a library when they have an information need.

31. Follow-up Survey Why do they visit the library? • 75%: Check out materials • 73 %: Research/Find Information • 59%: Browse • 40%: Get assistance from librarian • 25%: Study • 23%: Use library computers • 18%: Attend library programs • 10%: Attend non-library programs

32. Follow-up Survey Q and A NJ Usage: Why did they use us? • Expertise of librarian was cited as often as convenience and speed as reasons for using Q and A NJ. (66%) • “Like having a transcript” rated fairly low. (23.8%)

33. Follow-up Survey Q and A NJ Usage: Devoted Users! • 16% used service more than 10 times. • 27% used service more than 6 times. • 67% were repeat users. We must be doing something right!

34. Follow-up Survey Q and A NJ Usage: How did they find us? 35%: Link on library webpage 27%: W.O.M. Friends, family or teacher 13%: Article in paper or magazine 13%: Search Engine 9%: Posters/Bookmarks 7%: Link on non-library webpage 2%: Television

35. How has your experience with Q and A NJ affected your opinion of libraries? Improved: 66% Not affected: 32.5% Worsened: 1.5%

36. Our follow-up survey respondents were... • Educated • Female • Older • Computer savvy • Library card holders • Satisfied

37. We’re not making this up

38. QandANJ… Real feedback from real users

39. Interaction with a live person • I was talking to a real person not a computer! • Contact with a real human being – always a more pleasant experience than dealing strictly with computers. • Personalized one-on-one attention. • The personal, human contact. You’re not just communicating with a **!@#*!! machine! • Are you a person?

40. Interaction with a nice person • The librarian was on top of everything and very kind. • Very helpful and pleasant. • I felt like I was talking with a friend. • Very nice person who was interested in my predicament. • I liked that someone came on and didn’t make fun of me or anything about my question. • Valerie was cool. • Pat was so wonderful – make sure you pay her extra.

41. Expertise of the librarian • This particular question was something that stumped my daughter AND my husband AND me and the librarian not only found the answer, but told us how she found it! We were thrilled. • It’s exciting talking to a librarian who knows what she’s doing. • Very accurate!!! • Insurmountable quality.

42. Fast and convenient • That I don’t have to search around and get a million useless sites when I’m looking for the answer to a simple question. • Time is crucial!!! • The quick response. • Convenience and speed. • Fast, relevant information.

43. Always open • I got the answers I needed, especially when the library was closed.” [My] first contact was at 2 am Sunday morning. • Available 24 hours a day! • The library was closed. • Available Sunday morning! • So nice to find a librarian at 10:00 pm when my son remembered his homework.

44. Awesome • I am continually astounded by this service and it’s results never cease to please. • It was awesome and really helped. • I loved it!!! I think it is a fantastic service. • It was spectacular and Doris was very helpful. I’m telling all my friends!

45. In other words… “The speed of getting precise information. The fact that it was free. The polite professionalism of the librarian. The fact that it is valuable information. The fact that you can access it 24 hours a day. The fact that libraries are keeping up with the future of accessing information and sharing that with the public. It is quite a wonderful, helpful thing.”

46. Conclusions and surprises • Our follow up survey people -- middle aged, educated, computer savvy, library users -- exist! • Many QandANJ users may be library users or more likely to try us out. • People still need help finding answers. • Taking live reference service to the web 24/7 makes this much difference!

47. Contact Information: Karen Hyman, Executive Director Peter Bromberg, Program Coordinator South Jersey Regional Library Cooperative 10 Foster Avenue, Suite F-3 Gibbsboro, NJ 08026 856 346-1222 FAX: 856 346-2839 Email: hyman@sjrlc.org and bromberg@sjrlc.org

48. is administered by the South Jersey Regional Library Cooperative, a state tax-funded service of the New Jersey Library Network, and supported by Federal Library Services and Technology Act funds administered by the New Jersey State Library. Headquartered in Gibbsboro, the South Jersey Regional Library Cooperative is an organization of over 600 academic, public, school and special libraries in the southern seven counties and is one of four library cooperatives in New Jersey.