INTRODUCTION

1. Evaluating NCknows Chat Reference Service From the Patrons’ Perspective Lili Luo, Jeffrey Pomerantz School of Information & Library Science University of North Carolina at Chapel Hill, Chapel Hill, NC luo, pomerantz@unc.edu 3. Meeting expectation of the patron. Patron obtained the exact information or information object that they were looking for. (2) “Yes, I looked up an audio book successfully and go to listen to it”. No. I didn’t use it. The patron didn’t get a chance to use it yet. (3) “No. not yet. I plan to call the number I got from the service in a couple” of time. The information provided by NCknows didn’t meet patrons’ real information need or the patron’s question was not answered. (5) “I went to the sight [site] they advised, but it really didn't help.” The patron didn’t explain why the information was not used. (1) N/A. (1) Partially useful. Referred information source is not useful. (1) “NCknows told me the person to call at my local library (Cumberland County Main) who said they can't provide that information...” Patrons’ questions are not fully answered but relevant information or useful leads are provided. (4) “I got a 50% answer from the NCknows, so I have to adress what I got to the Danish National Archives for further research. But, I did get new information for NCknows.” Patron only partially used the information. (1) “I have not had a chance to actually go to the journals that were listed but the list let me know that I wasn't overlooking any particular journal.” Patrons’ Access to NCknows (how they found out about NCknows) Online search. Patrons started from searching the web but stumbled across the service during their search. (8) “I came across it when I tried to google for the information about Robert Lowell.” Local library system, including library website, newsletter, librarian’s recommendation, etc. (27) “I saw the link on our library's home page.” School system , including college orientation, class, school website. (3) “I take classes through Central Carolina University. My teacher had told me about it before it became operational.” Others. (2) “My husband saw it on a web sight and e-mailed your sight [site] to look up ” Figure 4. Satisfaction with ease of use of the chat software. INTRODUCTION RESULTS Patrons Satisfaction with NCknows Chat Reference Service From the launch of the service till late October, 340 survey responses have been collected. Figure 1. Satisfaction with the completeness of the answer received. • This poster contains the analysis of exit survey results and telephone/email interviews with NCknows patrons. NCknows (www.ncknows.org) is a collaborative state-wide chat reference service, coordinated by the State Library of North Carolina’s Library Development Section (statelibrary.dcr.state.nc.us). Libraries of all types are participating: academic, public, and government; in research universities and community colleges, urban and rural, large and small. NCknows was launched in February 2004 and currently it is in an 18-month pilot phase. • Research Questions: • How patrons perceive NCknows chat reference service according to their own experience? • How patrons used the information provided by NCknows chat reference service and how the information has impacted patrons’ information need? • Data has been collected from the pop-up survey at the end of each chat session and telephone or email interviews with patrons who were willing to be contacted for further feedback. The results of data analysis leads to evaluation of patrons’ satisfaction with NCknows, their reasons to use it and their use of the results provided by NCknows. • From late March till mid-October, fifty email and telephone have been carried out. A preliminary analysis was conducted based on thirty four email interviews and 9 transcribed telephone interviews. • Patrons’ Reasons of Using NCknows • For a Task • School related (10) • “It’s for my master paper.” “I did this for a class I am in (UNCP) internet class.” • Work related (11) • “ I’m a small business owner and I need the information to negotiate with other type of business people to do a joint venture.” • For Personal Reasons (12) • “Just curiosity, plus every time I come across those words [Filipino and Philippine] in a newspaper, it makes me wonder.” • To Get an Object, Information or Perform an Activity(10) • “I was looking for an audio book and couldn't understand the new system.” • Patrons’ Use of Information Provided by NCknows • Yes. The information is useful. I had a chance to use it. • 1. Task accomplishments. • Accomplishment of school related task. (4) • “I was able to look up ECU disserations in order to complete my class assignment.” • Accomplishment of work related task. (9) • “Yes, I used the information. It was very helpful. I used a recommended book during a read aloud with 5th grade.” • 2. Personal achievements. • Knowledge expansion. Information provide by NCknows expanded the patrons’ knowledge ken. (3) • “Well, now I know the reason for the different spellings, and can tell someone else if they ever ask.” • Actions taken and decisions made for personal interests. Patrons applied information provided by NCknows in taking actions or making decisions.(4) • “ yes, I've picked up some of the books they recommended.” • Information obtained based on information leads provided by NCknows. (1) • Philosophical effect. Information provided by NCknows had philosophical effect on the patron. (1) • General comments. (1) Figure 2. Satisfaction with the speed that the librarian answered the question. METHODS • Two instruments have been employed in collecting data on patrons’ feedback on NCknows service. • Method 1. Exit survey at the end of each chat session • At the end of each chat session, patrons are asked to fill out a survey with a few multiple choice questions. Satisfaction is used as the major criterion to reflect patrons’ assessment of the interaction with NCknows service. It is embodied in the following aspects: • satisfaction with the completeness of the answer received • satisfaction with the speed that the librarian answered the question • satisfaction with the helpfulness of the librarian • satisfaction with ease of use of the chat software • Method 2. Telephone/email interviews with the patron • Semi-structured interviews through telephone or email were conducted to further elicit patrons’ assessment of the information provided to them by asking them their motivation to use the service and their use of the service results in relation to a specific context in their life. Figure 3. Satisfaction with the speed that the helpfulness of the librarian. CONCLUSION Analysis of patrons’ evaluative responses to the service enables us to assess NCknows from the patrons’ point of view. Understanding how they perceive the strength and weakness of the service would lead to improvement of NCknows so as to serve the patrons in a more efficient way. By exploring their use of the information provided by NCknows could we assess the value of the service to them and the impact the service has on their information needs.