300 likes | 385 Views
Seeking Synchronicity: Evaluating Virtual Reference Transcripts. Presented by Marie L. Radford and Lynn Silipigni Connaway 2006 ALISE Conference San Antonio, TX, January 16-19, 2006 . Presenters Marie L. Radford, Ph.D. Associate Professor, Rutgers University, SCILS
E N D
Seeking Synchronicity: Evaluating Virtual Reference Transcripts Presented by Marie L. Radford and Lynn Silipigni Connaway 2006 ALISE Conference San Antonio, TX, January 16-19, 2006
Presenters • Marie L. Radford, Ph.D. • Associate Professor, • Rutgers University, SCILS • Email:mradford@scils.rutgers.edu • www.scils.rutgers.edu/~mradford • Lynn Silipigni Connaway, Ph.D. • Consulting Research Scientist • Email: connawal@oclc.org • www.oclc.org/research/staff/connaway.htm
Interpersonal Dimensions of Chat Research Stream • S.S. Green Award 44 transcripts • (courtesy LSSI) • Maryland AskUsNow! 245 transcripts • (courtesy Maryland AskUsNow! Statewide Consortium) • Library Lawline 113 transcripts • (courtesy NELLCO Regional Consortium) • Seeking Synchronicity IMLS Grant • Rutgers & OCLC • Today’s presentation 300 transcripts • (24/7, National & International) • Additional 1000+ transcripts (QuestionPoint & 24/7)
24/7 Transcript Analysis • Generated random sample • July 7, 2004 through June 27, 2005 • 263,673 sessions • 25 transcripts/month = 300 total • 256 usable transcripts • Excluding system tests and technical problems
3 Analyses • Type of Questions • Katz/Kaske Classification • Subject of Questions • Interpersonal Communication • Radford Classification • Manual and NVivo coding
Types of Questions Combined 1st and 2nd Questions n=273 questions
Subjects of Questions Combined 1st and 2nd Questions n=273 questions
Classification Methodology • Qualitative Analysis • Development/refinement of • category scheme • Careful reading/analysis • Identification of patterns • Time intensive, but reveals complexity
Research Questions • Interpersonal Communication Analysis • What relational dimensions are present? • Are there differences in relational dimensions/patterns of chat users & librarians? • If so, what are they? • How do users & librarians compensate for lack of nonverbal cues? • What is the relationship between content & relational dimensions in determining quality?
Results • Interpersonal Communication Analysis • 2 Major Themes • Relational Facilitators • Interpersonal aspects of the chat conversation that have a positive impact on the librarian-client interaction and that enhance communication. • Relational Barriers • Interpersonal aspects of the chat conversation that have a negative impact on the librarian-client interaction and that impede communication.
Manual Coding Results • 200 Transcripts • 177 Usable Transcripts
Librarian Relational Factors (1-200) Librarian Relational Facilitators: Manual Analysis n=177 transcripts
Client Relational Facilitators:Manual Analysis n=177 transcripts
Comparison Relational Facilitators: Manual Analysis n=177 transcripts
Librarian Relational Barriers: Manual Analysis n=177 transcripts
Client Relational Barriers: Manual Analysis n=177 transcripts
Comparison of Relational Barriers:Manual Analysis n=177 transcripts
NVivo Coding • 100 Transcripts • 79 Usable Transcripts
Librarian Relational Facilitators:NVivo Analysis n=79 transcripts
Client Relational Facilitators:NVivo Analysis n=79 transcripts
Comparison Relational Facilitators: NVivo Analysis n=79 transcripts
Librarian Relational Barriers: NVivo Analysis n=79 transcripts
Client Relational Barriers: NVivo Analysis n=79 transcripts
Comparison Relational Barriers: NVivo Analysis n=79 transcripts
NVivo Search Totals n=79 transcripts Includes words in scripts
NVivo Search Totals n=79 transcripts Includes words in scripts
NVivo Search Totals n=79 transcripts Includes words in scripts
NVivo 2.0: Advantages • Advantages • Move subnodes between nodes at any time • Ability to create a hierarchical classification scheme • Proximity and Boolean searching • Group documents and nodes into sets for use in searching and analysis • Programmatically calculate descriptive statistics after coding
NVivo 2.0: Disadvantages • Disadvantages • User interface does not match standard Microsoft application • Cannot double click on a document or node to open • Cannot scroll through a document using the mouse scroll wheel • Cannot import nodes • Cannot set certain tools to always appear • Must manually display the Coder bar and coding strips each time for every document/transcript • NVivo 7 to be released in Feb. 2006 • May alleviate disadvantages
End Notes This is one of the outcomes from the project Seeking Synchronicity: Evaluating Virtual Reference Services from User, Non-User, and Librarian Perspectives. Funded by IMLS, Rutgers University and OCLC, Online Computer Library Center. Project web site: http://www.oclc.org/research/projects/synchronicity/