Analyzing OPAC Search Logs and Google Analytics for Assessing User Behavior

1. Utilizing OPAC Search Logs and Google Analytics Assessing OPAC Effectiveness and User Search Behavior VALE Users'/NJLA CUS/NJ ACRL Conference January 9, 2009 Rutgers University Jia Mi Cathy Weng

2. Presenters Jia Mi Electronic Resources/Serials Librarian The College of New Jersey Cathy Weng Head of Cataloging The College of New Jersey

3. The College of New Jersey (TCNJ) Located in Ewing, NJ 6000 students Over half a million volumes in the Library’s collection Ex Libris Voyager System in use since 2000 Activated Google Analytics in summer 2008

4. OPAC Search Transaction *Blecic, et al, 1998 OPAC search transaction -- “Consists of a question or query by the user followed by an answer or response from the system” (*) Transaction log analysis – “Detailed and systematic examination of each search command or query by a user and the following database result output by the OPAC.”(*)

5. Sample Search Transaction Log page

6. What is Google Analytics? “Google Analytics (abbreviated GA) is a free service offered by Google that generates detailed statistics about the visitors to a website. ” GA can track visitors from all referrers, usage, entry points, searches, etc. Reports can be viewed, emailed and exported. This free version is limited to 5 million pageviews a month

7. Why Use Google Analytics? Find out… How users get to our site Why they use our site: Which pages are the most popular How they navigate the site How long they stay at the site What are their patterns of behavior Whether the web site effectively supports the institution’s goals and mission regarding provision of access and services

8. Google Analytics Dashboard

9. Data Collection • Voyager transaction log • Two OPAC transaction logs were collected: April 2007 and November 2008 • Comparisons made in many aspects • Library website was redesigned in August 2007; resulted in different search patterns • Google Analytics • November 2008 data collected • Data from September to November 2008 also examined and compared

10. Voyager Transaction Log • Can assess • Overall statistics of search types used • Query traffic, both on- and off-campus • How searches were issued and refined • Query trends • Cannot assess • Queries successful or not • Queries made by librarians or other users

11. When Analyzing the Log • Many unanswered questions • Many queries do not have session IDs • How “Simple Search” was formed? • Many queries do not have “search type” • Some educated guesses • Queries from redirect (i.e., user clicked hyperlinked field) • Comprehensive analysis not possible • Focus on a few areas only

12. Study Focus • Comparison • Query statistics • Default searches • Usage of Advanced Search • Usage of Subject search • Other findings related to user search skills and OPAC effectiveness

13. Query Statistics

14. Title Phrase vs. Keyword Search

15. Subject Searches Browse search: log does not show result hits Heading (phrase) search: difficult to use as users rarely know the established terms Search logs revealed two types: user-issued searches (Subject Heading Browse) and record redirect (Subject Browse)

16. Bibliographic Record Display

17. Subject Browse Screen

18. Subject Queries

19. Advanced Search (Guided Search) April 2007: 3,331 (6% of total searches) searches issued. (3057 <gui>+ 274 <bui>) Nov. 2008: 1,761 (2% of total searches) searches issued. (1253 <adv> + 508 <bui>) Data suggest that library users use Advanced Search interface less now than they did 18 months ago

20. Advanced Searches • Statistics do not include searches issued by local host (127.0.0.1) which has quite a few searches using Advanced Search interface.

21. Call Number Browse • By clicking the call number hyperlinked field, user will receive a call number browse screen of nearby items • Approximately 500 queries in Nov. 2008 and 600 queries in April 2007 using call number browse hyperlink redirect • It is assumed that users clicked call number hoping to get information about item shelf location in the Library (not nearby items)

22. Other Findings • Related to system retrieving • When doing title phrase search using Advanced Search interface, it only searches main title, not subtitle. • E.g., “Rich and the super rich, a study in the power of money today”. • Main title and subtitle. • User would not know the difference. • System retrieval mechanism issue.

23. Other Findings • Related to OPAC Display • When doing keyword search using Basic Search interface, search results are arranged by relevance. • When doing keyword search using Advanced Search interface, search results are arranged by publication date. • More relevant results might not be in the first results display page.

26. Next page

27. Other Findings • Keyword in Basic vs. Advanced Search interface • In Basic Search words like “the,” and “and” are acceptable • In Advanced Search, typing in “the” or “and” will retrieve 0 hits • System retrieval mechanism issue

30. Other Findings In Basic Search, using “Author (last, first)” results in a browse search and the x-ref in authority record works. In Advanced search using author search key results in a heading search (instead of browse search) and the x-ref in authority record does not take effect.

31. Author Query in Basic Search

32. See Reference OPAC Display

33. Author Query in Advanced Search

34. In Advanced Search, need to use established form of name for records to be retrieved.

35. Other Findings • Advanced Search interface • Using “as a phrase” as default might have better chance retrieving 0 records. • Should the default be changed to “All of these”? Advanced Search Interface

36. What Can Be Done Encourage use of author browse in Basic Search interface to take advantage of x-ref in authority file. Provide information on search help page. Investigate making author browse possible in Advance Search interface.

37. What Can Be Done • Look into system indexing structure more closely and their connection to OPAC searches • E.g., Title search in Advanced Search interface to change to TALL (whole title), not just 245A (main title) • E.g. , Enhance Keyword search in Advanced Search to accept “stop words” (the, and, etc.)

38. What Can Be Done Investigate the possibility of enhancing query results display from Advanced keyword search to be ranked by relevance.

39. What Can Be Done • Enhance local search skills. OPAC transaction log OPAC search results

40. Google Analytics Dashboard: Comparison

41. Visitors Overview

42. Geographic Region: International

43. Geographic Region: United States

44. Geographic Region: New Jersey

45. Site Visiting Time Nov. Sept. Oct.

46. Site Usage Trend

47. Where Did Users Come From?

48. Direct Visits to TCNJ Library Catalog

49. What Content are Users Looking at?

50. /cgi-bin/Pwebrecon.cgi?DB=local &PAGE=First Pageview: 2816 This page was viewed 19 times via 1 keywords