Measuring the Effectiveness of Facets in the Next Gen OPAC 2008 Library Assessment Conference

1. Measuring the Effectiveness of Facets in the Next Gen OPAC2008 Library Assessment Conference Kathleen Bauer and Alice Peterson-Hart Yale University Library Usability and Assessment

2. Usability process • Assess current behavior • Conduct usability protocol testing • Share findings with staff • Implement changes and assess effect

3. Assess current behavior Collecting evidence from Yale’s Voyager-based catalog, called Orbis, using log file data.

4. Log file line 12/2/07 20:06 20071202200646 Title Index TALL JOHN DOGGET JNR 0

5. Log file sample descriptive data 5 Most common search: title (41.8%) followed by keyword (31.4%) Average search phrase is 2.5 words long Most common phrase searched is united states Most common number of hits: 0 (21.4% )

6. Examples of title searches • BOOKS ON PHOTOGRAPHY? • ALABAMA 2004? • PATHOPHYSIOLOGY CHILDREN? • JOHNSON, JAMES? • INTRODUCTION TO AFRICAN AMERICAN STUDIES: A READER? Topic Name Close

7. Why do title searches get 0 hits?

8. Decreasing search types 2007-2008

9. Some conclusions from log analysis • Patrons commonly use field specific searches unsuccessfully • Majority of searches don’t use catalog metadata beyond title (decreasing) • Errors happen a variety of ways and current Voyager system is not helpful in avoiding those errors

10. Next Gen OPAC -- Vufind

11. Conduct usability tests After deciding to implement Vufind, use data from log files to help design usability tests

12. Creating usability tests for Yufind • Two opportunities for tests: • medical implementation of subject-based ebooks display • alternative interface for catalog (aimed at undergraduates) • Objective: Would the participant see and choose facets? (more than 26% of the time) • Objective: Would the participant use facets successfully?

13. Questions • Find books within a set by a particular author (John Adams). • In a set of books about healthcare, find some that are about healthcare reform. • Within a subject-based display of ebooks find a topic-based subset.

15. Facets: key findings • Were seen and sometimes used but • not always successfully for author and topic • Subsets sometimes didn’t make sense • Hard to navigate (not alphabetical) • Little use of call number facet • Some preferred search refinement to facets • Participants noted potential usefulness: format, language, and topic facets

16. Did facets increase metadata use? YES

17. Share findings with staff Staff need to engage with findings of usability testing and current behavior in Orbis to decide best practices and to build empathy for user experience.

18. Relevance ranking and facets

19. Review keyword searches with subject specialist librarians • Times square red times square blue • hague conference on peace hague conference on peace 1907 second hague conference on peace 2nd hague conference on peace • cuban revolution cuban revolution of 1959 • … (list of 50 to start)

20. Set best practices for facet display • Display facets on left • Develop consistent terminology between different applications • Consistent order of display

21. Implement changes and assess

22. Implement changes and assess • Move facets to the left side (standard) • Demoted (in list order) topic facet • Add a sort option—will that increase successful use of facets? • Can we promote use of call number facets?

23. Usability process • Assess of current behavior • Conduct usability protocol tests • Share findings with staff • Implement changes and assess effect

24. Slides and findings available http://www.library.yale.edu/libepub/usability/yulstudies.html/arl.ppt Yale University Library Usability and Assessment Department Kathleen Bauer Alice Peterson-Hart Send questions to kathleen.bauer@yale.edu