You Only Think You’re Like Google : Comparative User Experience of Discovery Platforms

1. You Only Think You’re Like Google : Comparative User Experience of Discovery Platforms Rice Majors Faculty Director of Libraries Information Technology University of Colorado, Boulder AMICAL Conference – April 2012 – American University of Sharjah

2. Purpose(s) of study • Create a “common good” set of data • Vendors lack access/resources for consistent & comparative studies • Many libraries lack access/resources to carry out studies (especially comparative) • Working on a campus, I have relatively easy access to study participants • Pursuing this research justified acquiring usability resources we can use for other purposes

3. Study design & methodology • Task based testing with undergraduate students • Usability software (Morae) • Video capture of actions taken • Video/audio capture of “thinking aloud” • Survey instrument • Tested against five discovery tools / next-gens

4. Partner libraries • James Madison University (EBSCO Discovery Service) • University of Colorado Boulder (Encore Synergy) • Vanderbilt University (Primo Central) • Arizona State University (Summon) • Auraria Library, Denver (WorldCat Local)

5. Participants • Undergraduate students only • Library student employees were excluded • 28 participants in total • January 26-March 17, 2011 • Each participant tested one interface only (to avoid “learning”)

6. Participant tasks • Find three books on a particular topic toward writing a paper. • Find three articles on a particular topic toward writing a paper – two articles must be peer-reviewed. • Find recordings by a particular artist. Remind yourself to look at these again later. • The library doesn’t own the book “[title].” Have the library get this book for you.

7. Survey Instrument • 7 Likert items assessing perceptions of ease of use: • I was able to find what I need for these tasks using this discovery platform • If I were doing my own research, I would be able to find what I needed using this discovery platform [etc] • Short answer questions: • What is easy to use about this platform? • What is hard to use about this platform? • What one change would make the biggest improvement to this discovery platform?

8. Perceptions of ease of use

9. Features appreciated – all • Finding different types of resources & narrowing search (12) • Ability to email books & articles (7) • Found relevant articles/results (2) • Ability to save records for later • Smooth interface to library website • Simple interface • Integration with consortial borrowing • Clearly labeling peer-reviewed articles

10. Comments specific to each product • Summary recommendations for each product are based on consensus of study participants only • Where there are fewer recommendations, there was simply less consensus on what issues to address • Note that some vendors have meanwhile addressed some of the issues documented here • Testing was done January 26-March 17, 2011 • Plausible that vendors heard similar feedback from existing and potential customers

11. Product-specific feedback • EBSCO Discovery • Call it something else • Connect users to ILL / resource-sharing more easily • Make it clear[er] how to use the structured search boxes • Address the need to login just to perform searches • Encore Synergy • Simplify / streamline the interface • Add an advanced search option • Primo Central • Make it easier to find options in “additional services” • Connect users to ILL / resource-sharing more easily

12. Product-specific feedback • Summon • Make it easier to find & use email option • Connect users to ILL / resource-sharing more easily • Address misconception of “add results beyond your library collection” • Rely less on link resolver technology • Make it clear[er] when/why options reset • WorldCat Local • Make it easier (in several ways) to use email option • Connect users to ILL / resource-sharing more easily • Make it clear[er] when/why options reset • Improve layout (e.g. so it’s clear[er] when an abstract isn’t available)

13. Feedback for all products • Introduce further enrichment options to allow patrons to evaluate resources (c.f. Amazon) • Make it possible/intuitive to add all results to a list & export all results • Provide context so that it’s clear what has been searched (& what is not included) • Partner (better) with libraries on who does what • Make it easy to get help (“ask a librarian” etc.)

14. Inadvertent findings • Jargon & library practices • Do know what “interlibrary loan” is • Do not know what “government publications” or “electronic resources” are, nor the catalog nickname • Musings about “book” qua “book” • Want FRBR solutions (maybe) • Participant behavior • Will type anything into the search box • Will look for expected features

15. Issues with study design • User defined success • Inescapable testing of information literacy • Students already have multiple frameworks for using library catalog software • Library implementation choices • Library website design choices

16. More information & intentions • “Highlights” video footage to demonstrate findings (available on request) • Article forthcoming in Library Trends • Follow up studies • Involve students from several campuses, with different article database providers and different discovery tools • Use multiple implementations of each interface • More tasks (with randomization of order)

17. Thank you!Rice.majors@colorado.edu Rice Majors Faculty Director of Libraries Information Technology University of Colorado, Boulder AMICAL Conference – April 2012 – American University of Sharjah