VuFind@Illinois: Usability Testing & Results

VuFind@Illinois:Usability Testing & Results

Usability & Feedback: An Overview Asked for Feedback Survey Formal Usability testing

Feedback

Feedback: Comments Love it love it love it! MUCH more user friendly. I don’t understand what “barcode” is for requesting items. How do you set up an account? Can it be used to search I-Share? Where is call number searching? I like the facets, but want more facets Is reserves information in VuFind?

Feedback Needed more data Began to think about formal usability testing Then, Yale did a survey…

Survey: Results

Survey: Results 75% Liked VuFind 86% Found what they were looking for 73% Thought the search results were what they expected 54% Used the facets 63% Thought facets were helpful

Survey: Facets “I didn’t even really pay attention to them. They were not generally helpful in the kind of search I was doing.” “I expected to find articles when clicking journals/magazines, like WorldCat.” “…because VuFind has the look and feel of Amazon, I only focused on the book listings, choosing instead to ignore the screen areas usually laden with advertising.”

Survey: Facets “They really helped pare down the tidal wave of results I initially got.” “…they were very helpful in making my search more specific since there aren’t a lot of default search options.” “I think it helped by narrowing down the results, sort of like Google does these days.”

Survey: Facets “The results were not what I expected, but it was a pleasant surprise. I found things in my research that I didn’t know we had.” “…the most useful part of the groupings for me was helping me sort of mentally focus what I was looking for.”

Survey: Results In the future, if you need to find books, the first place you’d start is:

Survey: Results I think that I would like to use VuFind frequently:

Survey: Results I found VuFind unnecessarily complex:

Survey: Results I thought the system was easy to use:

Survey: Results I felt very confident using VuFind:

Survey: Comments - Pluses • Not very overwhelming & user friendly • Easy to navigate and narrow results • Liked cover images • Interface more user friendly than current catalog • Similar interface to area public libraries • Not as intimidating as WebVoyage

Survey: Comments - Improvements • Did think results were well sorted and not clear how it does sort the results • Facet and “facets now applied” are confusing terms • Text formatting and displayed information is sometimes confusing • Could use some more polish • Too much scrolling – very vertical

Survey: Comments - Improvements • Users want to use it to search for articles • Want even more information about an item • Not sure of relevancy of user added content • Some facets are confusing • No ability to sort by a particular campus library

Survey: Comments – Feature Requests • Direct export to RefWorks • Keyword search option • Call number searches • Social networking integration • Have a “Repeat Search in I-Share” link in record • Nice to customize display to show what you wanted on results page • Ability to collapse facets

Survey: Comments – I-Share • Perception interface with I-Share needs to be clarified and streamlined • Not clear how to conduct search in I-Share • Liked not inputting library card number multiple times requesting items • Confusion over why you need to create an account

Survey: Comments “Please replace the current system with VuFind right now, I really don’t see a reason not to. It is a huge improvement” “VuFind is going to make the old online catalog as awkward as using the card catalog is now” “VuFind strikes me as a dumbed down version of the Google search page designed to appeal to freshmen who are used to doing Google searches rather than a serious catalog geared towards scholars conducting serious research.”

Usability Testing - Setup • In April & May, 2009, conducted 15 hands on usability tests with Morae • 5 faculty, 5 graduate students, 5 undergrads • Very diverse group • Used task based test adapted from statewide WorldCatLocal usability testing

Usability Testing: Instrument • Asked general interface/layout questions and if search and features are clear • Searches • Broad subject search for “climate change” • Search for James Joyce • Prince audio recording • “Gone with the Wind” DVD • “Jack London” and “White Fang” • Find a public speaking DVD

Usability Testing: Results Lack of keyword searching

Usability Testing: Results Language Selection

Usability Testing: Results Red Light/Green Light for holdings—need yellow light for reference or reserve items (need picture)

Usability Testing: Results Direct Export to RefWorks:

Usability Testing: Results • Want URLs to stand out more:

Usability Testing: Results Ability to link out to I-Share VuFindExample:

Usability Testing: Results Favorites feature:

Usability Testing: Results Facets:

Usability Testing: Results

Usability Testing: Results Format

Usability Testing: Results Topic/Subject Area

Usability Testing: Results

Usability Testing: Results Staff view

Usability Testing: Results Tagging

Usability Testing: Results • Interface improvements: • Could use more polish • Too vertical • Ability to customize number of results on a page • Change order of facets in sidebar • Make electronic resources/ebooks a clearer limit in the advanced search and facets • Searching for multimedia was cumbersome

Usability Testing: Results • Really liked how visual the interface was—less text, more pictures • Users care less about “librarian features” such as MARC and call number searching • Users aren’t familiar with next generation catalogs or other library tools, but use online bookstores

Usability Testing: Results • All but one subject thought VuFind was best for most of their catalog use • Only 1 user preferred layout of Amazon more, citing the star ratings • All will use VuFind again • Everyone thought it was a dramatic improvement to WebVoyage

Summary VuFind isn’t perfect, but is a huge improvement over WebVoyage.

VuFind@Illinois: Usability Testing & Results