Assessing Library Web Portals: Usability and Beyond

1. Assessing Library Web Portals: Usability and Beyond Yu-Hui Chen University at Albany, State University of New York ENY/ACRL 2012 Conference Mohawk Valley Community College, Utica, New York May 21, 2012

2. Web Site Evaluation Methods • Evaluation with user participation • Think aloud • Card sorting • Prototyping (paper/online) • Eye tracking • Focus group discussions • Field study • Log analysis • Web Survey • Evaluation without user participation • Cognitive walkthrough • Heuristic evaluation

3. Think Aloud • Users are asked to complete specific tasks • As users are attempting to complete each task, they verbally report their thoughts and feelings of their actions • Observers watch, listen, and takes notes

4. Card Sorting • Open card sorting • Give users labels representing the content of the Web site • Users review these labels and then group them into categories. • Users assign category names to these groups • Closed card sorting • Provide category names for users • Users sort the labels into categories

5. Prototyping (Paper/Online) • Provide users with descriptions and purposes of an intended Web site • Have users brainstorm the design • Have users draw the design • Test the design

6. Eye Tracking • Setting up a lab • Training users in using the equipment • Giving users tasks • Review the reports

7. Cognitive Walkthrough • The evaluators design specific task scenarios • The user’s goals and purpose for each task are defined and tasks are broken down into relatively small pieces • The evaluators role play the part of the user working with the site, noting problems, path, and barriers, essentially reviewing the ease of learning the site

8. Heuristic Evaluation • Have a small group of evaluators (2-5) review the site using usability heuristics (e.g., Nielsen), standards (e.g., ISO), or guidelines (e.g., US Dept. of Health and Human Resources) • Inspect the task flow • Inspect details of individual elements • Evaluators review the site independently • Reconvene and discuss findings

9. Information Systems Success Model DeLone, W. H., & McLean, E. R. (2003). The DeLone and McLean model of information systems success: A ten-year update. Journal of Management Information Systems, 19(4), 9-30. (p. 24)

10. Measures of Information Quality • Accuracy • Currency • Sufficiency • Reliability • Relevance • Format options

11. Measures of System Quality • Accessibility • Ease of use • Flexibility • Response time • Reliability

12. Measures of Service Quality • Assurance • Empathy • Responsiveness • Reliability

13. Measures of Use • Frequency of use • Extent of use • Motivation to use

14. Measures of User Satisfaction • System quality satisfaction • Information quality satisfaction • Service quality satisfaction • Overall satisfaction

15. Measures of Net Benefits • User productivity • User performance

16. Assessment Approach • Quantitative • Qualitative

17. Bibliography • DeLone, W. H., & McLean, E. R. (2003). The DeLone and McLean model of information systems success: A ten-year update.Journal of Management Information Systems, 19(4), 9-30. • International Standards Organization (1994). Ergonomic requirements for office work with visual display terminals. Part 11: Guidance on usability (ISO DIS 9241-11). London: International Standards Organization. • Nielsen, J. (1993). Usability engineering. Boston, MA: Academic Press. • Popp, M. P. (2001). Testing library Web sites: ARL libraries weigh in. Proceedings of the ACRL Tenth National Conference, 277-281. • United States Department of Health and Human Services. (2006). Research-based Web design & usability guidelines. Washington, DC : U.S. Government Printing Office. Other entertaining resources: • Chen, Y., Germain, C. A. , & Yang, H. (2009). An exploration into the practices of library Web usability in ARL academic libraries. Journal of the American Society for Information Science and Technology, 60(5), 953-968.