Observation and Usability Studies

1. Observation and Usability Studies Ericka Brunson Dorothy Hirsch Mollee March Joanna McCloud Denise Tiller

2. Focus on human behavior & actions • Part of mixed-methods approach • Naturalistic: how people really behave What they have in common

3. Test validity of other findings • Study specific processes or behaviors • Access behaviors of those unable to communicate Observation studies can

4. Three types of studies • Participant Observation • Reactive Observation • Unobtrusive Observation

5. Study reference transactions • Analyze patron/librarian behavior • Study patron use of resources and tools Library uses

6. Participant observation is • a thorough study over time • conducted in a natural environment • when the observer becomes part of the group being studied • rarely used by libraries because of expense in time and money • the basis of most cultural anthropological/ethnology group studies

7. Limitations • Only studies behaviors, not cognition • Prone to research bias/misconceptions • Can be expensive, both in time and money • Ethical considerations

8. Ethical issues • How information is obtained • What is done with the information • Do no harm: emotionally mentally socially economically

9. Reactive observation Are you watching me right now…or am I watching you?

10. A “real world” observational research method that studies individuals and groups of people in a natural, controlled setting • Participants are aware that they are being studied. What is reactive observation?

11. Behavior reflects aspects of social desirability • Observations can be manipulated to fit the research design Reactive observation positives

12. Individuals are reacting to observation • External validity Reactive observation negatives

13. Continuous monitoring • Time allocation Common methods of reactive observation

14. Continuous monitoring • Involves studying the behavior of participants and recording as much as possible. • Works best in organizational settings such as: the workplace educational institutes when monitoring nutrition (exercise, food intake, etc.)

15. The “Hawthorne Effect” • The “Hawthorne Effect” states that “workers react to the attention they are getting from the researchers and in turn their productivity increases.”

16. Done at random • Researcher will choose the time and place to record activities before the researcher’s presence is known Time allocation

17. Variables to consider in evaluating observations • Descriptive • Inferential • Evaluative

18. Subjects are unaware that they are being observed and researchers do not intrude on the study. Unobtrusive measures

19. Behavior trace studies • Disguised field observations Types of measures

20. Natural behavior of subjects • Reduces bias Unobtrusive positives

21. Unobtrusive negatives • Researcher has little control over type of data collected • Not always a usable method • Subjects unaware of data being gathered • Other ethical issues

22. Tests system performance with typical users • Many library applications • Quick, easy, inexpensive Usability studies - a bright idea

23. Four types of usability studies • Exploratory – early stage using screen shots and paper visuals • Assessment – mid-point focusing on the actual product • Verification – final stage to ensure product meets standards • Comparison – used in any stage to compare design options

24. Usefulness • Ease of learning • Ease of use • User satisfaction Usability studies test

25. Methods used to collect data • Card sort • Prototypes • Usability tasks • Focus groups • Individual interviews

26. Examples of library usability studies Johnson County Public Libraries JoCoKids Usability Study April 25, 2009 • Type of study Exploratory • Purpose Test JoCo Library Children’s Web site • Methods used Prototypes, usability tasks, focus group • Participants Eleven children ages 6 to 11

28. Johnson County Public Libraries 6 by 6 Usability Study September 1-2, 2009 • Type of study Assessment • Purpose Test new web site • Methods used Prototypes, usability tasks, focus group • Participants Five mothers with children under age 6

29. Step 1 Let participants explore the prototype web site and record comments • Step 2 Usability tasks 6 by 6 usability study

31. Johnson County Public Libraries • New Paper Application Card Usability Study • June 16, 2009 • Type of study Verification • Purpose Verify usability of new paper application • Methods used Usability task • Participants 10 patrons

32. Anschutz Library Renovation Project 2009-2010 • Dec Emailed survey to faculty, staff and students • Mar-Sept Redesigned the main library floor and created the Learning Studio • Oct “Snapshot Day”

34. References Beck, S.E. & Manuel,K. (2008). Observation and usability. In Practical research methods for librarians and information professionals. New York, N.Y.: Neal-Schuman Publishers, Inc. Brown, L. & Trochim, W.M. (2006). Research methods knowledge base. Retrieved from Cornell University website: http://researchmethods.net. Brown, L. (n.d.). Observational ield Research [Online tutorial]. Retrieved from Cornell University Library website: http://www.socialresearchmethods.net/tutorial/Brown/lauratp.htm. Hendry, J. (1999). Other people's worlds: An introduction to cultural and social anthroplogy. New York, New York: Washington Square University Press. Kumar, K. (1996). Using direct observation techniques [ CDIE Tips Series,4]. Retrieved from USAID Evaluation Publications website: http://pdf.usaid.gov/pdf_docs/PNABY208.pdf. Lee, R.M. (2000). Introduction to unobtrusive methods. In Unobtrusive methods in social research, pp. 1-16. Norlin, E. (2000). Reference evaluation: a three strep approach – surveys, unobtrusive observations, and focus groups. College and Research Libraries, 61(6),pp. 546-553. Paul, B.D. (1953). Interview technigues and field relationships. In Kroeber, A.L. (1953). Anthropology Today: an encyclopedic inventory. Chicago, Illinois: University of Chicago Press. Radin, P. (1966). The method and theory of ethnology: An essay in criticism[2nd ed.] New york and London, England: Basic Books. Reece, R.D. & Siegal, H. A. (1986). Studying people: A primer in the ethics of social research. Macon, Georgia: Mercer University Press. Romanczyk, R.G. ,Kent, R.N., Diament, C., & O'Leary, D. (1973). Measuring the reliability of observational data: a reactive process. Journal of Applied Behavior Analysis, 6(1), 175-184. Trochim, W.M. The research methods knowledge base, [2nd ed.]. Retrieved from: http://www.socialresearchmethods.net/kb/. Last revised: 10/20/2010. Usability studies at Johnson County Libraries (n.d.) Retrieved from: http://jocolibrary.org/templates/JCL_InfoPage.aspx?id=3063&epslanguages=EN.