Diary studies

1. CSCI 4163/6904, Summer 2011 Diary studies

2. Diary studies… • Participants collect data about events • As they happen • In the context of the event (in situ) • Can think of like a small, longitudinal questionnaire • Often used to prompt interview discussion (similar to observations in a contextual inquiry) • Can help understand rare/infrequent events

3. Data collection methods • Survey style forms • Paper (little training required, but hard to monitor) • Online entry forms • Google docs • Photographs, video • Digital/disposable camera, mobile phone • Audio recording • Voice recorder, mobile phone • Aggregating data online • Blogs (text, audio, video) , Twitter • Rich and timely information

4. Type of data recorded • When • Date/time • Duration • Activity/task • What • Activity/task • feelings/mood • Context (environment/setting)

5. When is data recorded? • Randomly • In response to prompts • At specific intervals • Based on activity

6. General types of diary studies • Unstructured • Ask participants to report on everyday activities • Trying to elicit general themes • Structured • Ask participants to report on everyday activities by answering specific questions about the activity • Combination of question types • Can also be used as a form of usability tests and problem reports • Ask them to complete a task and report results, identify bugs, etc.

7. Variations based on purpose Feedback (now) Elicitation (later) • Record everything in situ • No follow up • Focus on the “what” data • Mostly structured • Can be burdensome on participants and researchers • Record aspects or triggers of an activity • Follow up with an interview to gain more information • Data captured is used to elicit the “why” and the how” • Mostly unstructured • Can be problems with participant recall

8. Cultural Probe • Diary study++ • Used to not only record activities, but capture more of “felt life” • Stimulate thought as well as capture experiences • http://www.hcibook.com/e3/casestudy/cultural-probes/ - pack contained a small disposable camera and a listening glass that participants used to listen at walls and doors and write what they heard. It also contained a small solid state recorder packaged in a sleeve that said 'dream recorder'. • Was used by designers at the Royal College of Art, London, to study the way people see their own homes. The results were used to enable designers to get a 'feel' of the meaning of home for many people. • Experience sampling method (measure feelings, moods) • Technology Probes (prototypes, experimental artifacts)

9. Coordination Challenges • Keeping participants on track • Periodic reminders • Feedback about the level of detail in responses • Progressive incentives, surprise incentives/gifts (may or may not be allowable by BREB) • Adapting to changes • Start analysis as soon as first results arrive • May need to re-evaluate the diary format if the data being captured is not what you expected!

10. Advantages • Time efficient for researchers • Lower cost than direct observations • Can have a broad geographic distribution • Supports contextual reports over time • Natural environment • Can give rich data about contexts of use

11. Disadvantages • Participant fatigue • Missed responses (frequency data is generally unreliable – lower bound) • Study drop outs • “after the fact” reporting to catch up • Behaviour adjustment • Participants need reminders • Can be expensive for long durations • Volume of collected data can be overwhelming

12. Study protocol • Introductory letter • Consent form • Needs to motivate their participation • Very specific instructions • Can be helpful to give an example of a filled out form (take care not to prime/limit) • Diary form / collection aids

13. Forms of analysis • Track temporal patterns • Look for semantic patterns in visual data • Combine with interview analysis