Reconexp: Improving ESM

1. Reconexp: Improving ESM Vassilis-Javed Khan, Panos Markopoulos, Berry Eggen, Boris de Ruyter, Wijnand IJsselsteijn Mobile HCI, Amsterdam, 04 september 2008

2. INDEX • What is Experience Sampling Method? • Our Application and Study • How does it improve ESM?

3. Experience Sampling Method • A quasi-naturalistic method that involves signaling questions at subjects at random times throughout the day.

4. ESM is gaining popularity in HCI • Hudson et al. (2002) have used the ESM to explore attitudes about availability of managers at IBM Research • Consolvo and Walker (2003) have used the ESM for evaluating an Intel Research system called Personal Server • Froehlich et al. (2007) used ESM to investigate the relationship between explicit place ratings and implicit aspects of travel such as visit frequency

5. HOWEVER… • Reported loss of data: at least 20%

6. HOWEVER… • Reported loss of data: at least 20% • How significant is the data lost?

7. GOAL OF OUR STUDY • Find out what information are “busy parents” willing to automatically share among themselves + under which context

8. 1. Insert info to personalize the ESM OVERVIEW 2. Execute ESM for a week 3. Foreach day review answers & fill out the missing points

9. OVERVIEW OF THE STUDY • INITIAL TASK (WEBSITE): • Name Places (during a typical working day)

10. Name Places

11. OVERVIEW OF THE STUDY • INITIAL TASK (WEBSITE): • Name Places & activities (during a typical working day)

12. Link Activities to places

13. OVERVIEW OF THE STUDY • INITIAL TASK (WEBSITE): • Name Places & activities (during a typical working day) • Link information to context (place & activity)

14. Link info to context

15. Link info to context – name other info

16. OVERVIEW OF THE STUDY • INITIAL TASK (WEBSITE): • Name Places & activities (during a typical working day) • Link information to context (place & activity) • A TYPICAL WORKING WEEK (WEBSITE & PDA): • Ask several times about context (at that time) & info willing to automatically share,

17. MOBILE DEVICE

18. REVIEW & UPDATE ON WEBSITE

19. REVIEW & UPDATE ON WEBSITE

20. REVIEW & UPDATE ON WEBSITE

21. OVERVIEW OF THE STUDY • INITIAL TASK (WEBSITE): • Places and activities (during a typical working day) • Link information to context (place & activity) • A TYPICAL WORKING WEEK (WEBSITE & PDA): • Ask several times about context (at that time) & info willing to automatically share, • SEMI-STRUCUTRED INTERVIEW.

22. 11 PARTICIPANTS • Mean Age: 38(max: 44, min: 28, σ = 5.72) • Mean Number of children: 1.91(max: 4, min: 1, σ = 0.79) • Mean Age of children: 5.47(max: 8.5, min: 0.7, σ = 2.57) • Mean Years of marriage: 10.86(max: 20, min: 2, σ = 5.22) • Mean Hours of work per week: 28.18(max: 40, min: 20, σ = 6.63)

23. LOG • Log into the system • Link information statements to previously not answered question • Name Activity which was not answered • Name Activity which was not answered using existing value • Name Location which was not answered • Name Location which was not answered using existing value • Name “Other” Activity • Name “Other” Activity using existing value • Name “Other” Information • Name “Other” Information using existing value • Name “Other” Location • Name “Other” Location using existing value

24. RESULTS • Mean number of actions performed (logins not counted in this number) 55.64 • Mean logins (in 5 days) 2.91 • Mean percentage of activities (2nd question) not answered: 48.81% • Mean percentage of activities recovered (with the website use): 60.13% • Overall improvement of the website to the method is 29.35%

25. CONCLUSIONS • Improves on the data loss of ESM • Participants & Researcher can have an overview of the data during the execution of the study • Answers are personalized • Answers can be annotated at a later point in time • V.j.khan@tue.nl • HTTP://WWW.AWARENESS.ID.TUE.NL