Challenges in usage research and User Centered Design

1. Challenges in usage research and User Centered Design Joke Kort, TNO Information and Communication Technology

2. Outline • Some background information: • Context of Personal Network Pilot 2008 (PNP) • PNP first year field research • Challenges in field research • Possible solutions, logging & sensing • Discussion

3. Personal Network Context of PNP2008 • PNP2008*, a 4 year project, is part of the research program Freeband Communication • What is a Personal Network? home network personal area network (PAN) company network car network *http://www.freeband.nl/

4. Context of PNP2008 • Work package “Users”: • Insight in markets and target groups • Business market as well as consumer market • Yearly cycles: develop Personal Network service concepts (health care professionals) • Needs for PN services • Context awareness • Interaction design • Research methodologies, methods and tools to support this service development

5. PNP2008 first year field research • Target group characteristics: • Health care professionals • Lots of communication and information exchange • (Potential for) high ICT usage • Work spread out over multiple locations • Task/work characteristics: ad-hoc, independent, complex and diverse

6. Operation planning PNP2008 first year field research • What needs and desires do users have concerning PN services/functionality (observations, contextual inquiries and workshop) Patient information Monitoring brain activity Video/audio feed to other room Monitoring vital functions

7. Event observation Event Event Event Event Time Challenges in field research Largely unknown interests Observations Probe & diary studies An overwhelming amount of information that needs structuring and some form of processing (time consuming, unstructured and sometimes biased results) Due to limited resources and the time consuming nature of observations and contextual inquiries these methods are often used over a limited period of time and with small sample sizes (real world validity and a lack of quantitative insight in events) Contextual inquiries Workshops Interviews Questionnaires Known interests Use cases & scenarios Task analysis Etc.

8. Event observation Event Event Event Event Time Event selection Event Event Event Event Time Challenges in field research Largely unknown interests Observations Probe & diary studies Contextual inquiries Workshops Interviews How representative is our view on the world of a health care professional (real world validity)? How can we select events that indicate real problems (priority of events)? Questionnaires Known interests Use cases & scenarios Task analysis Etc.

9. Event observation Event Event Event Event Time Event selection Event Event Event Event Time Event exploration Event Event Event Event Time Challenges in field research Largely unknown interests Observations Probe & diary studies Contextual inquiries Workshops Interviews • preference • behaviour • experiences • problem or annoyance • context variable • means and tools used • processes and procedures • etc. Questionnaires Known interests Use cases & scenarios Task analysis Etc.

10. Event observation Event Event Event Event Time Event selection Event Event Event Event Time Event exploration Event Event Event Event Time Challenges in field research Largely unknown interests Observations Probe & diary studies Contextual inquiries Workshops Interviews Questionnaires Known interests Use cases & scenarios How can we further explore specific events and gain detailed insight in e.g. user preferences, experiences, context variables related to the events, etc. Task analysis Etc.

11. Possible solutions, logging and sensing Solutions should address the following research challenges in early phases of design: • Resource intensive • Unstructured, subjective results • Real world validity (small sample sizes, lack of insight in the quantitative nature of events, difficult to assign priorities to events) • Gaining insight in event details And support research in the following aspects, based on the characteristics of a PN: • Searching, sharing, accessing and creating information • Communication (mediated & F2F) and (social) context • Multiple locations • ICT usage, different devices • Complex and divers tasks and work structures

12. Possible solutions, logging and sensing • Logging: automatic data capturing without active involvement of the researcher or the participant during this capturing (e.g. screen capturing, video recordings, server logs, localization, etc). • Sensing: Data capturing without active involvement of the researcher but with active involvement of the participant (e.g. experience sampling). • Experience Sampling study with Caes* on iPaq’s performed within Freeband User Experience (FRUX)** • Not really resource intensive • Structured results • Quantitative insights • Larger sample sizes • Gain more insight in in specific events PC iPaq *http://web.media.mit.edu/~intille/caes/ **http://www.freeband.nl/

13. Possible solutions, logging and sensing • We would like to apply Experience sampling in earlier phases of PN service design as well: • Support for different devices and media • Combining different methods • Ongoing, continuous access to data • Easily applicable in other research Other methods Other methods Triggered Random/set time intervals Event Sample Immediate follow up study Request * User’s observation Event Moblogging** * http://www.noldus.com/site/doc200402086, spectacles camera ** http://www.newsfacts.net/tno/, Moblogging

14. Thank you • Other information: • PNP2008 and FRUX: http://www.freeband.nl/ • SocioXensor: http://socioxensor.freeband.nl • Contact: J. Kort & A. de Vries • Thanks to Arnout de Vries and Marjan de Jonge • Do you have a question? • Discussion • Ideas about methods for research during early phases of design (event observation, selection, exploration)?