Hvad er Remote Usability Testing?

1. Hvad er Remote Usability Testing? Jan Stage Professor, PhD Forskningsleder i Informationssystemer (IS)/Human-Computer Interaction (HCI) Aalborg Universitet, Institut for Datalogi, HCI-Lab jans@cs.aau.dk

2. Oversigt • Forhindringer i praksis • Ideen om remote usability testing • Realisering af ideen

3. Oversigt • Forhindringer i praksis • Ideen om remote usability testing • Realisering af ideen

4. To undersøgelser af forhindringer i praksis • Primært Nordjylland • Software-virksom-heder, typisk svar fra mellemledere • Deltagere i en-dages konference om usability • Tallene angiver procent af svarene i hver undersøgelse Første undersøgelse Anden undersøgelse

5. Oversigt • Forhindringer i praksis • Ideen om remote usability testing • Definition og baggrund • Tidligere arbejde • Synkrone og asynkrone metoder • Realisering af ideen

6. Definition and Motivation • Remote usability testing is a situation where“the evaluators are separated in space and/or time from users”(Castillo, Hartson and Hix, 1998) • Well-known challenges • Building and setting up usability labs • Getting users to the lab and making them feel comfortable • New challenges • Users, developers and evaluators are distributed geographically, sometimes even in different countries and time zones • Outsourcing • Open Source Software development • Global software development • Remote usability evaluation is becoming increasingly relevant

7. Earlier Work (1) • The idea of remote evaluation emerged between 1994 and 1998 • Literature survey on remote usability testing • 21 references • Early: presents a variety of methods • Recently: practical advice and experiences combined with pros/cons

8. Earlier Work (2)

9. Earlier Work (3) • Two types of methods: synchronous / asynchronous (various forms) • The majority of references focus on synchronous usability evaluation:Simulating a conventional laboratory-based think-aloud test • Overall: • Few systematic comparisons of remote methods • Mostly comparisons of a single remote method and the conventional lab-based method

10. Synchronous Methods • Laboratory testing (LAB) • Conventional method • Benchmark for comparison • Remote synchronous testing (RS) • Spatial separation: User and test monitor in separate rooms • Remote sharing of desktop • Webcam connection with picture in the bottom corner of the screen • Audio connection • Data: Video recording of the test monitor’s screen and the audio

11. Asynchronous Methods • Remote asynchronous user testing (AU) • Based on the idea of users reporting the critical incidents they experience(Castillo, Hartson and Hix, 1998; Hartson and Castillo, 1998; Hartson, Castillo, Kelso and Neale, 1996) • The users solve the tasks with the system • Simultaneously fill in a web-based form each time they encounter a usability problem • The form briefly explains what a usability problem is and how it can be categorized • In the form, they describe each problem and categorize it’s severity • Spatial and temporal separation • Remote asynchronous expert testing (AE) • Same method as AU but with users that have expertise in usability engineering (graduate students in HCI) • Data: A set of described usability problems and a start and end time for each user

12. Survey of Remote Asynchronous Methods

13. Oversigt • Forhindringer i praksis • Ideen om remote usability testing • Realisering af ideen

14. Realisering af ideen • Forhindringer og udfordringer • Tekniske: Kan man få det til at fungere • Metodiske: giver det brugbare resultater • Praktiske: kan man gøre det i praksis • Det skal vi arbejde med resten af dagen …

15. aa