Evaluation in the FEEL Project - A Pilot Study

1. Evaluation in the FEEL Project - A Pilot Study Peter Lönnqvist and Hillevi Sundholm The Future Ubiquitous Service Environments Research Group,Stockholm University and KTH Center for Wireless Systems, Kista, Sweden

2. Presentation outline Overview of the study Experiment movies Conclusions/Future Work YOUR input!

3. Overview of the Study Procedure/Case Study iLounge Experiment goals/the FEEL project Experiment procedure

4. iLounge The iLounge in Kista is funded by the Wallenberg foundation Ready during spring 2002 Ubiquitous Service Environment Platform for Development Study Environment Showroom

6. Experiment goals/the FEEL project Non-intrusive services to support Focussed, Efficient and Enjoyable Local activities. The main objective of the project is to deal with the problem of the intrusiveness of today's mobile technology and how work in local environments can be enhanced by introducing the idea of non-intrusive services realized partly by disappearing computer environments. The work in the project will be based on the detailed analysis of a number of concrete scenarios where today technologies for mobile and distributed work obviously create intrusions on the local activities going on.

7. Experiment goals/the FEEL project cont´d • Based on the analyses of the concrete scenarios we will: • design appropriate disappearing computer environments • design mechanisms by which multiple services simultaneously can share and effectively use the disappearing computer environment • for the last purpose to build on the software agent paradigm for service design • establish a software platform which handles software components on small artefacts transparently and manages open sets of services • making usability studies for the disappearing computer environments

8. Experiment procedure Ravens Matrices

9. Experiment procedure cont´d Different modalities for (routed) information on different displays Banner iClock Sound – No sound

10. Experiment movies Instructing the participants * Participants working with the task * Intrusions *

11. Method and Results Method Qualitative, experimental, explorative PILOT study Results Video material remaining to be analyzed… Task and intrusion experiences evaluated

12. Task and intrusion experiences evaluated • The main result shows that: • The iBanner (personal) was the most preferable mode, although the participants noticed less notifications than the participants who received the notifications at the iClock or with the sBanner (public). • The iClock was the least preferable mode. Interesting is that the subjects did not notice the iClock’s bell ring even though we thought it was a clear signal. • The rate of noticed messages was highest among the participants who received the notification on the sBanner. • The users did not think that the notifications interfered with their group work.

13. Some user comments on the work environment M 35: “More people get engaged! Good to have many displays to work on! It was some problems with the table, because it only could handle one at the time. You could concentrate at one part (your display), and at the same time work with the whole problem.” F 24: “Flexible with three displays so you could cooperate, but also to ‘mind you own business’. After a while we found a good routine to work at the table, but sometimes the routine was disturbed, because only one person could select [a picture] at a time.” F 25: “Very nice, it makes it possible to have a good dialogue, everyone are involved, and ‘make their part’, and doesn’t have to feel set aside. Definitely good for problem solving, and other collaborative work.” M 27: “I think this is great fun and exciting with this technology. The possibilities to move object between different areas increase the interaction degree, I think. The overview is very good with the two big displays.” M 29: “Interesting with direct manipulation of the objects with respect to the fingers. Frustrating that only one could touch the display at the same time. It was difficult to get a fast overview of the material on the three displays. It was difficult to compare the pictures between the table display and a wall display – it was easier to compare between the wall displays.” F 23: “It was funny and easy if you ignore that two people couldn’t work at the table at the same time. Easy to grasp.” F 40: “Good, especially for group work, in for example design phases, analysis and decision support. Three people I think is optimal, if you are getting more people it will get little space, especially due to that only one person at a time could influence the display. With optimal I mean that everyone is active.” F 44: “The room is soothing. It was quite easy to concentrate on the task. Good that you could move around the table, and also to have additional displays around [the table]. Good as a ‘focus room’.”

14. Future Work New Prototype During the autumn of 2002 a new scaled up prototype will be developed for the iLounge. For the chosen task, physical space and scenario, the mechanisms for non-intrusiveness will be extended, redesigned and reimplemented. In the new prototype the distributed events occuring will not only be shown as notifications but small scale interactive sessions of sms character. The prototype will be iteratively enhanced until end of February 2003. A more realistic User Study During March-May 2003, a larger scale user study will be performed based on student groups from a design oriented course at KTH. The students on this project based course will be using the iLounge for the series of sessions they need to finish their project work. In this study there will be real distributed events occuring during the group sessions. This study will run for a longer time period and we will be able to also study other aspects of the problem, compared to the first user study described above.

15. Here! Visit the FEEL project exhibition

16. Your input! Peter Lönnqvist {peterl@dsv.su.se} Hillevi Sundholm {hillevi@dsv.su.se}