Evalueren van Groupware

1. Evalueren van Groupware J.H. Erik Andriessen Technische Universiteit Delft

2. Examples of ICT supported groups • Teams of consultants in Germany • Aviation design teams • Home nursing in Sweden • Unilever communities • Ambulance ‘teams’

3. New work arrangements Virtual offices • Telework / Telecommuting • Hotdesking / Hoteling • Home workers • Mobile work Virtual groups • Virtual fixed teams • Virtual mobile teams • Virtual communities

4. Developments in work settings • Many modern organisations are “networking organisations”. • Many problems/tasks require people with different skills and experiences • Workforces are more distributed, and organizations are becoming more “virtual” • Much work is done in groups • Groups often work inefficient • Dispersed cooperation is possible, but to make such teams effective, knowledge is needed about: • the functioning of groups in general, • virtual groups, and • the possibilities of ICT support

5. Virtual teams So: Virtual teams have advantagesBut: Virtual teams have also problems: • Interpersonal communication is more difficult : No non-verbal signals; no unplanned encounters; no context awareness • And therefore : Difficulty in all kinds of group processes: collaboration, coordination, developing trust, exchanging knowledge • ICT tools have to match with the type of group and its task; And they have to be supportive for the group processes

6. Global virtual teams GVT’s have even more problems: • Time zone differences • Language differences \ Cultural differences Research suggests: In individualistic cultures (EU, USA) people prefer direct expression of opinions; In virtual groups they prefer synchronous communication, through telephone, video and chat. In collectivist cultures (Asia, Africa) people are sensitive to non-verbal signals and group relations. In virtual groups people prefer asynchronous communication, to be able to express themselves more carefully: e-mail

7. What is “Groupware?” • Tools (hardware, software, processes) that support person-to-person collaboration • This can include e-mail, bulletin boards, conferencing systems, decision support systems, video and workflow systems, etc… • Some common groupware acronyms: • Group Support Systems (GSS) • Group Decision Support Systems (GDSS) • Electronic Meeting Systems (EMS) • Bulletin Board Systems (BBS) • Group Collaboration Systems (GCS) • Computer-Supported Cooperative Work (CSCW) systems

8. Three types of interaction All collaboration technology always implies: • 1. Human - computer interaction • 2. Human - database interaction: (information seeking) • Internets • Intra-net • Group networks • 3. Mediated interpersonal interaction (communication)

9. A Simple Classification of Groupware (adapted from Johansen, 1991) Time Same Different GDSS; Support for FtF Meetings, Email, Bulletin board, Comp. Conf Same Location Teleconferencing, Instant Msg Chat, Whiteboard, Video Email, Bulletin board, Comp. Conf Web based CS Different

10. A GDSS Example

11. Video Conferencing

12. Groupware system

13. Five Basic group processes co-operation co-ordination communication learning by knowledge sharing social interaction team building

14. Individual interpretation and performance Individual rewards cooperation coordination communication Group vitality social interaction learning Organisational outcomes Emerging structures Changes in organisational setting Organisational environment Dynamic Group Interaction model Processes Outcomes Group characteristics Group Technology Persons Task Life cycles Formal structure Group Culture Reflection Learning Appropriation Physical setting

15. Basic Principles • The effectiveness of a group can be expressed in terms of three types of outcomes, i.e. (quality and quantity of the )products, individual ‘rewards’ and vitality of the social relations. • Effectiveness depends on the quality of the individual preformance and six group processes, which have to match • The quality of the group processes depends on the support of six conditions, and on the interaction with the environment. • The six aspects of the context-of-use have to fit to each other. • Groups develop and tools become adopted and adapted to, through interaction processes and feedback. SUPPORT – MATCH – ADAPTATION

16. To evaluate the role of groupware tools. Evaluate • Technical properties • Degree of fit to task users, group, setting, other characteristics • Degree of support for processes • Effects on outcomes • Possibilities to adapt

17. Evaluation issues in more detail • Describe the tool characteristics: reliability, portability, maintainability, network performance, costs, infrastructural quality, security/privacy and evaluate whether this is adequate (ISO-9126) • Describe the functionalities • Analyse the task and evaluate whether the functionalities fit the task • Analyse the users and evaluate whether the tool fits the users (usability) • Analyse the group (structure, culture, setting) and evaluate whether the tool fits the group • Evaluate whether the tool supports (or at least does not hinder) the group processes:communication, co-operation, co-ordination, learning, social interaction. • Evaluate whether the tool contributes to (or at least does not hinder) individual, group, organisational outcomes. • Evaluate to which extend the tools can be adapted to learning and new uses

18. Evaluation principles • - Evaluation should be integrated in the design process from the very beginning. • - The design and the evaluation process should be iterative and stakeholder centred; critical success factors of stakeholders should be formulated. • - Evaluation can take place at three periods in the design life cycle • a. Concept evaluation • b. Prototype evaluation • c. Operational assessment

19. Analysis of task, context technology, stakeholders Identification of technological and organisational options Development of Future Usage Scenarios Identification of Stakeholders’ success factors Design requirements Choice of Evaluation approach Concept Evaluation’ Analysis of potential impacts Prototype Evaluation Iterative tests of prototypes Iterative design and implementation Operational Evaluation Use, experience, adaptation of new system

20. Lessons learned (1) 1.     Groupware is part of a social system. Design not for a tool as such but for a new socio-technical setting. 2.    Design for several levels of interaction, i.e. for user friendly human computer interaction, adequate interpersonal communication, group co-operation and organisational functioning. 3.    Design in a participative way, i.e. users and possibly other stakeholders should be part of the design process from the beginning. 4.   Analyse carefully the situation of the users. Success of collaboration technology depends on the use and the users, not on the technology. Introduction should match their skills and abilities, and also their attitudes, otherwise resistance is inevitable. 5. Analyse carefully the context, since success of collaboration technology depends on the fit to that context. The more a new setting deviates from the existing one the more time, energy and other resources should be mobilised to make it a success.

21. Lessons learned (2) 6. Introduce the new system carefully. Apply proper project management, find a champion, try a pilot, inform people intensively 7. Train and support end-users extensively 8. Measure success conditions and success criteria before, during and after the development process. Only in this way you can learn for future developments. 9. Plan for a long process of introduction, incorporation, evaluation and adaptation. Groupware is not a quick fix. 10. Despite careful preparations groupware is appropriated and adapted in unforeseen ways. Keep options open for new ways of working with the groupware, because this may result in creative and innovative processes.

22. Evaluation methods 1. Inspection methods • Heuristic Evaluation: 2. Performance analysis • Human Reliability Analysis 3. Behaviour analysis • Diagnostic Recorder for Usability Measurement (DRUM) 4. Effort and satisfaction • NASA-Task Load IndeX (NASA-TLX) • Measuring the Usability of Multi-Media Systems (MUMMS) • MultiMedia Communication Questionnaire (MMCQ)

23. Evaluation Methods (2) 5. Task aspects and relations • Extended Delft Measurement Kit 6. Network performance 7. System usage and interaction registration • Automatic registration of the use of the system • Coding schemes for communication content

24. Situational constraints • Interpretation : • personal goals / task and perceived • usefulness of tools • social relations and norms • perceived ease of use of tools and • perceived situational constraints Tool Person Task Tool Choice and Use Task Performance Setting Motivation to act and to choose and use tools Reflection Learning Appropriation Organisational environment Outcomes

25. Diversity Hypothesis Fully Dispersed Three Subgroups Two Subgroups Most ConflictLeast Trust Least ConflictMost Trust