VERDI C-Vibe Overview & Evaluation Results

1. VERDI C-Vibe Overview & Evaluation Results

2. Research Perspective: Targeted Innovation Dimension Future of Learning: Advanced Simulations • Technically • Adv. User Interface • (3D & 2D real-time, multi-user Multimedia) • Adv. User Experience • (multiuser, avatar-based navigation & interaction) • Adv. Distribution Platform • (Internet, multicasting, satellites, distributed dynamically inter-operating servers) • Conceptually • Adv. Simulation Models • (based on Personality & Social Psychology) • Adv. ‘Social Games’ • (from quantitatively- to qualitatively-based interactions and interventions simulations) • Adv. Scenarios/Dynamics • (relevant, complex domains such as Change Management and Virtual Teams Dynamics)

3. Advanced Management Simulations – Research & Practice Dynamic Models of Agent-based Scenarios Agents in C-Vibe IP Distributed Teams Experiential Learning & Change EIS Simulation Kernel humans • Socio-cognitive Agents for: • - Mimiking human behavior in social contexts • - Providing socially rich Experimentation playgrounds • Formalizing and validating models from Personality Theory, Social Psychology, Network Analysis, Innovation & Diffusion Models in different (sub)cultural contexts Angehrn, A.A., Y. Doz and Jill Atherton, Business Navigator: The Next Generation of Management Development Tools, Focus, 1, 1995, pp. 24-31; or 93/37, INSEAD/CALT Working Paper 1. Angehrn, A.A. and J.F. Manzoni, A High-Tech Spin on Organizational Learning, Chief Executive, April 1996. Angehrn, A.A. and T. Nabeth, Leveraging Emerging Technologies in Management Education: Research and Experiences, European Management Journal, 15, 3, 1997. Manzoni, J.-F. and A.A. Angehrn, Understanding Organizational Dynamics of IT-Enabled Change: A Multimedia Simulation Approach, Journal of Management Information Systems, 14, 3, 1997. Angehrn, A.A. and T. Nabeth, C-VIBE: A Virtual Interactive Business Environment addressing Change Management Learning, Proceedings of ICALT 2001, Madison, USA.

4. VERDI C-Vibe Specific Focus: Change Management. Why? 75% of all transformation efforts fail 50-75% of re-engineering projects fail 78% of IT projects fail Key Factors: Organizational resistance 82% Insufficient exec. sponsoring 72% Unrealistic expectations 65% Inadequate programme mgmt 54% Unclear business case 46% Lack of qualified resources 44% Scope of expansion/uncertainty 44% Ineffective leadership 43%

5. VERDI C-Vibe Pedagogical Objectives Secondary Objective Primary Objective Understanding Interpersonal and Group Dynamics (Distributed Virtual Team) Understanding Change Management & Diffusion Dynamics

6. VERDI C-Vibe Overall Value Proposition & Experience -Streaming Videos -Discussion -Streaming Videos -Discussion Motivation Follow-Up Introduction to the Challenge Debriefing Phase -Intro experience -Agents’ advice -Other members -Knowl. Exchange -Initiatives/Practice -Play it again, Sam 3D Simulation Experience Change Management Experience Virtual Team in 3D Spaces Experience

7. VRSAT SERVER C-VIBE Shared Objects Upstream commands Upstream commands Upstream commands Downstreamed served worlds User Client 1 (e.g. student) User Client 2 (e.g. expert) Application Client HTTP reply HTTP request EIS Simulation Server VERDI C-Vibe Technical Architecture (Overview)

8. Constituencies Involved in Finetun’g, Evaluation and Dissemination -Peers and Professors -MBA Students -MBA Student (INSEAD) -Top Schools Network -Executives in Organizations (CIA, LinKS) -Academic/Research/Education Community -Broader Public/Local/Regional Communities VERDI C-Vibe Immediate Target/Impact Virtual Group Dynamics & Role-Playing Lab Next Step Target/Impact EdComNet eLearning in Urban Communities

9. Subject Category Type of Subjects Size of Description of Subject Groups Sample Size 1 Management students 10 INSEAD MBAs 10 2 Management trainers and researchers 4 faculty members from three French business schools and one English university, and 4 researchers in management 8 3 Practicing Managers 13 managers operating in France, Greece, England, Germany and France in different functions within companies including IBM, France Telecom and EdF/France, Styria/Austria, Pouliadis and Ernst &Young/Greece. 13 4 Community members/citizens from a French town (Chartrettes, Seine-et-Marne) 14 subjects representing a variety of profiles ranging from university students (3), elected municipality representatives (3), employed or self-employed managers/salespeople (5) and retirees (3) 14 Constituencies Involved in Finetun’g, Evaluation and Dissemination Table: Subjects overview

10. Highlight of Lessons Learned from Evaluation Phase • Technical/Usability Dimension (1) • 3D Familiarization Challenge • Visualization Challenge • Real-time Coordination Challenge Although an increasing number of people have recently been exposed to 3D applications, typically through single-user games, familiarity with 3D, multi-user, real-time environments is still not widespread. Whatever the application/game is in which users have to be involved, it is highly desirable to design brief experiences to be suggested to novice users in order to enable them to familiarize with basic navigation in 3D spaces and interaction with avatars and bots (human, and software-controlled dynamic entities inhabiting such environments). When provided enough time and guidance to get familiar with the 3D environment and with avatars interactions even the subjects with a high age (which are usually not very comfortable with computers) found basic navigation in 3D spaces not only easy but even an entertaining new experience, becoming ready for advanced applications like C-Vibe.

11. Highlight of Lessons Learned from Evaluation Phase • Technical/Usability Dimension (2) • 3D Familiarization Challenge • Visualization Challenge • Real-time Coordination Challenge The design of 3D spaces with the appropriate size and which include objects with embedded textual information or which can be manipulated/re-positioned realtime, is non-trivial. Appropriate size of 3D rooms and objects, readability of textual information and visualization of dynamic characteristics of objects (e.g. ‘passive’ versus ‘active/clickable’ objects) can be achieved only through extensive user testing. In particular, in applications including a large amount of textual content like C-Vibe (explanations, narratives, etc.) it is critical to decide which information components are better presented separately in 2D windows linked to specific 3D objects.

12. Highlight of Lessons Learned from Evaluation Phase • Technical/Usability Dimension (3) • 3D Familiarization Challenge • Visualization Challenge • Real-time Coordination Challenge Applications like C-Vibe (compared to the majority of 3D games and edutainment products) take advantage of the multi-user dimension and require users to coordinate real-time. For instance, the C-Vibe simulation requires distributed users to first formulate and agree upon a joint strategy. Our findings confirm that tasks like this one, which are relatively simple in face-to-face settings, can become very complex and time-consuming in distributed environments. After developing a strategy, C-Vibe (and all similar simulation/educational experiences which will be developed in a near future) requires users to take a large number of group decisions and coordinated actions. Current coordination mechanisms (like simple chat rooms and whiteboards) can provide only limited support to such tasks, and will need to be complemented with more sophisticated 3D coordination and decision support tools suited to distributed teams operating in 3D environments.

13. Highlight of Lessons Learned from Evaluation Phase Learning Value Dimension (1) Confirmation of underlying Hypothesis concerning Learning Value VERDI3D realtime, multiuser technology + (1) experiential and (2) collaborative eLearning approach adopted and developed in the C-Vibe application supports successfully + enables fits + address management learning in new significant application areas(particularly change management, group/meeting dynamics, and the development of competencies related to high-performing virtual teams) advanced expectations of how to best integrate Info Comm Technologies (ICT) in existing learning processes

14. Constituencies Involved in Finetun’g, Evaluation and Dissemination Learning Value Dimension (2) Supporting Evidence 1 The capability of modelling and simulating ‘social’ and ‘inter-personal’ contexts (managerial situations involving interacting with people rather than taking quantitative decisions) is a key distinguishing characteristic of eLearning systems like C-Vibe. 2 The multi-user dimension of C-Vibe introduces a valuable ‘learning from each other’ factor enhancing the perceived usefulness of the system. 3 Requiring users to coordinate decisions and actions in distributed settings (see also last section) introduces a high level of complexity users are not familiar with and which should be particularly supported/trained in advance in order to become a productive learning experience (rather than a frustrating one). 4 The value of the 3D dimension (versus 2D applications) in systems like C-Vibe appears particularly (and probably only) when the application requires extensive interaction among groups of simultaneously online users (like when formulating a strategy and taking decisions in C-Vibe).

15. Constituencies Involved in Finetun’g, Evaluation and Dissemination Learning Value Dimension (3) Supporting Evidence • 5 The 3D dimension introduces a ‘game’ element which can potentially negative influence/ distract from the learning focus of the application. In addition, this ‘game’ component can have a discouraging impact (‘gimmick’ association) and generate a-priori resistance on users associating learning with a ‘serious’/non-game situations. This resistance can be only eliminated after involving subjects into direct experimentation (after which, this form of a-priori resistance typically dissolves). • 6 Although not intended to operate in this direction, C-Vibe helps users to learn (by doing) about group dynamics. Educational tools like C-Vibe could be effectively used to make people learn how to operate effectively in distributed teams and using or designing virtual environments supporting distributed managerial decision making and meetings in general. • Educational systems of the type of C-Vibe are potentially attractive for broader audience than management. Once initial barriers (scepticism towards 3D in the learning process) are overcome, they could serve the acquisition and development of ‘social skills’ also beyond organizations (other educational as well as social/political contexts). • 8 Compared to other learning methods, C-Vibe provides a more intensive/immersive experience which is particularly relevant in developing management skills like the mastering of complex group dynamics.

16. Constituencies Involved in Finetun’g, Evaluation and Dissemination Learning Value Dimension (4) Supporting Evidence • 9 The learning value of 3D simulation experiences like C-Vibe can only be realized when followed by structured debriefing sessions in which the experience is reviewed and analysed. This should be ideally supported by the possibility to ‘record and replay’ simulation sessions (like with videos). In addition, to support learning, the (behavioural) models underlying the simulation need to be made explicit to the user (‘opening the ‘black box’) after a session. • C-Vibe, rather than just providing a ‘virtual amphitheatre’ in which users can discuss their experience (ideally with experts supporting the analysis of their simulation sessions) should be extended with tools supporting follow-up collaborative learning among the users (from the same or from different teams) in forms of virtual learning communities operating both realtime and asynchronously. The current 3D virtual spaces (Virtual Boardroom and Virtual Amphitheatre) should be extended and adapted accordingly. • A specific value of learning experiences like the one provided by C-Vibe is the one of ‘forcing’ users to address challenging situations in which they can realize the limits of their competencies. Compared to more traditional learning approaches, the resulting dissatisfaction of the status quo generates motivation for further learning and competency development.

17. Constituencies Involved in Finetun’g, Evaluation and Dissemination Evaluation Conclusions (1) Evaluation provided sound basis for SPECIFIC GOIDELINES towards: (1) improving the design of future versions of C-Vibe and similar eLearning systems  increase learning value and decrease adoption barriers (2) identifying the dimensions along which to compare C-Vibe to other eLearning systems  factors for in-depth comparative studies focussing on learning value (3) integrating C-Vibe in broader educational and eLearning contexts  design effective blended learning approaches ‘around’ C-Vibe

18. Constituencies Involved in Finetun’g, Evaluation and Dissemination Evaluation Conclusions (2) In summary, the findings indicate a high acceptance of the approach and the recognition of a very high potential for 3D educational experiences like C-Vibe. On the technical/usability level a number of challenges have been identified thanks to the developed C-Vibe prototypes, including a 3D Familiarization Challenge, a Visualization Challenge, and a Real-timeCoordination Challenge - particularly important for multi-user, real-time environments in which different players/learners need to form and coordinate their decisions as a group, like in the case of C-Vibe. At the level of the underlying concept and the learning value, the evaluation confirmed the hypotheses of the project. Provided the 3 technical challenges mentioned above are addressed adequately, 3D, multi-user, real-time environments resulted to provide a promising way of complementing and extending traditional educational approaches. This appears to be particularly the case when addressing the development of relevant but complex ‘soft skills’ or social competencies such as the capability to manage change and innovation effectively in organizational or community contexts, as well as the development of competencies related to learning to operate effectively in distributed teams and virtual environments supporting group decision making and collaboration.