Collaborative Technology Adoption: A Case Study of Success and Challenges

1. Steven E. Poltrock Mathematics & Computing Technology Phantom Works The Boeing Company Presentation to the 2003 International Symposium on Collaborative Technologies and Systems (CTS 2003) January 22, 2003 Collaborative Technology Adoption: A Case Study of Success and Challenges

2. Agenda • Collaboration technology in a social context • Case study: A data conferencing system • Adoption rate and uses of data conferencing • Studies of 5 early adoption teams • A model of innovation diffusion (technology adoption) • A survey of 194 data conference users • Results and conclusion

3. Complex Systems Are Built through Teamwork

4. Teamwork Across Major Boeing Sites Requires Collaboration Technology

5. Collaboration Technology Supports Diverse Collaboration Activities Real time Asynchronous • AV conferencing • Telephone • Chat • Broadcast video • E-mail • Voice mail • FAX Communication Information sharing • Whiteboards • Application sharing • Meeting facilitation • Collaborative virtual environments (CVEs) • Document management • Threaded discussions • Knowledge repositories • Team workspaces • Program repositories • Floor control • Session management • Workflow management • Case tools • Project management • Calendar & scheduling Coordination

6. Collaboration Occurs in, and Is Shaped by, a Social Context Organization: A number of persons or groups having specific responsibilities and united for a specific purpose Team:A group organized to work together Community: A group or class having common interests Program / Project: An undertaking requiring concerted effort

7. Collaboration Technology Supports Diverse Collaboration Activities Teams Real time Asynchronous • AV conferencing • Telephone • Chat • Broadcast video • E-mail • Voice mail • FAX Communication Information sharing • Whiteboards • Application sharing • Meeting facilitation • Collaborative virtual environments (CVEs) • Document management • Threaded discussions • Knowledge repositories • Team workspaces • Program repositories • Floor control • Session management • Workflow management • Case tools • Project management • Calendar & scheduling Coordination

8. Collaboration Technology Supports Diverse Collaboration Activities Organizations Teams Real time Asynchronous • AV conferencing • Telephone • Chat • Broadcast video • E-mail • Voice mail • FAX Communication Information sharing • Whiteboards • Application sharing • Meeting facilitation • Collaborative virtual environments (CVEs) • Document management • Threaded discussions • Knowledge repositories • Team workspaces • Program repositories • Floor control • Session management • Workflow management • Case tools • Project management • Calendar & scheduling Coordination

9. Collaboration Technology Supports Diverse Collaboration Activities Organizations Teams Communities Real time Asynchronous • AV conferencing • Telephone • Chat • Broadcast video • E-mail • Voice mail • FAX Communication Information sharing • Whiteboards • Application sharing • Meeting facilitation • Collaborative virtual environments (CVEs) • Document management • Threaded discussions • Knowledge repositories • Team workspaces • Program repositories • Floor control • Session management • Workflow management • Case tools • Project management • Calendar & scheduling Coordination

10. The Challenge of Collaboration Technology Adoption • Technology adoption is a slow, phased process • Physical distance is an obstacle to adoption • People learn from neighbors • Organizational mandates have limited range • Collaboration technologies require universal adoption but have inherent limiting properties • Tragedy of the Commons • Critical mass • Difficulty of learning infrequent features • Visibility of performance Late adopters Early adopters Time

11. Data Conferencing Technology: A Case Study • Developed and piloted Boeing’s data conferencing infrastructure in 1997 • Transitioned the technology to a support organization • Studied its use by Boeing teams and provided feedback • Evaluated new technologies as they emerged

12. Data Conferencing Infrastructure Instructions & Downloads Directory Server Conference Server Public Internet Corporate Intranet SGI Meeting MS NetMeeting Sun Forum HP Visualize

13. Conferences Hosted on Workstations Instructions & Downloads Directory Server Conference Server Public Internet Corporate Intranet SGI Meeting MS NetMeeting Sun Forum HP Visualize

14. Conferences Hosted on a Server Instructions & Downloads Directory Server Conference Server Public Internet Corporate Intranet SGI Meeting MS NetMeeting Sun Forum HP Visualize

15. Growth of Data Conference Use

16. Overcoming distance Add distant participants to a face-to-face meeting Virtual meetings with no face-to-face participants In teams and small groups Team meetings to review progress Document collaboration Telecommuting from home or other sites In organizations Training Computer assistance Presentations In communities of practice Distributed meetings Presentations Demonstrations Examples of Data Conferencing Uses at Boeing

17. Observational and Experimental Studies of Early Adopter NetMeeting Usage • In the first six months of deployment we observed 4 teams that used similar approaches • NetMeeting to show and interact with information • Teleconferencing for voice • 2 meeting configurations • Face-to-face meetings with remote participants at desktops • Virtual meetings with no meeting room • For a 5th team, we manipulated their physical and technology configurations

18. General Benefits and Problems • Application-sharing was the most used feature and clearly added value • Audio-conferences alone were unsatisfactory • Access to last-minute changes, e.g. microphotograph • Shared reference improved efficiency of distributed teams • Problems coordinating interactions • Meetings started late, technology use limited • Interaction hardest for remote members • Difficulty in knowing who was present, identifying remote speakers, coordinating participation • “Are they pausing for a comma, or a period?” • Could not make sense of others’ on-line behaviors • Face-to-face meetings were markedly different: side-discussion, story-telling, spontaneity • “I hear the voice, but there is a vacancy for the whole human being.” • New roles emerged in successful team: meeting and technology facilitators

19. Use of Technology Was Minimal • Some did not acquire NetMeeting in this early phase of deployment • Heterogeneous computing and support environments, little or no help, and resistance to learning new technologies • One heard that a lot of time would be wasted getting synchronized • Mostly presentation mode • One person shared, others just observed • No instances of using collaboration, whiteboard, or file transfer • Chat used in one group • Even this limited use was difficult • Frequent problems connecting to a session • Cannot find the shared window under the NetMeeting interface • When email or calendars were shared, others were surprised that this was possible

20. Technology Enabled Greater Participation from More Locations • Barriers are diminished for some • D at main site: Does anyone in this room understand what he’s saying? • Remote site: I do. • D: You’re not in this room. • Remote site: I’m in the global room. • Face-to-face or virtual is a choice for some • Evolution: face-to-face  mixed  virtual • Scientific team collected data

21. Face-to-face meeting Audio conferencing NM Number of Attendees Meeting Date Scientific Team Attendance

22. NM Face-to-face meeting Audio conferencing Number of Sites Meeting Date Number of Scientific Team Sites

23. Staff Meeting Experiment • Weekly meeting held for years • 18-question post-meeting survey covering meeting productivity, process, technology use • Meetings surveyed: 4 in room, 2 split between CRs, one with manager and OA in one room,others in CR • Experimenters took notes on meeting process

24. Satisfaction: FTF versus Distributed Meetings

25. Given these Challenges, How Did Data Conferencing Achieve Such Widespread Adoption? • The typical drivers for adoption of innovations are: • Management mandate • Collocated colleagues • Opinion leaders or change agents • What were the drivers at Boeing? • Management used it but few mandated its use • The IT organization provided it but did not advertise or advocate it

26. Rogers’ Model of Innovation Diffusion • Key properties of innovations • Compatibility: is it needed and does it fit existing work and systems? • Observability: how can people “observe” across distance? • Relative advantage: i.e. over other technologies in use? • Five stages of adoption • knowledge of the innovation • persuasion to use it • decision to adopt it • implementation of the innovation • confirmation that adoption was appropriate • Distance can be a barrier

27. Barriers to Early Adoption • Barriers to decision, persuasion, and implementation • Lack of support from managers • Different platforms • Team participation was part-time • No local technical support • Lack of peer pressure • Discouragement at local site

28. One Site Required an Exception Process • Management’s concerns • “You might say, what’s the harm in using it? On the surface, there’s no harm, but the really damning thing of what happens is that people then begin to alter or implement new processes and procedures surrounding that capability on that product.” • “Somewhere, someone, would need to place a value on collaboration. What’s the payback? What are you gonna get out of it to offset the costs?” • Some thought that management could control its use • “Without approval, you will not be able to enter any meetings even if you have NetMeeting loaded.” • A champion supported its adoption while accepting management’s limits • “I particularly find it rewarding to hear back from people when they are using the more powerful tools of NetMeeting to collaborate on documents together as I think this product can change the way we work together as a company.” • Critical mass became a compelling force • “I was told yesterday that I was the only one who did not have NetMeeting.”

29. Surveyed 194 Data Conference Users in October 2000 – February 2001

30. Frequency and Duration of Usage Are Independent

31. They Learned about It from Colleagues

32. They Told Many Others About Data Conferencing “Yeah, just as a course of doing business. You ask them if they have NetMeeting. If they’d say no, you’d say, ‘well you might want to get it loaded on your PC so we can use it.’”

33. Reasons for Introduction

34. Remarkably Little Use of Most Features

35. Great Diversity in Conference Configuration

36. Rogers’ Model of Innovation Diffusion Again • Key properties of innovations • Compatibility: is it needed and does it fit existing work and systems? • Observability: how can people “observe” across distance? • Relative advantage: i.e. over other technologies in use? • Five stages of adoption • knowledge of the innovation • persuasion to use it • decision to adopt it • implementation of the innovation • confirmation that adoption was appropriate • Distance can be a barrier

37. Reviewing Rogers’ Key Properties • Compatibility • Well integrated in the IT infrastructure • Increasing geographic diversity created the need • Observability • The results were immediately observable • Less observable features were rarely used • Relative advantage • Telephone alone was insufficient • Other approaches were too expensive (video) or staying synchronized was too difficult

38. Reviewing Rogers’ Stages • Knowledge • Learned about it from local and distant collaborators • Persuasion • Necessary for participation in meetings • Opinion leaders were often at other sites • Decision to adopt • Facilitated by ready availability at no cost • Implementation • Some early adopters struggled • Confirmation • Frequent participation in distributed events • But new problems emerge such as need for security

39. Summary • A collaboration technology was widely adopted and heavily used • There were many barriers to adoption • There was no management mandate • The usual adoption paths were not open • Adoption was driven by working together across distance • Adoption is a slow process even for a single person • People most frequently learned about it while attending meetings • Useful results are obtained by simply joining a meeting • Most people used few features • People misunderstood the system architecture and capabilities

40. Conclusions • Avoid collaboration technologies that require time or effort to learn • Ensure that some value is achievable with minimal effort • Few will invest the time required to learn an infrequently used technology • Complex features will be used by few people • Important features must be visible • Users build mental models based on what they see • Consider how one user will learn from others • Some users learned how to join meetings by watching • Provide compelling value