Hypothesis: Reuse of End-User Code is an Adoption Process

1. Hypothesis: Reuse of End-User Code is an Adoption Process Christopher Scaffidi CMU ISR SSSG Apr 20, 2009

2. End-user programmers (EUPs) often reuse programs, code, and other computational artifacts. • Code created by other EUPs • Web macros • Spreadsheets • Matlab scripts • JavaScript and HTML • Agentsheet images and rules • Code created by professional programmers (PPs) • JavaScript interpreter • Spreadsheet evaluation environment • Other components designed by PPs for reuse • Code snippets / examples provided by PPs • All of which I’ll refer to as reuse of code in this talk. I am aware that this might be a misnomer… discuss later. Introduction• Models • Hypothesis

3. Systems supporting reuse of EUP-created codeare based on… • What seems like a “natural choice” of features • Web macro wiki (Little 2007) • What features proved successful in prior systems • Agentsheet repository (Rausch 1996) • Tope repository (Scaffidi 2009) • Reflections about the process of reuse (Biggerstaff 1987) • Spreadsheet repository (Walpole 1997) • Empirically trying out features • Matlab FileExchange (Gulley 2006) Can we know, in advance, what to build? Introduction• Models • Hypothesis

4. Needed: a model describing the reuse of EUPs’ code. • What is a good empirical model for reuse of EUP code? • Key steps involved in reuse? • Key considerations for judging if reuse is successful? • Key factors driving successful reuse? • Such a model could help… • Designers of EUP systems • Organizations that use EUP systems • Eg: suppose the model says that EUPs usually… • Want to try out code before reusing it • Have more success in reuse if code is “try-out-able” • then EUP system designers would know that they should work very hard to support “try-out-ability” Introduction• Models • Hypothesis

5. Known: Adoption models accurately describe many (re)use processes. • “Adoption” = “a decision to make full use of an innovation as the best course of action available” (Rodgers 1995) • Adoption models have accurately described (re)use of… • Hybrid strains of corn seed by Iowa farmers (Rodgers 1995) • CASE, structured programming, RDBMSs by PPs (Fichman 1993) • A new SE methodology by PPs (Riemenschneider 2002) • Computers by end users (Scaffidi 2005) • Information systems by end users (Delone 2003) • Spreadsheet tools by EUPs (Brynjolfsson 1996) … and others discussed later. Introduction• Models • Hypothesis

6. Hypothesis: Adoption models accurately describe EUPs’ reuse of EUP-created code. Since adoption models work well for many kinds of reuse by end users, how about reuse of EUP code by EUPs? • Reasons to question this hypothesis: • EUPs focus on work… is reuse a “decision” aimed at attaining a “best” course of action? • EUPs rarely design for reuse… is their code intelligible enough to support rational decisions? • Other theories might help, but lack the right focus: • Agency theory – incentives, delegation, decision structure • Post-rational behavior models – motivation, trust, activities • Behavioral economics – emotions, framing, cognitive limitation Introduction• Models • Hypothesis

7. Outline • Introduction • Models from adoption theories • TAM with external variables • Diffusion models • Elaboration from economics • Hypothesis • (Future) evaluation • Implications Introduction• Models • Hypothesis

8. Basic Technology Acceptance Model (TAM) (Davis 1985) Introduction • Models• Hypothesis

9. TAMplus external variables (Lee 2003) Introduction • Models• Hypothesis

10. Diffusion models –Rate of adoption (Rodgers 1995) Introduction • Models• Hypothesis

11. Diffusion models –Cumulative adoption (Scaffidi 2005; Rodgers 1995) Introduction • Models• Hypothesis

12. Key elements in diffusion models • Innovation’s properties • Relative advantage • Compatibility • Complexity • Trialability • Observability • Communication channels • Heterophily • Time • Innovators-early adopters-early majority-late majority-laggards • Knowledge-persuasion-decision-assimilation-confirmation • Social system • Structures (eg: hierarchy) • Coordination (in decisions) • Norms (Rodgers 1995) Introduction • Models• Hypothesis

13. Prior successesof adoption models • Adoption by PPs • CASE, structured programming, RDBMSs (Fichman 1993) • Components (Tahvildari 2004) (Smith 2004) • SE methods/processes (*Riemenschneider 2002) (Pfleeger 2000) • Adoption by end users • Computers (*Scaffidi 2005) • Information systems (*Delone 2003) (*Lee 2003) • Spreadsheet tools (*Brynjolfsson 1996) • EUP tool enhancements (Kienle 2002) • Web browsers (*Agarwal 1997) • Communication technologies (*Forman 2005) • Online services (*Parthasarathy 1998) * = quantitative model (ie: not just a qualitative adoption approach) Introduction • Models• Hypothesis

14. Particularly interesting findingsrelated to adoption by PPs and EUPs • Key factors limiting reuse of PP code by PPs • Prior technology drag – affects compatibility w/ existing network • Irreversibility of investments – affects trialability • Limited sponsorship – calls for more than “communication” • High expectations – sets a high bar for each adoption stage (Fichman 1993) • Key factors limiting reuse of PP code by EUPs • Smaller organization size – less risk capacity, network effects • More organization hierarchy – limits innovation’s impact (Forman 2005) Introduction • Models• Hypothesis

15. Particularly interesting findingsrelated to adoption by PPs and EUPs • Key factors limiting assimilation of PP code by PPs • Knowledge barriers – complexity + incompatibility with training • Unrealistic expectations about positive network effects (Fichman 1999) • Key factors raising discontinuance of PP code by EUPs • Later adopters lack resources to make full use of innovation • People who encounter trouble with incompatibility lose benefits (Parthasarathy 1998) Introduction • Models• Hypothesis

16. Is reuseof EUPs’ code anadoption process?

17. Is EUP reuse of EUPs’ codean adoption process? • When an EUP reuses another EUP’s code, does he go through the typical stages of adoption? • Knowledge-Persuasion-Decision-Assimilation-Confirmation • Contextual inquiry • Identify reused code • Uncover source, history, expectations, communication, sponsorship • Did source ever provide other code? What happened? • Any sign of assimilation & confirmation for users’ own code? • Look for evidence of “not-invented-here” • Complement with survey? • Possibly a survey about spreadsheets, modeled after (Hall 1996) • Alternatively, integrate with CoScripter survey (next slide) Introduction • Models • Hypothesis

18. Is EUP reuse of EUPs’ codefundamentally an adoption process? • How well is the rate of EUP code adoption described by diffusion models? • Logistic increase in adoption? • Do users fit early/late adopter stereotypes? • Analyze logs of CoScripter web macro repository • Non-linear regression of usage to logistic curve • Use chained analysis to check how well users fit stereotypes • Complement with survey of users within IBM? • Group heavy users of popular macros, based on adoption date • Survey about key characteristics of stereotypes Introduction • Models • Hypothesis

19. Is EUP reuse of EUPs’ codefundamentally an adoption process? • What characteristics of EUP code affect reuse? • Consistent with adoption models? • Which characteristics are more/less crucial? • Return to last year’s analysis of CoScripter logs • Are the identified predictors consistent with adoption models? • Extend analysis, using additional adoption-motivated variables? • Repeat analysis using regression or factor analysis? • Do later adopters & lower assimilators tend to discontinue? • Complement with similar study of Matlab logs? Introduction • Models • Hypothesis

20. Tie together resultsinto a coherent model Perceived ease-of-use Perceived usefulness Factor 1 Decision to (re)use Factor 2 Factor 3 Assimilation … Discontinuance • Key factors in decision, assimilation, discontinuance • Properties of EUP code • Properties of communication channels and social structure • Properties of EUPs • Perhaps something like… (The interesting cascading arrowson the right are consistent with empirical researchdiscussed earlier.) Introduction • Models • Hypothesis

21. Research opportunities ifEUP reuse of EUPs’ code is an adoption process… • Helping EUPs… • To create code with key characteristics leading to adoption (eg: compatibility, observability, trialability, …) • To provide constructive sponsorship without sacrificing primary job responsibilities (ie: how to sponsor while remaining an EUP) • Helping organizations… • Manage heterophily in communication channels • Possibly through creative use of limited hierarchy • Accelerate adoption cycle • Knowledge-persuasion-decision-assimilation-confirmation • Learn from discontinuance Introduction • Models • Hypothesis

22. Thank You… • To Mary Shaw for early feedback on this work • To EUSES and NSF for funding • To you for your attention, insights, and feedback

23. References (1) Agarwal 1997 - R. Agarwal and J. Prasad. The Role of Innovation Characteristics and Perceived Voluntariness in the Acceptance of Information Technologies. Decision Sciences, Vol. 28, No. 3, 1997, pp. 557-582. Biggerstaff 1987 - T. Biggerstaff and C. Richter. Reusability Framework, Assessment, and Directions, IEEE Software (4), 2, March 1987, 41-49. Brynjolfsson 1996 - E. Brynjolfsson and C. Kemerer. Network Externalities in Microcomputer Software: An Econometric Analysis of the Spreadsheet Market. Management Science, 1996, pp. 1627-1647. Davis 1980 - F. Davis. A Technology Acceptance Model for Empirically Testing New End-User Information Systems: Theory and Results, PhD. Thesis, Sloan School of Management, Massachusetts Institute of Technology, 1985. Delone 2003 - W. Delone and E. McLean. The DeLone and McLean Model of Information Systems Success: A Ten-Year Update. Journal of Management Information Systems, Vol. 19, No. 4, 2003, pp. 9-30. Fichman 1993 - R. Fichman and C. Kemerer. Adoption of Software Engineering Process Innovations: The Case of Object Orientation. Sloan Management Review, Vol. 34, 1993, pp. 7-22. Fichman 1999 - R. Fichman and C. Kemerer. The Illusory Diffusion of Innovation: An Examination of Assimilation Gaps. Information Systems Research, Vol. 10, No. 3, 1999, pp. 255-275. Forman 2005 - C. Forman and A. Goldfarb. Diffusion of Information and Communication Technologies to Businesses, Handbook of Economics and Information Systems, Elsevier Science, 2005. Gulley 2006 - N. Gulley. Improving the Quality of Contributed Software and the MATLAB File Exchange, 2nd Workshop on End User Software Eng., 2006, 8-9. Hall 1996 - Hall, J. A Risk and Control-Oriented Study of the Practices of Spreadsheet Application Developers. Proc. 29th Hawaii Intl. Conf. System Sciences, 1996, 364-373.

24. References (2) Kienle 2002 - H. Kienle, A. Weber, J. Jahnke, and H. Muller. Tackling the Adoption Problem of Domain-Specific Visual Languages. 2nd Domain-Specific Modeling Languages Workshop At OOPSLA 2002, 2002, pp. 77-88. Lee 2003 - Y. Lee, K. Kozar, and K. Larsen. The Technology Acceptance Model: Past, Present, and Future. Communications of the Association for Information Systems (Volume 12, Article 50), Vol. 752, No. 50, pp. 752-780. Little 2007 - G. Little, T. Lau, A. Cypher, and J. Lin. Koala: Capture, Share, Automate, Personalize Business Processes on the Web. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 2007, pp. 943-946. Parthasarathy 1998 - M. Parthasarathy and A. Bhattacherjee. Understanding Post-Adoption Behavior in the Context of Online Services. Information Systems Research, Vol. 9, 1998, pp. 362-379. Pfleeger 2000 - S. Pfleeger and W. Menezes. Marketing Technology To Software Practitioners. IEEE Software, Vol. 17, No. 1, 2000, pp. 27-33. Rausch 1996 - M. Rausch. The Agent Repository-Supporting Collaborative Contextualized Learning with a Medium for Indirect Communication, Master's Thesis, Department of Computer Science, University of Colorado, 1996. Riemenschneider 2002 - C. Riemenschneider, B. Hardgrave, and F. Davis. Explaining Software Developer Acceptance of Methodologies: A Comparison of Five Theoretical Models. IEEE Transactions on Software Engineering, Vol. 28, No. 12, 2002, pp. 1135-1145. Rodgers 1995 - Rogers, E. Diffusion of Innovation, 4th edition, Simon & Schuster, Inc., New York, NY, 1995.

25. References (3) Scaffidi 2005 - C. Scaffidi, M. Shaw, and B. Myers. Estimating the Numbers of End Users and End User Programmers. Proceedings of the 2005 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC 2005), Dallas, TX, September 2005, pp. 207-214. Scaffidi 2009 - C. Scaffidi, Topes: Enabling End-User Programmers to Validate and Reformat Data, PhD thesis, Technical Report CMU-ISR-09-105, School of Computer Science, Carnegie Mellon University, 2009. Smith 2004 - D. Smith, E. Morris, D. Carney. Adoption Centric Problems in the Context of System of Systems Interoperability, 4th International Workshop on Adoption-Centric Software Engineering, pp. 63-68. Tahvildari 2004 - L. Tahvildari. Testing Challenges in Adoption of Component-Based Software. 4th International Workshop on Adoption-Centric Software Engineering, pp. 21-25. Walpole 1997 - R. Walpole and M. Burnett. Supporting Reuse of Evolving Visual Code, 1997 Symp. Visual Lang., 1997, 68-75.

26. Updated Delone & McLean modelof information system (IS) success you built the right IS you built the IS to spec you give good user support (Delone 2003) Introduction • Models• Hypothesis