Understanding and (Re)Designing Software Development Processes

1. Understanding and (Re)Designing Software Development Processes Walt Scacchi Institute for Software Research University of California, Irvine Irvine, CA 92697-3425 USA http://www.ics.uci.edu/~wscacchi/Presentations/JPL-Process-Oct01.ppt wscacchi@uci.edu

2. Recurring problems • What is the best way to organize software development project? • How to speed up development, reduce costs and improve software quality? • How to achieve the quickest development effort, lowest effective development cost, and best available product quality? wscacchi@uci.edu

3. Possible solutions • Attain and improve maturity of software development capabilities (image) • Get best people to practice standards-based development process supported by IDEs • (Re)Design your software production architecture (SPA) to optimize use of development resources, processes, and people. wscacchi@uci.edu

4. Goals • Present an approach for how to optimize software production • Identify key concepts, techniques, and tools that enable better optimization • Describe optimization transformations from business process redesign studies • Describe opportunity areas for exploitation and use wscacchi@uci.edu

5. Definitions and Differences • Software production: enterprise processes and resources that produce software • Production strategies: business strategies guiding overall approach to building software • Production architecture: configuration of enterprise capabilities to enact strategies • Optimizing production: minimizing enterprise configuration to maximize strategic options wscacchi@uci.edu

6. (Re)designing software production • What first: to-be goal vs. as-is mess? • If you don’t know where you are, any road will do (proverb) • Observation: people at work cannot describe the processes they do with high fidelity (tacit knowledge) • Redesign necessitates understanding as-is, to-be, and here-to-there • Creating high-performance work groups • Empowerment, participation, incentivization (resource sovereignty), and recognition W. Scacchi, Redesigning Contracted Service Procurement for Internet-based Electronic Commerce: A Case Study, J. Information Technology and Management, 2(3), 313-334, 2001. wscacchi@uci.edu

7. Reduce costs Reduce cycle time Improve cash flow Customer satisfaction Increase sales Improve customer service Increase productivity Open new markets Open new “channels” Be innovation leader Increase market share Enable just-in-time service delivery Software Production Strategies wscacchi@uci.edu

8. Software Production Architecture • A composite model that interrelates • software system architecture • software process architecture • development organization architecture • network infrastructure and development tools/environment configuration • documentation architecture • customer-support knowledge base architecture P. Mi and W. Scacchi,A Meta-Model for Formulating Knowledge-Based Models of Software Development, Decision Support Systems, 17(4):313-330, 1996. wscacchi@uci.edu

9. Optimizing Software Production • Strategies provide global constraints or opportunities for optimizing software production • Constraints and opportunities realized in software production enterprise • Constraints and opportunities are distributed across the software production architecture wscacchi@uci.edu

10. Optimizing Software Production • Optimization must address composite architecture of software production • Local optimization of any component architecture does not guarantee global optimality of software production • Diagnostic analyses and transformation heuristics applied to composite architectural models lead to optimization opportunities wscacchi@uci.edu

11. Optimizing Software Production • Transformation heuristics classified taxonomically • Taxonomy classifies domain-independent and domain-specific hueristics • DI transformations applied in any software production setting • DS transformations applied to specific component architectures wscacchi@uci.edu

12. Optimizing Software Production • DI transformation classes (sample): • Job scope • Worker empowerment • Organization design • Workflow streamlining • Information technology (IT) wscacchi@uci.edu

13. Research grant justification and approval process at Office of Naval Research (c. 1995) W. Scacchi and J. Noll, Process-Driven Intranets: Life Cycle Support for Process Reengineering IEEE Internet Computing, 1(5):42-49, 1997. wscacchi@uci.edu

14. Optimizing Software Production • IT transformation sub-classes (sample): • Extend IT-based support to manual process steps • Extend IT-based communication facilities to encourage information sharing activities • Extend IT-based automation to incorporate new kinds of application packages • Extend IT-based integration to interconnect and interrelate existing "islands of automation" wscacchi@uci.edu

15. wscacchi@uci.edu

16. As-is vs. to-be process wscacchi@uci.edu

17. Redesign/Optimization Results • Reduction in procurement process cycle times of 20X, annual operational savings of $10-15M. • Via transformation and realignment of information systems, business processes, corporate strategy, and work practices in a 1-2 year time frame. • Participatory design, development and refinement of computational models of new work processes, resource configurations and work practices, together as an organizational system. W. Scacchi, Redesigning Contracted Service Procurement for Internet-Based Electronic Commerce: A Case Study, Information Technology and Management, 2(3):313-334, 2001. wscacchi@uci.edu

18. Tools and Techniques • Software process redesign case web • Knowledge web for software production • Process-driven intranets • Organizational transformation wscacchi@uci.edu

19. Case study: Software development teamwork • Comparative analysis of software specification teamwork (in complex setting) • Five teams, 5-7 members, two-week (part-time) process that incorporates planning, formal notation, automated tool use, reusable assets, documentation tasks, and team shared responsibility. wscacchi@uci.edu

20. What to Understand • Work Structures and Shifts: Resource arrangements, historical circumstances, division of labor and expertise, etc. • Work Processes: Routine, habitual or emergent patterns of how work flows among people through/onto work structures • Work Practices: Behavioral discourse and social dynamics enacted through work processes wscacchi@uci.edu

21. What to Understand • Structures are domain independent constructs • Prescriptive abstractions • Processes are classes of workflow • Descriptive and derived • Prescriptive and composed • Practices are instances • Descriptive, historic and situated wscacchi@uci.edu

22. Comparative analysis of software specification teamwork • Six work structure types observed: Negotiated, Integrated, Replicated, Delegated, Prediscriminated and Separated • Three structural shift types observed: • anticipated ->, • unanticipated -->>, • role shift within work structure +. wscacchi@uci.edu

23. Work structures and shifts (data) wscacchi@uci.edu

24. Findings • Highest (lowest) quality product (measured by automated tools): T1 (T5) • Highest (lowest) productivity (self reported time expended): T5 (T1) • Note the coincidental relationship • Effectiveness of planning, automated tool use, asset reuse not clearly associated with high(low) quality or high(low) productivity wscacchi@uci.edu

25. Findings • Teams falling primarily into Negotiative and Integrative structures had higher quality • Teams falling primarily into Delegative, Pre-discriminative or Separative structures had higher productivity • Computer supported work environments must account for teamwork structures as a usage parameter. wscacchi@uci.edu

26. Tools and Techniques • Software process redesign case web • Knowledge web for software production: • Software production ontology • Taxonomy for as-is diagnosis, redesign heuristics • Best practices and lessons learned cross-linked • Process-driven intranets • Organizational transformation W. Scacchi and A. Valente,Developing a Knowledge Web for Business Process Redesign, Presented at the 1999 Knowledge Acquisition Workshop, Banff, Canada, October 1999. wscacchi@uci.edu

27. wscacchi@uci.edu

28. wscacchi@uci.edu

29. Current field study • Understanding open source software practices and processes in different domains • Academic research vs. Commercial development • Deep Space Astronomy, Software Architecture • Internet infrastructure, Networked computer games • To produce and compare case studies using narrative, hypertext, and computational renderings. wscacchi@uci.edu

30. Tools and Techniques • Software process redesign case web • Knowledge web for software production: • Process-driven intranets: • Model, prototype, and enact software production architectures • Organizational transformation wscacchi@uci.edu

31. Process-driven intranets • Enable rapid configuration of virtual enterprises (VEs) across multiple organizational domains • Virtual enterprises for wide-area software development have been demonstrated • Software production in VEs supported and enacted via process navigation (“process surfing”) J. Noll and W. Scacchi, Supporting Software Development in Virtual Enterprises, Journal of Digital Information, 1(4), February 1999. wscacchi@uci.edu

32. Process-driven intranets • Accommodate organizational autonomy and computer-supported cooperative work • Accommodate heterogeneous repositories of development artifacts (source code, etc.) • Accommodate use of local tools and development environments wscacchi@uci.edu

33. Process-driven intranets Also see, J. Noll and W. Scacchi, Specifying Process-Oriented Hypertext for Organizational Computing, J. Network and Computer Applications, 24(1):39-61, 2001 wscacchi@uci.edu

34. Process-driven intranets wscacchi@uci.edu

35. Tools and Techniques • Software process redesign case web • Knowledge web for software production • Process-driven intranets • Organizational transformation • What’s involved? • How long does it take? • How much bang for the buck? wscacchi@uci.edu

36. Organizational transformation • Collaborative participation to understand as-is, to-be, and here-to-there SPAs within constraints and contexts. • Timeframe: 6-18 months • Baseline investment (ROI): • External: Invest $1 to realize $10 savings/gain. • Internal: Staff time, infrastructure usage, etc. wscacchi@uci.edu

37. Conclusions • Software production can be optimized • Optimizing software production is a strategic option/choice that can be realized • Software production architectures and supporting technologies enable optimization • Process improvement and production optimization are complementary efforts wscacchi@uci.edu

38. References • S. Bendifallah and W. Scacchi, Work Structures and Shifts: A Study in Software Specification Teamwork, Proc. 11th. Intern. Conf. Software Engineering, Pittsburgh, PA, IEEE Press, 260-270, 1989. • R. Conradi and A. Fuggetta. Software process improvement: what can be improved? Politecnico di Milano, Dipartimento di Elettronica e Informazione, Submitted for publication, 2001. • W. Scacchi and P. Mi, Process Life Cycle Engineering: A Knowledge-Based Approach and Environment, Intelligent Systems in Accounting, Finance and Management, 6:83-107, 1997. • W. Scacchi, Understanding Software Process Redesign using Modeling, Analysis, and Simulation, Software Process--Improvement and Practice, 5(2/3), 183-195, 2000. • J.S. Choi and W. Scacchi, Modeling and Simulating Software Acquisition Process Architectures, Journal of Systems and Software, to appear, 2001 • Available at http://www.ics.uci.edu/~wscacchi/publications.html wscacchi@uci.edu

39. wscacchi@uci.edu

40. wscacchi@uci.edu