The Impact of Software Reuse and Incremental Development on the Quality of Large Systems

1. The Impact of Software Reuse and Incremental Development on the Quality of Large Systems Parastoo Mohagheghi Dept. Computer and Information Science (IDI) University of Science and Technology (NTNU) Trondheim, Norway parastoo@idi.ntnu.no Doctoral thesis presentation, 21 September 2004 Parastoo Mohagheghi- 21 Sept.2004

2. Motivation of the study • Software organizations (and researchers) have always been looking for effective strategies to develop software faster, better and cheaper: • OO analysis, CASE tools, formal methods, software reuse, agile methods etc. are proposed, but there is no “silver bullet”. • There is an absence of models and theories that can help to reason about the software without building it. • Rapid evolution of technologies does not allow proper assessment. • Collecting context-independent knowledge or representing context is difficult. • Software development is inherently complex, especially for large systems. • There is a lack of empirical studies in industry on technologies that can answer what they are good at, and not so good at. • The INCO- INcremental and COmponent-based Software Development project (2001 to 2004) tries to advance the state-of-the-art and practice for these technologies. Parastoo Mohagheghi- 21 Sept.2004

3. Definitions • “Software reuse is the systematic practice of developing software from a stock of building blocks..” [Morisio,2002]. • Reusable assets can take many forms: Component libraries, COTS and OSS components, or entire software architectures and components in a product family approach. • A reusable component in this study is a component that is reused in more than one product. • Two distinct activities: development for and with reuse. • Component-Based Development (CDB) covers all the aspects of developing (reusable, executable?) software components and assembling system from them. Components adhere to a component model. • Incremental development is covering (or discovering) requirements gradually and developing systems in increments that accumulate functionality. Iterative development is recursive application of development activities and recursive elaboration of artifacts. Parastoo Mohagheghi- 21 Sept.2004

4. Company background • Ericsson has developed two large-scale telecom systems (470 KSLOC or 1000 KSLOC in equivalent C each) that share software architecture, components and a component model, development environment and software process. • Systems are developed incrementally (5 releases of one system and 2 of the other). In periods, over 200 developers in different countries were working with design and test. • Lots of data has been gathered in 5 years in different repositories. Data from several releases are analyzed in this study. It covers trouble reports, change requests, effort, size of components, measures from internal measurement programs and inspections. • Personnel turnover has traditionally been small, but software development in Norway stopped in mid-2002. Parastoo Mohagheghi- 21 Sept.2004

5. Overview of the Ericsson packet-switched core network- GPRS Parastoo Mohagheghi- 21 Sept.2004

6. The original architecture The hierarchical architecture Rev. & GPRS for WCDMA Application - specific Re - eng. GPRS for GSM layer GPRS for GSM 3’ 4 1’ GPRS for GSM Business - specific 1 2 3 layer 6’ 2 8 Reused Common services 4 5 6 5’ 7 Layer (including component framework) Wireless Packet Platform (WPP) Wireless Packet Platform (WPP) Adaptation of RUP Adapted RUP Initial Process Evolution of the GSN software architecture and the software process model Parastoo Mohagheghi- 21 Sept.2004

7. Characteristics of the system • The approach to initiate a product family has been an extractive one and involved high risks regarding cost, quality, training, coordination and management support. • Quality requirements: Performance, availability, scalability and maintainability (or evolvability) are of great importance. There are no direct user interfaces. • Large systems face challenges in all phases of development that small systems do not: difficulties in iteration planning, complexity of design, integration and test and maintenance costs. Large systems are designed to be long-lived. • Components are developed in-house by different Ericsson organizations. Parastoo Mohagheghi- 21 Sept.2004

8. Research questions • RQ1. Why is a reuse program initiated and how is it implemented? • RQ2. What is the impact of software reuse, CBD and incremental development on the quality? The impact of development approaches on product quality metrics and on project attributes such as schedule or effort are sought. • RQ3. How to improve the practice of incremental development of product families in some aspects? • Research questions are further refined in studies. • Some research questions and hypotheses were identified from earlier work (top-down style) -> replication in new context and confirmatory studies. • Other questions and hypotheses are results of exploratory work on available data and practices in the industry, in a bottom-up style -> explorative and descriptive studies. Parastoo Mohagheghi- 21 Sept.2004

9. Research methods • Quantitativestudies by exploring (‘data mining’) company databases, studying distributions and assessing hypotheses. • Qualitative data collected from web pages, textual documents, discussions with personnel and own experience on the process and practices are combined with results from quantitative studies. • Several case studies (post-mortem), a small survey and a controlled experiment are performed. • The rationale for mix of methods has been: • The impact of introducing reuse or incremental development is widespread. • It is useful to take benefit of all available data. • Triangulation of data/methods. • Answering questions that are not possible to answer otherwise. Parastoo Mohagheghi- 21 Sept.2004

10. Overview of all studies Parastoo Mohagheghi- 21 Sept.2004

11. Contributions and research questions Parastoo Mohagheghi- 21 Sept.2004

12. Contributions in software reuse • C1. Empirical verification of reuse benefits (RQ2): • Quantitative studies of Trouble Reports (TRs) and Change Requests (CRs). 13 000 TRs and 169 CRs were analyzed. • Reused components have significantly lower defect-density than non-reused ones, and are more stable between releases (less modified). Data from 3 releases. • There is no difference in change-proneness between reused and non-reused components (#CRs/KLOC). Data from 4 releases. • First large-scale industrial study on reuse benefits. • C6a. Adaptation of the Rational Unified Process (RUP) regarding reuse (RQ1, RQ3): • Proposing changes in workflows and activities for development for and with reuse. • No empirical studies have been found on evaluating or adapting RUP in this aspect. Proposals are generic and may be reused in other adaptations of RUP. Parastoo Mohagheghi- 21 Sept.2004

13. Reuse and defect-density, release 3 • H0: Reused components have the same defect-density as non-reused ones. Rejected. • Reused components have lower defect density, but they have proportionally more defects with severity A (highest severity). • Reused components have proportionally less defects • after delivery -> more reliable. Parastoo Mohagheghi- 21 Sept.2004

14. Size vs. defect-density H0: There is no relation between defect-density and component size for all/reused/non-reused components. Not rejected. Parastoo Mohagheghi- 21 Sept.2004

15. Contributions in incremental development • C2. Increased understanding of the origin and type of changes in requirements or artifacts in incremental development (RQ2): • A quantitative analysis of CRs showed that perfective changes (enhancements and optimizations) are most common. Of these, non-functional changes to improve “quality attributes” are more frequent than pure functional changes. • Most CRs (66%) are issued by the organization itself (and not external actors). Only 34% of CRS are adaptive or customer-initiated. • Hypotheses are grounded in data. Generalization needs further study. Parastoo Mohagheghi- 21 Sept.2004

16. Contributions in incr. dev.- cont. • C3. Developing an effort estimation method using use case specifications and effort distribution in different phases of incremental software development (RQ3): • The method was adapted for incremental changes in (large) use cases. It also includes a factor to estimate effort for building on a previous release. • The impact of effort-breakdown profile on the estimation results is discussed. • Method adapted to the context and scaled up. • C6b. Improving techniques for incremental inspection of UML models (RQ3): • A small experiment (first in industry) was performed, comparing company’s inspection technique with the new and adjusted OORTs (Object-Oriented Reading Techniques). The results showed no difference in cost-efficiency, but difference in the types of detected defects. • Useful for improvement of OORTs and adapting these to the context. Parastoo Mohagheghi- 21 Sept.2004

17. Contributions in software reuse and incr. dev., and research methods • C4.Identifying metrics for a combination of reuse of software components and incremental development (RQ3): • Other literature discusses metrics for CBD. However, these metrics should be adapted for incremental development and for reuse. • C5. A data mining method for exploring industrial data repositories (RQ3): • A data mining method is developed that is based on experience from the quantitative studies. It combines top-down theory search with bottom-up hypotheses generation. • Challenges in integrating the results of several studies with one another are classified into physical and conceptual ones. Parastoo Mohagheghi- 21 Sept.2004

18. Assessing development approaches Parastoo Mohagheghi- 21 Sept.2004

19. The Impact of development approaches and context on practices Parastoo Mohagheghi- 21 Sept.2004

20. Answers to research questions • RQ1.Why a reuse program is initiated and how is it implemented? • A product family is initiated because of the similarity between requirements of the emerging system and an existing system. Software architecture is evolved. • A lightweight or extractive approach to product family adoption was chosen. Management support, common goals and common infrastructure, and experienced personnel have been critical for the success of reuse. The software process model is not adapted to the same degree. • RQ2.What is the impact of software reuse, CBD and incremental development on the quality? • Lower defect density and higher stability of reused components. • Product is getting more change-prone (in %of accepted CRs and reduced requirement stability). Quality is improved incrementally, and functionality is both enhanced and improved. Earlier releases are not evolved. • High share of integration and testing effort. Parastoo Mohagheghi- 21 Sept.2004

21. Answers to research questions- cont. • RQ3.How to combine the qualitative and quantitative results to improve the practice in some aspects? • The effort estimation method, metrics for a combination of development approaches, the research method for data mining, and proposals for adapting RUP. • Relation to INCO goals: • G1. Advancing the state-of-the-art of software engineering. Better understanding of approaches is achieved. Contributions C1, C2 and C4. • G2. Advancing the state-of-the-practice in software-intensive industry and for own students. Some feedback is given to Ericsson. Contributions C3 to C6 are reusable. • G4. Disseminating and exchanging the knowledge gained. Most results are published. 11 students were involved in the studies. Teaching in courses. Parastoo Mohagheghi- 21 Sept.2004

22. Discussion • Case study research: • Studies are performed in a real context. Context-independent knowledge is rare in SE and generalization is difficult. • Affiliation in the company provided access to data (revelatory case). • The system is a critical case for verifying reuse benefits (extractive approach, only two products and high risks). • The size of the system allows evaluating techniques for large-scale development. • In some cases, an example of industrial practice, e.g. RUP. • Incremental development is combined with reuse of software in multiple products and CBD. • All aspects of software development need adaptation; e.g. estimation method, software process model, data collection routines etc. Parastoo Mohagheghi- 21 Sept.2004

23. Conclusions • Describing different aspects of software development; i.e. the power of example. • The work also identified areas for improvement like the process model and the data collection routines. • Verifying existing theories and assessing existing methods in new contexts; i.e. the power of replication: • Reuse benefits in a large industrial system, • Estimation method and inspection techniques in a large project and with incremental development, • RUP in the context of reuse. Parastoo Mohagheghi- 21 Sept.2004

24. Conclusions- cont. • Generating new theories, hypotheses or methods by analyzing data from new perspectives (as in grounded theory) or combining the results of several studies; i.e. the power of generalization: • The origin of change requests and the distribution over functional and quality attributes, • The distribution of effort over development phases for a large project with incremental development, • The impact of development approaches on quality attributes, • Metrics for a combination of development approaches, • A data mining method based on experience. Parastoo Mohagheghi- 21 Sept.2004

25. Why such studies are important for research community and industry? • Methods should be tested in real context and for development in large. • Companies are increasingly using mainstream methods, tools and programming languages. Results are of interest to a larger community. • The studies analyzed data that was hardly studied before: evaluation of data collection routines and metrics for the company, in addition to concrete results. Parastoo Mohagheghi- 21 Sept.2004

26. Future work • Study of RUP adaptations. • Empirical studies on software evolution. • Study of effort distribution. • Estimation of incremental projects • Use cases for effort estimation. • Relevant metrics for incremental development of large systems. Parastoo Mohagheghi- 21 Sept.2004

27. Acknowledgements • Thanks to INCO, Simula and NTNU for financial support. • Thanks to my supervisor, Prof. Reidar Conradi, and colleagues at INCO and IDI. • Thanks to Ericsson for the permission to perform studies and to publish the results. Parastoo Mohagheghi- 21 Sept.2004