Written by: Daniela Damian and James Chisan IEEE Transactions on Software Engineering

1. An Empirical Study of the Complex Relationships between Requirements Engineering Processes and Other Processes that Lead to Payoffs in Productivity, Quality, and Risk Management Written by: Daniela Damian and James Chisan IEEE Transactions on Software Engineering vol. 32, No. 7, July 2006 Presented by: Bennie Lewis and Volodymyr Pryyma EEL 6883: Software Engineering II

2. Involves all life-cycle activities devoted to identification of user requirements, analysis of the requirements to derive additional requirements, documentation of the requirements as a specification, and validation of the documented requirements against user needs, as well as processes that support these activities. Requirements Engineering EEL6883 – Spr. 2008 – Lewis, Pryyma

3. Introduction • This paper presents research toward filling the gap between claims in the literature and Requirements Engineering practice. The authors conducted a 30-month explanatory case study at an organization that had revised its requirements engineering process and which provided them with the opportunity to assess the effects of improved Requirements Engineering over an entire project lifecycle. EEL6883 – Spr. 2008 – Lewis, Pryyma

4. Introduction cont. • Provides a detailed explanation of how improvements in RE practice can lead to improvements in productivity, quality, and risk management. EEL6883 – Spr. 2008 – Lewis, Pryyma

5. Related Work • Reviews the limited evidence that exists about RE payoffs. • Discusses expected benefits as a result of rigorous RE practice. EEL6883 – Spr. 2008 – Lewis, Pryyma

6. Related Work cont. • The paper reviewed the literature on RE and found a number of reports on REP improvement: • C. Claus, M. Freund, M. Kaiser, and R. Kneuper, “Implementing Systematic Requirements Management in a Large Software Development Programme,” • A.F. Hutchings and S.T. Knox, “Creating Products: Customers Demand,” • S. Jacobs, “Introducing Measurable Quality Requirements: A Case Study,” • M. Kauppinen and S. Kujala, “Starting Improvement of RequirementsEngineering Processes: An Experience Report,” • M. Kauppinen, S. Kujala, T. Aaltio, and L. Lehtola, “Introducing Requirements Engineering: How to Make a Cultural Change Happen in Practice,” • M. Kauppinen, M. Vartiainen, J. Kontio, S. Kujala, and R. Sulonen,“Implementing Requirements Engineering Processes throughout Organizations: Success Factors and Challenges,” • B. Regnell, P. Beremark, and O. Eklundh, “A Market-Driven Requirements Engineering Process—Results from an Industrial Process Improvement Programme,” • J.A. Villanlon, G.C. Augustin, T.G. San Feliu, and A.A. Seco, “Experiences in the Application of Software Process Improvement in SMEs,” EEL6883 – Spr. 2008 – Lewis, Pryyma

7. Related Work cont. • The conclusion that's drawn from these reports is that RE activities are bound tightly with other system engineering activities and that a complex interaction between REP and other processes in the organization may exist. As a result, it becomes difficult to measure the REP improvement benefits in isolation, particularly before a project has been completed. EEL6883 – Spr. 2008 – Lewis, Pryyma

8. Research Questions and Methodology • The paper describes the overall research question and the methodology that the authors used to develop an understanding of the relationship between improved RE processes and improvements in software development within the organization. EEL6883 – Spr. 2008 – Lewis, Pryyma

9. Discussed in this paper is information about a case study designed to examine the effects of RE at the Australian Center for Unisys Software (ACUS), an organization that improved its software processes with a focus on redefining their requirements engineering process. Intended to observe how RE affected software development at ACUS and explain how these effects were realized. Research Questions and Methodology cont. EEL6883 – Spr. 2008 – Lewis, Pryyma

10. Research Questions and Methodology cont. • The research was designed to answer the following questions: • How do improvements in the RE practice impact the early stages of development? • How do improvements in the RE practice impact the downstream development stages? • Which components of the RE process were more significant in contributing to this impact? • How could the interaction between REP and other processes have contributed to these results? EEL6883 – Spr. 2008 – Lewis, Pryyma

11. Research Questions and Methodology cont. EEL6883 – Spr. 2008 – Lewis, Pryyma

12. Research Questions and Methodology cont. • The authors identified a positive relationship between improvements in the RE process and other processes at ACUS and benefits in the early stages of development, in downstream development, and in post deployment software quality. EEL6883 – Spr. 2008 – Lewis, Pryyma

13. Research Context: The Company and its RE process improvement program • The paper provides background information on the organization where the authors conducted their research, the revised RE process, and the evidence collected in the earlier stages of research and which indicated that improvements in REP were related to gains in productivity, product quality, and risk management. EEL6883 – Spr. 2008 – Lewis, Pryyma

14. Research Context: The Company and its RE process improvement program In August 2001, ACUS engaged in a software process improvement initiative to reach CMM (Capability Maturity Model) Level 2. RE was identified as the primary key process area that needed improvement. the product development group had limited experience with formal requirements management processes. As a result, ACUS faced significant challenges. Projects at ACUS typically suffered from significant requirements creep, schedule overruns, and cost overruns. ACUS management had difficulty understanding the requested features and providing reasonably accurate development estimates. Managers at ACUS cited ineffective negotiations between ACUS and their marketing unit. Ineffective negotiations made it difficult to align development capacities and marketing needs. EEL6883 – Spr. 2008 – Lewis, Pryyma

15. Research Context: The Company and its RE process improvement program ACUS developed an RE process that sought to specifically address these challenges. The revised process defined a distinct, discrete phase of the project during which requirements would be elicited, analyzed, and negotiated. When this requirements phase had ended, requirements were to have been agreed on, committed to, and then baselined during project planning. Subsequent requirements changes would only be allowed after being approved in a formal requirements management process. EEL6883 – Spr. 2008 – Lewis, Pryyma

16. Research Context: The Company and its RE process improvement program • The new REP consisted of the following components: • Feature Decomposition • Requirements Traceability • Group Analysis Sessions • Cross Functional Teams • Structured Requirements • Testing According to Requirements EEL6883 – Spr. 2008 – Lewis, Pryyma

17. Summary of Observed REP Payoffs during Development EEL6883 – Spr. 2008 – Lewis, Pryyma

18. Payoffs of the RE Practice at ACUS • Rigorous REP was perceived as having provided a strong foundation for improvements in developer productivity, product quality, and risk management. • This evidence represents important background information on how these payoffs were realized in ACUS. EEL6883 – Spr. 2008 – Lewis, Pryyma

19. Data Collection and Analysis Methods • During the first two stages in the case study, one researcher was on site for 12 months of the project. • A questionnaire provided some initial input about the perceived effectiveness of the REP as well as its immediate benefits. • Additionally, change requests documents, project development estimation data, and entries within the requirements management tool were analyzed in the early stages of the project. EEL6883 – Spr. 2008 – Lewis, Pryyma

20. Evidence of RE Payoff in Early Stages • Perceived improvements in productivity, quality, and risk management were directly attributed to the revised REP. • Developers perceived that their ability to understand customer’s requests, ability to make more informed decisions, and quality of communication improved. • Data indicates a significant decrease in support requests and post-deployment defects. • 55% less defects reported when compared to previous projects. • Managers greatly increased their ability to manage risks. EEL6883 – Spr. 2008 – Lewis, Pryyma

21. The Need for Further Investigation • The first two stages of the project showed that the relationship between REP and its payoffs is much more complex than initially expected. • REP may have additionally interacted with other development processes at ACUS in leading to these payoffs. • Processes such as “cross-functional teams,” “sizing,” and “peer reviews” may have contributed to payoffs as well. • Payoffs may be the result of REP interacting with these other processes. EEL6883 – Spr. 2008 – Lewis, Pryyma

22. More In-Depth Analysis of REP • A total of 20 managers, team-leaders, and senior engineers from software engineering and product management departments were invited to participate in the third stage of the project. • Only 15 participated in the questionnaire. All of them were familiar with product history and the requirements process in place for the past 15 years. • Questionnaire was designed to determine the impact of the revised REP and to determine which component of the REP contributed to the impact most. EEL6883 – Spr. 2008 – Lewis, Pryyma

23. REP Impact on Other Processes REP Impact on Process Areas EEL6883 – Spr. 2008 – Lewis, Pryyma

24. REP Improvement and Impact REP Impact Component Responses for Highest-Scoring Sub-processes EEL6883 – Spr. 2008 – Lewis, Pryyma

25. RE Process and Expected Payoffs • Feature decomposition, sizing, and change management led to more accurate estimates. • Upfront test scenario definition, requirements validation, and peer-reviews led to improved feature coverage. • Enhanced feature understanding, change management, and project tracking led to managed requirements creep. • Change management and feature sizing led to effective project scope negotiation. EEL6883 – Spr. 2008 – Lewis, Pryyma

26. RE Process and Expected Payoffs • Traceability links, peer-reviews, and requirements validation led to fewer defects. • Common ground and cross-functional teams in feature development led to more effective communication. • Feature decomposition, specification conformance, and team reorganization led to reduced rework. EEL6883 – Spr. 2008 – Lewis, Pryyma

27. Implications for Research and Practice • Improvements in REP led not only to improved productivity, better quality, and effective risk management, but it also enhanced testing, peer-review, and project tracking processes. • This work should encourage future research to further understand the relationship between REP and improvements in other software engineering processes. • Future research topics may include development of instruments for objectively measuring interactions between REP and other processes. EEL6883 – Spr. 2008 – Lewis, Pryyma

28. Implications for Research and Practice • Certain REP improvements dominated in their contributions to other processes. • Collaboration emerged as a powerful theme in many of the improvements that were introduced in the revised REP. • For software practice, these insights provide practitioners with further incentives to adopt RE processes and offer more concrete guidance for such initiatives. EEL6883 – Spr. 2008 – Lewis, Pryyma

29. Strategies for Implementing RE Process • Iterative Feature Decomposition – iterative process that empowers both stakeholders to reach an agreeable outcome despite any geographical separation. • Frequent Negotiation Sessions – an effective way to overcome mistrust and power struggles. • Change Management – an approval process which takes effect when the project scope is agreed upon. • Early Test Scenarios – assures that final realization of requirements is considered early on in the project. • Cross-functional Requirements Analysis Sessions – establishes common understanding among groups. EEL6883 – Spr. 2008 – Lewis, Pryyma

30. Summary • The empirical study at ACUS has proven very beneficial in unveiling details that are much needed but are lacking in the current software engineering research and practice. • The study had to overcome the challenges of a lack of a well-defined theoretical base in the area of process interactions. EEL6883 – Spr. 2008 – Lewis, Pryyma

31. Personal Critique • RE Process description: • Very well defined goals • Detailed description of methodologies applied • Results clearly explained • Technical language: • Well written • Includes many Software Engineering specific terms EEL6883 – Spr. 2008 – Lewis, Pryyma

32. Paper Citation D. Damian and J. Chisan, “An Empirical Study of the Complex Relationships between Requirements Engineering Processes and Other Processes that Lead to Payoffs in Productivity, Quality, and Risk Management,” IEEE Transactions on Software Engineering, vol. 32, no. 7, Jul. 2006 EEL6883 – Spr. 2008 – Lewis, Pryyma

33. Q/A Any questions? EEL6883 – Spr. 2008 – Lewis, Pryyma