Effective Requirements Management for Successful Software Development

1. Διαχείριση αντικειμένου εργασιών & Καταγραφή απαιτήσεων Βασίλης X. Γερογιάννης, PhD

2. Laws of project management American Production and Inventory Control Society – Cyril Northcote Parkinson • No major project is ever installed on time, within budget or with the same staff that started it. Yours will not be the first • Projects progress quickly until they become 90% complete, then they remain at 90% complete forever • When things are going well, something will go wrong • When things just cannot get any worst, they will • When things appear to be going better, you have overlooked something • If project content is allowed to change freely, the rate of change will exceed the rate of progress • No system is ever completely debugged. Attempts to debug a system inevitably introduce new bugs that are even harder to find • A carelessly planned project will take three times longer to complete than expected. A carefully planned project will take only twice as long • Project teams detest progress reporting because it vividly manifests their lack of progress.

3. The Requirements Problem

8. Requirements Problem • What is the ultimate objective? • Quality SW • On time • Within budget • That meets users’ needs!! • Don’t lose sight of the goal!

9. The requirements problem • The goal of software development is to develop qualitysoftware—on time and on budget—that meetscustomers real needs. • Project success depends on good requirementsmanagement. • Requirements errors are the most common type ofsystems development error and the most costly to fix. • A few key skills can significantly reduce requirementserrors and thus improve software quality.

10. Major software development problems European Software Process Improvement Training Initiative (ESPITI) 2 largest problems are: • Requirement specs • Managing reqs Coding is rarely a major problem Though it is typically the biggest cost item!

11. Requirements Problem • Why do SW projects fail? Standish study (1994) reports that in US… • The 3 most commonly cited factors • 13% Lack of user input • 12% Incomplete reqs & specs • 12% Changing reqs & specs • All directly related to managing reqs!! • All related with communication issues!!

12. Requirements Problem • Why do projects succeed? • 16% User involvement • 14% Executive management support • 12% Clear requirements

15. Defect summary(Capers Jones - 1994) Delivered Defects = Defect Potentials x (1 - Removal Efficiency) • Reqs contribute most defects to delivered SW (Because they’re so hard to remove) • Design defects 2nd only to Reqs. defects • 56% of defects due to Reqs & Design!

16. Example Calculate (pre-delivery) Defects Removal Efficiency?

17. Example (pre-delivery) DRE = (250 / 300) x 100 = 83,3%

18. Defect Removal Effectiveness • How defects are injected and removed during software development • How to measure defect removal effectiveness at the different stages of software development

19. Defect Removal Effectiveness Background • Defect removal effectiveness is important because: • It improves the quality of the software produced • e.g. less defects are left in the shipped release • It improves the productivity and scheduling of the development process • e.g it ensures defects are found early and less time is spent fixing mistakes found late in the process • It helps to reduce the cost of development • e.g. if productivity improves and scheduling does not fall behind then costs go down.

20. Relative Cost to Repair

21. Defect Injection and Removal

22. Defects found by removal operation ________________________________ X 100 Removal Effectiveness = Defects present at removal operation Defect Removal Effectiveness • Defects present at removal operation • Defects found during removal operation + defects found later • Defects found later may be determined ex post facto (after the fact), or using a statistical prediction model • Since the latent defects in a software product is unknown at any point in time, it is approximated by adding the number of defects removed during the phase to the number of defects found later (but that existed during that phase).

23. Defect Injection and Removal

24. Defects Injected (new mistakes) Undetected Defects Net Defects Defect Detection (inspection) Life Cycle Phase Unfixed Defects + Bad Fix Defects NetDefectsfrom previous Phase Known Defects Bad Fix Defects Fixed Defects Defect Injection & Removal Model Applies to each life cycle phase

25. Defect Injection and Removal (similar model)

26. Defect Removal Effectiveness • Defect removal effectiveness = (# of defects found by inspection) /(# of defects originally present) *100

27. Defect Matrix Assumptions • Defects are removed in the same life cycle phase when they are found • No defects are knowingly left unfixed • No bad fixes – or at least they are blended into the number of defects created in that life cycle phase

28. Sample Defect Matrix—When Originated vs. When Found W H E N F O U N D / F I X E D WHEN ORIGINATED (INJECTED OR CREATED) Calculate DRE for I0, I1, UT etc. ?

29. Defects Created this Phase Defects passed from Previous life cycle phases Defects passed to Next life cycle phase Life cycle Phase(s) Defects Found & removed this Phase Defect Removal Effectiveness Next = (Previous + Created) – FoundDRE = Found / (Previous + Created) * 100

30. Sample Defect Matrix—When Originated vs. When Found DRE for I0 W H E N F O U N D / F I X E D WHEN ORIGINATED (INJECTED OR CREATED) Defects found and removed at I0 = 730 Defects existing on step entry (escapes from Reqs) = 122 Defects injected in current phase (I0) = 859

31. High Level Design Effectiveness • High (Top) Level Design (I0) Inspection Effectiveness • Defects found and removed at I0: 730 • Defects existing on step entry (escapes from requirements phase: 122 • Defects injected in current phase: 859 • E(I0) = 730/(122+859) x 100 = 74%

32. Sample Defect Matrix—When Originated vs. When Found DRE for I1 W H E N F O U N D / F I X E D WHEN ORIGINATED (INJECTED OR CREATED) Defects found and removed at I1 = 729 Defects existing on step entry (escapes from Reqs & I0) = 122 + 859 – 730 = 251 Defects injected in current phase = 939

33. Low Level Design Effectiveness • Low Level Design (I1) Inspection Effectiveness • Defects found and removed at I1: 729 • Defects existing on step entry (escapes from requirements phase and I0): 122+859-730 = 251 • Defects injected in current phase: 939 • E(I1) = 729/(251+939) x 100 = 61%

34. Code Inspection Effectiveness • Code Inspection (I2) Effectiveness • Defects found and removed at I2: 1095 • Defects present on step entry (escapes from requirements phase, I0, and I1): 122+859+939-730-729 = 461 • Defects injected in current phase: 1537 • E(I2)= 1095/(461+1537) x 100 = 55%

35. Unit Testing Effectiveness • Unit Testing (UT) Effectiveness • Defects found and removed at UT: 332 • Defects existing on step entry (escapes from previous phases): 122+859+939+1537-730-729-1095 = 903 • Defects injected in current phase (bad fixes): 2 • E(UT) = 332/(903+2) x 100 = 37%

36. Summary Effectiveness Measures • Overall Design & Coding Inspection Effectiveness • IE = (730+729+1095)/(122+859+939+1537) x 100 = 74% • Overall Effectiveness of all Testing activities • TE = (332+387+111)/(903+2+4+1)x100 = 91% • Overall Defect Removal Effectiveness of the development process • DRE = (0+730+729+1095+332+387+111)/(122+859+ 939+1537+2+4+1) x 100 = 3384/3464*100 = 97.7%

37. Phase Detected Requirements Design Coding/Unit Test Requirements 10 - - Design 3 18 - Coding 0 4 26 Test 2 5 8 Field 1 2 7 For example, assume that the following table reflects the defects detected during the specified phases and the phase where those defects were introduced. WHEN ORIGINATED (INJECTED OR CREATED) W H E N F O U N D / F I X E D

38. The Defect Removal Effectiveness for each of the phases would be as follows: Requirements DRE = 10 / (10+3+0+2+1) x 100% = 63% Design DRE = (3+18) / (3+0+2+1+18+4+5+2) x 100% = 60% Coding DRE = (0+4+26) / (0+2+1+4+5+2+26+8+7) x 100% = 55% Testing DRE = (2+5+8) / (2+1+5+2+8+7) x 100% = 60% Defect Removal Effectiveness can also be calculated for the entire development cycle to examine defect detection efforts before the product is released to the field. According to Capers Jones, world class organizations have Development DRE greater than 95%. Development DRE = (Pre-release Defect) / (Total Defects) x 100% = (10+3+2+18+4+5+26+8) / (10+3+2+1+18+4+5+2+26+8+7) x 100 = 88%

39. Requirements management • A requirement is a capability that the system must deliver. • A software capability needed by the user to solve a problem to achieve an objective • A software capability that must be met or possessed by a system or system component to satisfy a contract, standard, specification, or other formally imposed documentation • Requirements management is a process of systematically eliciting, organizing, and documenting requirements for a complex system. • Our problem is to understand users‘ problems in their culture and their language and to build systems that meet their needs. • A feature is a service that the system provides to fulfill one or more stakeholder needs. • A use case describes a sequence of actions, performed by a system, that yields a result of value to a user.

40. Requirements Management (REQM) Focuses on basic fundamental activities of documentation and traceability Builds dialogue and accountability Manages changes to requirements and inconsistencies between requirements, work products and plans Requirements Management vs Development • Requirements Development (RD) • Focuses on discoveryof customer needs • Internal / external • Transforms the “voice of the customer” into product qualities or service functions and levels, and an understanding of what the customer will gain • Validates (based on evidence or sound footing) the requirements before going into production or servicing the customer • Usually comes first in the lifecycle Documentation & Traceability REQM Valid Requirements Management, Engineering Product & Service Solutions RD Alternativesolutions

41. Why we need to manage requirements? • Small software product has 1000 requirements • Boeing 777 has to satisfy 300,000 requirements • The questions are: • Which project team members are responsible for the wind speed requirement (#278), and which ones are allowed to modify it or delete it? • If requirement #278 is modified, what other requirements will be affected? • How can we be sure that someone has written the code in a software system to fulfill requirement #278, and which test cases in the overall test suite are intended to verify that therequirements have indeed been fulfilled?

42. Requirements management • The process of managing change to the requirements for a system • The principal concerns of requirements management are: • Managing changes to agreed requirements • Managing the relationships between requirements • Managing the dependencies between the requirements document and other documents produced in the systems engineering process • Requirements cannot be managed effectively without requirements traceability. • A requirement is traceable if you can discover who suggested the requirement, why the requirement exists, what requirements are related to it and how that requirement relates to other information such as systems designs, implementations and user documentation.

43. CASE tools for requirements management • Requirements management involves the collection, storage and maintenance of large amounts of information • There are now a number of CASE tools available which are specifically designed to support requirements management • Other CASE tools such as configuration management systems may be adapted for requirements engineering

44. Requirements management tool support • A database system for storing requirements. • Document analysis and generation facilities to help construct a requirements database and to help create requirements documents. • Change management facilities which help ensure that changes are properly assessed and costed. • Traceability facilities which help requirements engineers find dependencies between system requirements.

45. A requirements management system

46. Stable and volatile requirements • Requirements changes occur while the requirements are being elicited, analysed and validated and after the system has gone into service • Some requirements are usually more subject to change than others • Stable requirements are concerned with the essence of a system and its application domain. They change more slowly than volatile requirements (e.g. a hospital will always have doctors, nurses, etc.). May be derived from domain models) • Volatile requirements are specific to the instantiation of the system in a particular environment and for a particular customer (in a hospital, requirements derived from health-care policy)

47. Requirements change factors • Requirements errors, conflicts and inconsistencies • As requirements are analysed and implemented, errors and inconsistencies emerge and must be corrected. These may be discovered during requirements analysis and validation or later in the development process. • Evolving customer/end-user knowledge of the system • As requirements are developed, customers and end-users develop a better understanding of what they really require from a system. • Technical, schedule or cost problems • Problems may be encountered in implementing a requirement. It may be too expensive or take too long to implement certain requirements.

48. Requirements change factors • Changing customer priorities • Customer priorities change during system development as a result of a changing business environment, the emergence of new competitors, staff changes, etc. • Environmental changes • The environment in which the system is to be installed may change so that the system requirements have to change to maintain compatibility • Organisational changes • The organisation which intends to use the system may change its structure and processes resulting in new system requirements

49. Types of volatile requirement • Mutable requirements • These are requirements which change because of changes to the environment in which the system is operating • Emergent requirements • These are requirements which cannot be completely defined when the system is specified but which emerge as the system is designed and implemented • Consequential requirements • These are requirements which are based on assumptions about how the system will be used. When the system is put into use, some of these assumptions will be wrong. • Compatibility requirements • These are requirements which depend on other equipment or processes.

50. Requirements identification • It is essential for requirements management that every requirement should have a unique identification • The most common approach is requirements numbering based on chapter/section in the requirements document • Problems with this are: • Numbers cannot be unambiguously assigned until the document is complete • Assigning chapter/section numbers is an implicit classification of the requirement. This can mislead readers of the document into thinking that the most important relationships are with requirements in the same section