SE604 Software Testing and QA Lecture #4: SQA Defect Removal Effectiveness and Cost

1. SE604Software Testing and QALecture #4: SQA Defect Removal Effectiveness and Cost Slides by Mohammad El-Ramly, PhD Readings: Galin, Ch 7 http://www.acadox.com/class/48453 MRABFH

2. حديث شريف • إن الله يحب إذا عمل أحدكم عملا أن يتقنه • إن الله كتب الإحسان على كل شئ

3. Course Organization • Fundamentals of SW Quality Assurance 4~5 wks 2. Software Vulnerabilities 1~2 wks Guest 3- Software Testing 4~5 wks Guest / Me 4- Presentations 1~2 wks Students

4. Assessment • 40% Coursework A1= 5 + A2 = 7 + A3 ~= 8 A4, A5, A6 • 10% Midterm (Mid Nov.) • 50% Final

5. Readings so far … • Code Complete, Chapter 20 • Software QA from Theory to Practice, Chapters 1, 2, 5, 6, 7 • Laurie Williams, Reviews and Pair Programming, 2004 • Article on how much of the world runs on bad SW. Readingsare essentialfor this course.

6. Assessments so far …. • A1 on yourself and QA as applied in .companies in Egypt. • A2 on contract writing and review. • A3 on code review

7. Change and Configuration Management Software Quality Landscape Reviews Collaborative Development / Reviews Requirements V & V Guidelines, Standards and Templates Design Review Bug Tracking Surveys Contract Review QA Planning and Budgeting Testing Metrics Secure Coding Debugging Refactoring Deployment & Maintenance Planning Design Testing Requirements Development M a n a g e m e n t

8. Change and Configuration Management Software Quality Landscape Reviews Collaborative Development / Reviews Requirements V & V Guidelines, Standards and Templates Design Review Bug Tracking Surveys Contract Review QA Planning and Budgeting Testing Metrics Secure Coding Secure Coding Debugging Refactoring Deployment & Maintenance Planning Design Testing Requirements Development M a n a g e m e n t

9. Last Time6.3 Elements of QA Plan

10. Last Time6A Risks Potential Risks • (1) Scheduling and timing risks • (2) System functionality risks • (3) Subcontracting risks • (4) Requirements management risks • (5) Resource usage and performance risks • (6) Personnel management risks.

11. Lecture 4 OutlineIntegrating Quality Activities in Project Lifecycle • Factors affecting QA intensity • Effectiveness of defect removal techniques

12. 7.2 Factors affecting intensity ofquality assurance activities • Software Development Process SDP (also sometimes called SDLC) determines: who does what, when to produce whichartifact. • QA planners need to determine: • List of all QA activities • What is done, when, by whom to produce which artifact • Resources and duration.

13. 7.2 Factors affecting intensity ofquality assurance activities • It should be noted that various bodies may participate in the performance of quality assurance activities: 1- Development team 2- Managers and team leaders 3- Independent bodies such as external quality assurance team members or consultants.

14. 7.2 Factors affecting intensity ofquality assurance activities

16. Ex1: Consumer Club • Consumer Club for furniture company • Team did 11 similar projects • 7 man-months – 2 people x 4 months • 90% of code is reused for earlier projects What do you suggest as QA activities for this project ?

17. Ex1: Consumer Club • Team acquaintance with the subject • High percentage of software reuse • Size of the project (in this case, medium) • Severity of failure results.

18. Ex2: Patient Monitoring System • Real-time monitoring of patients • Patient's room sub-system will interface with multiple medical devices by multiple vendors to read patient’s health indicators. • Control room sub-system will be in the nurses station to monitor patients’ conditions. • Data is fed to mobile devices of doctors. • Estimate 40 mm = 5 people x 14 months • Only 15% of code is reused. What do you suggest as QA activities for this project ?

19. Ex2: Patient Monitoring System • High complexity and difficulty of the system • Low percentage of reusable software available • Large size of the project • High severity of failure outcomes if the project fails.

20. Ex2: Patient Monitoring System 30 38

22. 7.4 Model for SQA defect removaleffectiveness and cost • The SQA plan’s total effectiveness in removing project defects • The total costs of removal of project defects

23. 7.4.1 Data – Defect Origin Distribution

24. 7.4.1 Data – Effectiveness of QA Activities in Defect Removal

25. 7.4.1 Data – Cost of Defect Removal

26. Assumption of Defect Removal Model • The model deals with two quantitative aspects of an SQA plan designed for a specific project: (1) Total effectiveness of defect removal. (2) Total cost of defect removal.

27. Assumption of Defect Removal Model • The model is based on the assumptions: • The development process is linear and sequential (the waterfall model). • A number of “new” defects are introduced in each development phase. • Various review and testing activities serve as filters, removing % of the entering defects while passing the rest the next activity. • Incoming defects are the sum of defects from the former QA together with “new” defects created in the current development phase.

28. Model Terminology

29. Model Application

30. Ex 1: Standard QA Plan

31. Ex 1: Standard QA Plan – Cost of Removing 100 defects From Code

32. Ex 1: Standard QA Plan – Cost of Removing 100 defects

33. Ex 2: Extensive QA Plan

34. Defects removal effectivenessfor quality assurance plans

35. Ex 2: Extensive QA Plan

36. Ex 2: Extensive QA Plan

37. Ex1 vs. Ex2