1 / 24

Agenda

Significance of depth of inspection and inspection performance metrics for consistent defect management in software industry submitted to Professor Shervin Shirmohammadi in partial fulfillment of the requirements for the course ELG 5100 Yongkeng Guan 6548060 Ning Zhang 6533637.

clive
Download Presentation

Agenda

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Significance of depth of inspection and inspection performance metrics for consistent defect management in software industrysubmitted to Professor ShervinShirmohammadi in partial fulfillment of the requirements for the course ELG 5100Yongkeng Guan 6548060 Ning Zhang 6533637

  2. Agenda • Introduction • Background • DI and IPM • Case Study • Application of DI and IPM • Limitation and improvement • Conclusion

  3. Introduction How to performeffective defect management? Inspection Testing What those methods depend on? Process People quality

  4. Why software industry needs analytical modes in defect management? Problem: The resource estimation for defect management in terms of required process policies and people effort is intuitive or controlled by time and budget How to solve: analytical modes 1. DI: Depth of Inspection 2. IPM: Inspection Performance Management

  5. Background Other popular metrics Defect Removal Efficiency(DRE) Defect density Inspection rate Defect finding efficiency

  6. How to compute DI (Depth of Inspection)? DI evaluation is realized in two phases: 1: DI is calculated using shop floor count for a particular set of projects 2: Prediction of DI for a new project using Multiple Linear Regression(MLR)(A minimum of five old projects is required to evaluate the process coefficients)

  7. Phase 1: DI is calculated using shop floor defect count Ni: ＃ of defects captured by inspection process Td:＃ of defects captured by both inspection and testing approaches

  8. Phase 2: Prediction of DI x1=inspection time x2=preparation time x3=number of inspectors x4=experience level of inspector β0 to β4:process coefficients Phase 1

  9. How to compute IPM? Inspection Performance Metric(IPM) where Inspection effort(IE)=Total number of inspector(n) × Total amount of inspection times(T) ※T includes inspection time and preparation times.

  10. Prediction of IPM x1=inspection time x2=preparation time x3=number of inspectors x4=experience level of inspector x5=the complexity of the project measured using function point analysis in a logarithmic scale β0 to β5:team coefficients

  11. Case study Objective Illustrating the inconsistent and unpredictable defect capturing ability of the inspection team in several projects, thereby to indicate the importance of introducing DI and IPM metrics in inspection process. What to be analyzed defect capturing ability of the inspection team within the inspection influencing parameters such as • inspection time • inspection preparation time • number of inspectors • experience level of inspectors • complexity of the project

  12. Case study Object of study: The company under study is a CMMI 5 certified leading product-based software industry, which functions with business application projects. data sources: data sets for analysis purpose were collected from the empirical projects (from 2000 to 2010), which were available in the defect recording centers, and from quality assurance department of the company. To set the boundary of the research Hypothesis 1: Empirical projects under study are similar in nature of application and programming environment Hypothesis 2: Empirical projects are categorized as either medium or large sized project based on the complexity of the project. Hypothesis 3: DI and IPM are applicable as quality estimation and prediction metrics for the software projects.

  13. Case study Analysis on requirements phase large size projects Medium size projects

  14. Case study Analysis on requirements phase Fig.1 Quantitative analysis of defect capturing abilities by inspection team at requirements phase

  15. Case study Analysis on design phase Medium size projects large size projects

  16. Case study Analysis on design phase Fig. 2 Quantitative analysis of defect capturing abilities by inspection and test team at design phase

  17. Case study Analysis on implementation phase Medium size projects large size projects

  18. Case study Analysis on implementation phase Fig. 3 Quantitative analysis of defect capturing abilities by inspection and team at implementation phase

  19. Case study • Results and inferences • Defect management still in intuitive mode • inconsistent defect capturing ability of the inspection team • The defect capturing ability of the inspection team is weaker than estimated • High pressure for the test team intuitive mode analytical mode

  20. Application • How DI and IPM help to achieve a reliable defect capturing capability • Analytical approach, which is predictable and measurable • DI, a process metric, to quantify the depth in which the inspection process is performed • IPM, a people metric, for the inspection-performing team to be aware of their performance • Desire defect capturing ability can be achieved by choosing appropriate values of parameters • Transparency and visibility to the manager and customers • Test team formulates proper strategies to find the remaining defects • Continual process improvement

  21. Limitation • Only applicable in projects with similar functions and programming environment • Need for previous sample projects to find the coefficients • Estimation accuracy may be affected when team members change • Suggestion for improvement • Take performance of the individual into account to attain more precise estimation and reliability

  22. Conclusion • Introduce DI and IPM • Case study • How DI and IPM act on defect management • Limitation and improvement

  23. References [1]. T.R. Gopalakrishnan Nair V. Suma P. Kumar Tiwari, ‘Significance of depth of inspection and inspection performance metrics for consistent defect management in software industry’, Software, IET (Volume:6 , Issue: 6 ), Dec 2012 [2]. Gopalakrishnan Nair, T.R., Suma, V.: ‘Defect management using pair metrics, DI and IPM’, CrossTalk, J. Def. Softw. Eng., 2011, 24, (6), pp. 22–27 [3]. GopalakrishnanNair, T.R., Suma, V., Nithya, G.N.: ‘Estimation of characteristics of a software team for implementing effective inspection process through inspection performance metric’, Am. Soc. Qual., Softw. Qual. Prof. J., 2011, 13, (2), pp. 14–26 [4]. Stewart, R., Priven, L.: ‘Management’s inspection responsibilities and tools for success’, CROSSTALK J. Def. Softw. Eng., 2009, 22, (3), pp. 18–21

  24. Questions

More Related