Presentation of the Quantitative Software Engineering (QuaSE) Lab, University of Alberta

1. Honolulu, October 8, 2000 Presentation of theQuantitative Software Engineering (QuaSE) Lab, University of Alberta Giancarlo Succi Department of Electrical and Computer Engineering University of Alberta ISERN 2000 Meeting

2. The group • James Miller • Petr Musilek • Marek Reformat • Witold Pedrycz • Giancarlo Succi • 2 Visiting profs • 1 PDF • 12 Graduate students G. Succi – QuaSE Lab – UoA

3. Research interests “Application of quantitative methods to software engineering” • Software metrics (definitions and tools) • Advanced (Statistical and CI) models • Sensitivity of cost models • Certification of components and analysis of product lines • Inspections • Analysis of the nature of flexible methodologies • … G. Succi – QuaSE Lab – UoA

4. Sponsors • The University of Alberta • The Alberta government • ASRA • NSERC • Nortel • CFI • WaveRider • Valmet G. Succi – QuaSE Lab – UoA

5. Brief overview of 2 projects • Analysis of the ability of predicting defects using OO metrics • Study of the occurrences of software service requests G. Succi – QuaSE Lab – UoA

6. Analysis of the ability of predicting defects using OO metrics • Investigate and quantify the impact of the object-oriented design on the defect-proneness of classes • Empirically validate the ability of the object-oriented design metrics to identify classes with high number defects in commercial software applications • Build and evaluate explanatory statistical models applicable for the count data G. Succi – QuaSE Lab – UoA

7. The adopted measures • Object-oriented design metrics(Chidamber and Kemerer, 1991) • Depth of Inheritance Tree (DIT) • Number Of Children (NOC) • Response For a Class (RFC) • Coupling Between Objects (CBO) • Weighted Methods per Class (WMC/NOM) • Lack of Cohesion in Methods (LCOM) • Our tool: WebMetrics - metrics collection system • Dependent variable – number of defects for a class G. Succi – QuaSE Lab – UoA

8. Issues in the Statistical Analysis • Distribution of the dependent variable? • Count data • Poisson Regression • Equidispersion • Negative Binomial Regression • Gamma-distributed mean • Underprediction of zero values • Zero-inflated Negative Binomial regression • Two processes – different distributions G. Succi – QuaSE Lab – UoA

9. Building the models • Ordinal Least Squares (OLS) vs. Maximum Likelihood (ML) • ML - general solution for fitting model parameters • Consistency: the probability that the ML estimator differs from the true parameter by an arbitrary small amount tends toward zero as the sample size grows • Asymptotic efficiency: The variance of the ML estimator is the smallest possible • Selection of predictors • Stepwise regression based on the statistical significance • Resulting models • Univariate: RFC • Bivariate: RFC and DIT G. Succi – QuaSE Lab – UoA

10. Comparison of the models • Goodness of fit and overdispersion • Criticality prediction - Alberg diagram G. Succi – QuaSE Lab – UoA

11. Study of the occurrences of software service requests • Service Request • demand for a modification of the software system behavior • (early) life-cycle process attribute measure • Our goal: To define and validate a framework for pre-release SR analysis on three industrial datasets: • Predict effort, resources, and time to be allocated for a project • Predict the final number of SRs for a project • Provide a basis for comparisonand assessment of different projects and development processes G. Succi – QuaSE Lab – UoA

12. SRs and Reliability • Reliability • failure-centric quality measure that views the software system as a whole from a customer perspective • Software Reliability Growth Models (SRGM) • models describing failure detection over time • using calendar time, the number of tests run, or execution time • The evaluated models: • GO S-Shaped • Goel-Okumoto • Gompertz • Hossain-Dahiya/GO • Logistic • Weibull • Weibull S-shaped • Yamada Exponential • Yamada Raleigh G. Succi – QuaSE Lab – UoA

13. Criteria • Goodness of fit • Accuracy of the final point • Relative precision of fit • Coverage of fit • Predictive ability G. Succi – QuaSE Lab – UoA

14. Summary of Results G. Succi – QuaSE Lab – UoA

15. SRGM Sensitivity to Noise • How sensitive are the models to the human factor in the SRs data recording? • Monte Carlo analysis with normally distributed noise N(0,) added to the original data G. Succi – QuaSE Lab – UoA

16. Response Time and Gamma Analysis • When compared with other models, linear regression has the best performance G. Succi – QuaSE Lab – UoA

17. Future research plans • Extend the analysis of software models with “more advanced” techniques • Perform domain-specific studies, such as in product lines and extreme programming • Analyse the evolution of the software market to determine driving forces of software development processes • … G. Succi – QuaSE Lab – UoA

18. Netcraft survey: web server usage over the Internet G. Succi – QuaSE Lab – UoA

19. Expectation from ISERN • In- and out- flow of ideas • Sharing of experimental data and experimental protocols • Exchange of visits • Partnerships in projects (and funding proposals when possible) • … G. Succi – QuaSE Lab – UoA