Quality prediction model for object oriented software using UML metrics

1. Quality prediction model for object oriented software using UML metrics Ana Erika Camargo, Koichiro Ochimizu Japan Advanced Institute of Science and Technology 4th World Congress for Software Quality – Bethesda, Maryland, USA – September 2008

2. Outline • Objective • Scope • Our Approach • Related Work • Design complexity metrics and UML • Prediction Technique • Case Study • Conclusions and Future work

3. Objective To create amodel which is able : • To predictfault-prone* code in early phases of the life cycle of the software • To detect possible defects in the software (*) Fault-prone code: Code capable of having bugs.

4. : Variables S : Change in the Same Direction O : Change in the Opposite Direction : Scope of this study Scope Causal-Loop Diagram Fault-pronecode S S S Wrong Implementation S Complex Implementation ComplexSpecifications Complex Design O S O O O Wrong Design S Misunderstanding of Requirements Developers'experience Designers'experience O

5. Our Approach *Related existing works Predict Fault-prone code Design Complexity Metrics FROM: Code‏ Approximation: • To obtain good candidates of fault-proneness prediction • Predict before coding Design Complexity Metrics FROM: UML Artifacts

6. Related work: Fault prediction Prediction models of fault-proneness: CK : Chidamber & Kemerer, QMOOD: Quality Metrics for Object Oriented Design

7. Related work: Fault prediction From these studies, we identified useful metrics to predict fault-proneness of code : • Chidamber and Kemerer – CK • Depth of inheritance tree (DIT) • Number of children (NOC) • Weighted Methods per Class (WMC) • Coupling Between objects (CBO) • Response for class (RFC) • Lack of Cohesion of methods (LCOM) • Bansiyana and Davi's quality metrics - QMOOD • Average of DIT for all classes in the system (ANA) • Class Interface Size (CIS) • Data Access Metric (DAM) • Direct Class Coupling (DCC) • Measure of aggregation (MOA) • Measure of functionality abstraction (MFA) • Number of methods (NOM - same as WMC)

8. Related work: UML & Design Complexity Metrics • Tang et. al[2002]: Measures CK metrics from data structures , which are created from Rational Rose class, collaboration and activity diagrams. Issue: • To obtain accurate measures, assumptions are made, related to the level of details in the diagrams. For example: one activity diagram per operation in the system is required

9. Related work: UML & Design Complexity Metrics • Baroni [2002]: formal definition of CK and QMOOD metrics, among others. This work uses UML class diagrams. Issues: • RFC, LCOM calculations are code dependent • CBO calculation, does not have a clear inclusion of methods used or variables instantiated of different classes, within every method of a class.

10. UML & Design Complexity Metrics *Related existing works Predict Fault-prone code Design Complexity Metrics FROM: Code‏ Approximation: • To obtain good candidates of fault-proneness prediction • Predict before coding Design Complexity Metrics FROM: UML Artifacts

11. UML & Design Complexity Metrics Design complexity metrics that can be approximated using UML class diagrams: • Chidamber and Kemerer – CK • Weighted Methods per Class (WMC) • Depth of inheritance tree (DIT) • Number of children (NOC) • Coupling Between objects (CBO) • Response for class (RFC) • Lack of Cohesion of methods (LCOM) • Bansiyana and Davi's quality metrics - QMOOD • Average of DIT for all classes in the system (ANA) • Class Interface Size (CIS) • Data Access Metric (DAM) • Direct Class Coupling (DCC) • Measure of aggregation (MOA) • Measure of functionality abstraction (MFA) • Number of methods (NOM - same as WMC) • Can be obtained straightforward from CLASS Diagrams • Cannot be calculated precisely from CLASS Diagrams. Implementation of the bodies of the classes is needed.

12. UML & Design Complexity Metrics CBO Approximation • CBO-code: Num. Classes Couple to a given Class * • CBO-UML Approach 1 (UML Collaboration Diagram): A count of all messages Sent to different objects • CBO-UML Approach 2 (UML Collaboration Diagram): The same as Approach 1, but eliminating those which RETURN a value. (*) If a method within a class uses a method or instance of a variable of a different class, it is said that this pair of classes is coupled

13. aCustomer UML & Design Complexity Metrics CBO Approximation R7: fundsStatus : = CommtiFunds()

14. UML & Design Complexity Metrics CBO Evaluation using an e-commerce system (*). (*) Described in: Gomaa Hassan, Designing Concurrent, Distributed, and Real-Time Applications with UML, Addison Wesley-Object Technology Series Editors, July 2000.

15. UML & Design Complexity Metrics CBO Evaluation • For CBO-code and CBO-UML Approach 1 correlation coefficient = 0.81 • For CBO-code and CBO-UML Approach 2 correlation coefficient = 0.89 • CBO-UML Approach 2 is slightly more linear to CBO-code

16. UML & Design Complexity Metrics RFC Approximation • RFC-code: Num. of Methods of a given class + Num. of methods of other classes directly called by any of the methods of the given class. • RFC-UML Approach 1 (UML Collaboration Diagrams): Messages Received + Messages Sent • RFC-UML Approach 2 (UML Collaboration & Class Diagrams): (Messages Received + Number of attributes*2) + Messages Sent, where: (Messages Received + Number of attributes*2) ~ Num. of Methods of a given class. Considering 2 public methods per attribute to get and to set its value.

17. UML & Design Complexity Metrics RFC Approximation class C { A a; void m() { D d ; d.dosth(); …….. } void setA (A a) { this.a = a; } A getA() { return a; } } dosth() d c m() x RFC (C) = 3 + 1 = 4

18. UML & Design Complexity Metrics RFC Evaluation using the same e-commerce system.

19. UML & Design Complexity Metrics RFC Evaluation • For RFC-Code and RFC-UML Approach 1 correlation coefficient = -0.07 • For RFC-Code and RFC-UML Approach 2 correlation coefficient = 0.67 • RFC-UML Approach 2 has a stronger linear relationship with RFC-Code

20. UML & Design Complexity Metrics Remark If true that our 2nd approach’s assumption might not be all valid, it still obtained an acceptable performance. Which might be explained to the fact that private attributes in a class are moderate correlated to its number of methods, according to Olague’s research [2007].

21. UML & Design Complexity Metrics Design complexity metrics that can be approximated using UML diagrams: • Chidamber and Kemerer – CK • Weighted Methods per Class (WMC) • Depth of inheritance tree (DIT) • Number of children (NOC) • Coupling Between objects (CBO) • Response for class (RFC) • Lack of Cohesion of methods (LCOM) • Bansiyana and Davi's quality metrics - QMOOD • Average of DIT for all classes in the system (ANA) • Class Interface Size (CIS) • Data Access Metric (DAM) • Direct Class Coupling (DCC) • Measure of aggregation (MOA) • Measure of functionality abstraction (MFA) • Number of methods (NOM - same as WMC) • Can be obtained straightforward from CLASS Diagrams • Can be approximated by using COLLABORATION Diagrams • Can not be approximated precisely using UML Diagrams

22. Fault-prone code Predict : • How? Design Complexity Metrics (12) FROM: UML Artifacts Prediction Technique Related existing works Design Complexity Metrics (13) Predict FROM: Code‏ Approximation

23. Prediction Technique Logistic Regression • Use. When we have one variable (y) with two values (e.g. faulty /no faulty, 1/0) and one or moremeasurement variables (xs). • Goal. To predict the probability of getting a particular value of y , given xs variables, through a logit model. • Key Points. No assumptions on the distribution of variables are made.

24. Logistic Regression Prediction Technique

25. Prediction Technique Example. We want to estimate the probability of a class to be highly FAULTY, in terms of a design complexity metric: Mx.

26. Prediction Technique Faulty: Most Faulty (MF) = 1 Least Faulty (LF) = 2 Design complexity Metric: Mx CLASS FAULTY MxCLASS FAULTY Mx ---------------------------------------- --------------------------------------------- • 1 1 13 2 1 • 1 1 14 2 0 • 1 1 15 2 0 • 1 1 16 2 0 • 1 1 17 2 0 • 1 1 18 2 0 • 1 1 19 2 0 • 1 1 20 2 0 • 1 1 21 2 0 • 1 1 22 2 0 • 1 0 23 2 0 • 1 0 24 2 0 CLASS Mx=1 Mx=0 Total -------------------------------------------- MF=1 10 2 12 LF=2 1 11 12 -------------------------------------------- Total 11 13 24

27. Prediction Technique CLASS Mx=1 Mx=0 Total -------------------------------------------- MF 10 2 12 LF 1 11 12 -------------------------------------------- Total 11 13 24 Probabilities • The probability of any given CLASS will be MF: P(MF) = 12 /24 = 0.50 • The probability of any given CLASS will be MF given that Mx=1: P(MF|Mx=1) = 10/11= 0.909 • The probability of any given CLASS will be MF given that Mx=0: P(MF|Mx=0) = 2/13= 0.154

28. Prediction Technique CLASS Mx=1 Mx=0 Total -------------------------------------------- MF 10 2 12 LF 1 11 12 -------------------------------------------- Total 11 13 24 Odds • The odds of a CLASS being MF: Odds(MF) = 12 /12 = 1 • The odds of a CLASS being MF given that Mx=1 : Odds(MF| Mx=1) = 10/1= 10 …. (1) • The odds of a CLASS being MF given that Mx=0 : Odds(MF| Mx=0) = 2/11= 0.182 … (2)

29. Prediction Technique • Odds and Probabilities provide the same information but in different ways. • It is easy to convert odds y probabilities and vice-versa, e.g. : 10 = 0.909 odds (MF| Mx=1) = P(MF| Mx=1) = 1+10 1 + odds (MF| Mx=1) 0.909 = 10 P (MF| Mx=1) = Odds(MF| Mx=1) = 1-0.909 1 - P (MF| Mx=1)

30. Prediction Technique • Applying the natural log of (1) and (2) : ln [ Odds(MF|Mx=1) ] = ln ( 10 ) = 2.303 …………(3) ln [ Odds(MF|Mx=0) ] = ln (0.182) = -1.704 ………(4) • We can generalize (3) and (4) in the following: ln[ Odds(MF|Mx) ] = A + B*Mx ………..(5) • From (3) and (5), when Mx = 1: ln[ Odds(MF|Mx) ] = A + B = 2.303 ….(6) • From (4) and (5), when Mx=0: ln[ Odds(MF|Mx) ] = A = -1.704 ……..(7) • From (6) and (7): A = -1.704 , B = 4.007 • Finally we can re-write (5) as follows: ln[ Odds(MF|Mx) ] = -1.704 + 4.007 *Mx

31. p Odds(MF|Mx) = ; p = P (MF|Mx) 1 - p ln [ ] = -1.704 + 4.007 *Mx 1 - p Prediction Technique ln[ Odds(MF|Mx) ] = -1.704 + 4.007 *Mx • If: • We can re-write our final equations as: p 1 (1+e-(-1.704+4.007Mx) ) p = P (MF|Mx) =

32. Fault-prone code Design Complexity Metrics (12) FROM: UML Artifacts Case study Related existing works Design Complexity Metrics (13) Predict FROM: Code‏ Approximation Predict using: Logistic Regression Are the candidate UML metrics good enough to predict fault-proneness?

33. Case study Objective: Estimate the probability of having a faulty class during the testing phase, using Logistic Regression.

34. Case study Description. Using the design and implementation of the e-commerce system described in Gomaa’s book, this case study was carried out as follows: • Collection of UML and Code metrics (Xs) • Collection of data related to the faults of the e-commerce system from the logs of the CVS repository used (Y) • Evaluation of the relationship between each metric to fault-proneness, using Univariate Logistic Models

35. Case study Metrics to evaluate. Due to the manner the e-commerce system was designed and implemented, without inheritance classes: (*) Were found good predictors of fault-prone code in Olague’s work [2007].

36. Case study Estimation of the probability of a class of being faulty, using CBO-code. Correctness: 12/13 classes 92.3% classes correct classified Sensitivity: 8/9 faulty classes 88.8% Faulty classes correct classified Specificity: 4/4 no-faulty classes 100% No-faulty classes correct classified

37. Case study Results. From the univariate models using each one of the metrics proposed. CIS: Public Methods in a class DAM: Ratio of number of private and protected attributes to the total number of attributes DCC measures were not significant for this study

38. Case study Results • Our second approach to approximate RFC with UML diagrams performed equally to the RFC metric measured from code • UML CIS approximation performed similarly to the CIS metric measured from the code • The rest of the UML metrics’ performance was somewhat acceptable

39. Case study Can we apply the obtained models to other case studies?

40. Conclusions and Future work • UML metrics can be acceptable predictors of fault-prone code • UML CIS and UML RFC metrics showed strongrelationship to fault-proneness of code • We might be able to create a more robust model to predict fault-prone code before its implementation.

41. Conclusions and Future work • Further study and evaluation of other metrics using other UML artifacts (e.g. sequence diagrams, state diagrams and description of use cases) is needed. • Construction of a more robust model using multivariate logistic regression • Evaluation of the final model obtained, using different study cases

42. Quality prediction model for object oriented software using UML metrics Camargo Ana Erika erika.camargo@jaist.ac.jp