1 / 22

Prediction

Prediction. Basic concepts. Scope. Prediction of: Resources Calendar time Quality (or lack of quality) Change impact Process performance Often confounded with the decision process. Historical data. Y (dependent, observed, response variable). explained variance of observed Y i.

barclay-sykes
Download Presentation

Prediction

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Prediction Basic concepts

  2. Scope Prediction of: • Resources • Calendar time • Quality (or lack of quality) • Change impact • Process performance • Often confounded with the decision process

  3. Historical data Y (dependent, observed, response variable) explained variance of observed Yi prediction interval of new observation Y0 at x0 x0 X (independent, prediction variable) known unknown

  4. Methods for building prediction models • Statistical • Parametric • Make assumptions about distribution of the variables • Good tools for automation • Linear regression, Variance analysis, ... • Non-parametric, robust • No assumptions about distribution • Less powerful, low degree of automation • Rank-sum methods, Pareto diagrams, ... • Causal models • Link elements with semantic links or numerical equations • Simulation models, connectionism models, genetic models, ... • Judgemental • Organise human expertise • Delphi method, pair-wise comparison, rule-based methods

  5. Common SE-predictions • Detecting fault-prone modules • Project effort estimation • Change Impact Analysis • Ripple effect analysis • Process improvement models • Model checking • Consistency checking

  6. Introduction • There are many faults in software • Faults are costly to find and repair • The later we find faults the more costly they are • We want to find faults early • We want to have automated ways of finding faults • Our approach • Automatic measurements on models • Use metrics to predict fault-prone modules

  7. Related work • Niclas Ohlsson, PhD work 1993 • AXE, fault prediction, introduced Pareto diagrams, • Predictor: number of new and changed signals • Lionel Briand, Khaled El Eman, et al • Numerous contributions in exploring relations between fault-proness and object-oriented metrics • Piotr Tomaszewski, PhD Karlskrona 2006 • Studies fault density • Comparison of statistical methods and expert judgement • Jeanette Heidenberg, Andreas Nåls • Discover weak design and propose changes

  8. Approach • Find metrics (independent variables) • Number of model elements (size) • Number of changed methods (change) • Transitions per state (complexity) • Changed operations * transitions per state (combinations) • ... • Use metrics to predict (dependent variable) • Number of TRs

  9. Capsules

  10. State charts

  11. Data model package capsule class port protocol attribute operation State machine signal State transition

  12. Our project - modelmet • RNC application - Three releases • About 7000 model elements • TR statistics database (2000 TRs) • Find metrics • Existing metrics (done at standard daily build) • Run scripts on models • Statistical analysis • Linear regression, principal component analysis, discriminant analysis, robust methods • Neural networks, Bayesian belief networks

  13. Size Change Complexity Combined

  14. Metrics based on change, system A

  15. Metrics based on change, system B

  16. Complexity and size metrics, system A

  17. Complexity and Size metrics, system B

  18. Other metrics, system A 0.034 states – 0.965 protocols TRD = C + modelelements

  19. Other metrics, system B

  20. How to use predictions • Uneven distribution of faults is common – 80/20 rule • Perform special treatment on selected parts • Select experienced designers • Provide good working conditions • Parallell teams • Inspections • Static and dynamic analysis tools • ... • Perform root-cause analysis and make corrections

  21. Results Contributions: • Valid statistical material: • Large models, large number of TRs • Two change projects • Two highly explanatory predictors were found • State chart metrics are as good as OO metrics Problems: • Some problems to match modules in models and TRs • Effort to collect change data

More Related