Software Metrics and Measurements

1. Software Metrics and Measurements Supannika Koolmanojwong CS577

2. Outline • General Concepts about Metrics • Example of Metrics • Agile Metrics • Metrics from Empirical Data

3. Objectives of software measurement • “You can not control what you cannot measure.” – Tom DeMarco • “Not everything that counts can be counted. Not everything that is counted counts.” – Albert Einstein

4. Software Metrics • Numerical data related to software development • Strongly support software project management activities • Can be directly observable quantities or can be derived from one

5. A simplified measurement information model Information products Decisions / Actions Measurements Information needs Information Needs, Objectives, Control Attributes Process Work Products Results Ref: Ebert and Dumke 2007

6. How the software measurements are used? • Understand and communicate • Specify and achieve objectives • Identify and resolve problems • Decide and Improve

7. Measurement Standard How to do How to do better ISO/IEC 12207 Software Life Cycle Processes ISO/IEC 15288 System Life Cycle processes SWEBOK Software Engineering Body of Knowledge PMBOK Project Management Body of Knowledge CMMI Capability Maturity Model Integration ISO 15504 Software Process Capability Determination ISO 9001 Quality Management System ISO/IEC 9126 Software Product Quality TL 9000, AS 9100, etc. Objectives adaptations How to measure what you are doing ISO/IEC 15939:2002 Software Measurement Process

8. Ground rules for a Metrics • Metrics must be • Understandable to be useful • Economical • Field tested • Highly leveraged • Timely • Must give proper incentives for process improvement • Evenly spaced throughout all phases of development • Useful at multiple levels http://www.stsc.hill.af.mil/resources/tech_docs/gsam3/chap13.pdf

9. Measurements for Senior Management • Easy and reliable visibility of business performance • Forecasts and indicators where action is needed • Drill-down into underlying information and commitments • Flexible resource refocus

10. Measurements for Project Management • Immediate project reviews • Status and forecasts for quality, schedule, and budget • Follow-up action points • Report based on consistent raw data

11. Project management supporting metrics • Planning - Metrics serve as a basis of cost estimating, training planning, resource planning, scheduling, and budgeting. • Organizing - Size and schedule metrics influence a project's organization. • Controlling - Metrics are used to status and track software development activities for compliance to plans. • Improving - Metrics are used as a tool for process improvement and to identify where improvement efforts should be concentrated and measure the effects of process improvement efforts.

12. Measurements for Engineers • Immediate access to team planning and progress • Get visibility into own performance and how it can be improved • Indicators that show weak spots in deliverables • Focus energy on software development

13. What do you want to measure? • Processes • Software-related activities • Products • Artifacts, deliverables, documents • Resources • The items which are inputs to the process

14. Components of software measurements

15. Example of Metrics • Progress / Effort / Cost Indicator • Earned value management • Requirements / Code Churn • Defect-related metrics • Test-related metrics

16. Progress Indicator

17. Effort Indicator

18. Cost Indicator

19. Earned value management • Planned Value (PV) or Budgeted Cost of Work Scheduled (BCWS) • Earned Value (EV) or Budgeted Cost of Work Performed (BCWP) http://en.wikipedia.org/wiki/Earned_value_management

20. Burndown Chart http://en.wikipedia.org/wiki/Burn_down_chart

21. Requirements Churn/ Requirements Creep/ Requirements Volatility • number of changes to system requirements in each phase/week/increment

22. Code Churn • Software change history • Large / recent changes • Total added, modified and deleted LOC • Number of times that a binary was edited • Number of consecutive edits

23. Code Complexity • Gathered from code itself • Multiple complexity values • Cyclomatic complexity • Fan-In / Fan-Out of functions • Lines of Code • Weighted methods per class • Depth of Inheritance • Coupling between objects • Number of subclasses • Total global variables

24. Code coverage • Degree to which the source code is tested • Statement coverage • Has each node in the program been executed? • Branch coverage • Has each control structure been evaluated both to true and false?

25. Code Coverage http://www.firstlinesoftware.com/metrics_group2.html

26. Defect reporting metric • Can be categorized by • Status • Remaining / Resolved / Found • Defect Sources • Requirements / Design / Development • Defect found • Peer review / unit testing / sanity check • Time • Defect arrival rate / Defect age

27. Defect Status

28. Defect Density

29. Test Pass Coverage http://www.jrothman.com/Papers/QW96.html

30. Defect Density

31. Ref: Measuring Agility, Peter Behrens

32. Velocity = Work Completed per sprint Ref: Measuring Agility, Peter Behrens

34. Measurements in organizational level • Empirical analysis • Change from the top

35. Richard W. Selby, Northrop Grumman Space Technology, ICSP '09Title: "Synthesis, Analysis, and Modeling of Large-Scale Mission-Critical Embedded Software Systems"

38. Measurements for progress vs predictions Ref: Ebert and Dumke, 2007

39. References • http://sunset.usc.edu/classes/cs577b_2001/metricsguide/metrics.html • Fenton NE, Software Metrics: A Rigorous Approach, Chapman and Hall, 1991. • http://www.stsc.hill.af.mil/resources/tech_docs/gsam3/chap13.pdf • Christof Ebert, Reiner Dumke, Software measurement: establish, extract, evaluate, execute, Springer 2007 • http://se.inf.ethz.ch/old/teaching/2010-S/0276/slides/kissling.pdf • Richard W. Selby, Northrop Grumman Space Technology, ICSP '09Title: "Synthesis, Analysis, and Modeling of Large-Scale Mission-Critical Embedded Software Systems“ • Measuring Agility, Peter Behrens