A Metrics Program

1. A Metrics Program

2. Advantages of Collecting Software Quality Metrics • Objective assessments as to whether quality requirements are being met can be made during development • A quantitative assessment of quality can provide the basis for decisions regarding the software’s fitness for use • The effectiveness of the software development process can be objectively assessed

3. Steps to a Metrics Program according to Grady and Caswell • Define the objectives for the program • Assign responsibility • Do research • Define initial metrics to collect • Sell the initial collection of these metrics • Get tools for automatic data collection/analysis • Establish training in software metrics • Publicize success stories • Create a metrics database • Establish a way for improving the process

4. Important Points to Remember for a Software Metrics Program • Defining clear objectives is crucial • Objectives should address the following: • Expected costs • Possible cost savings • Expected improvements in quality • It must be part of an overall program for process improvement

5. The Software Quality Metrics Framework • Quality requirements that the software product must meet • Quality factors – Management-oriented attributes of software that contribute to its quality • Quality subfactors – Decompositions of a quality factor to its technical components • Metrics – quantitative measures of the degree to which given attributes (factors) are present

6. Example • Quality requirement – “The product will be easy to use” • Quality factor(s) – Usability (An attribute that bears on the effort needed for use and on the assessment of such use by users) • Quality subfactors – Understandability, ease of learning, operability, communicativeness

7. Subfactors • Understandability – The amount of effort required to understand software • Ease of learning – The degree to which user effort required to learn how to use the software is minimized • Operability – The degree to which the effort required to perform an operation is minimized • Communicativeness – The degree to which software is designed in accordance with the psychological characteristics of users

8. Direct Metrics • Understanding • Learning time: Time for new user to gain basic understanding of features of the software • Ease of learning • Learning time: Time for new user to learn how to perform basic functions of the software • Operability • Operation time: Time required for a user to perform operation(s) of the software • Communicativeness • Human factors: Number of negative comments from new users regarding ergonomics, human factors, etc.

9. Types of Metrics • Direct metric – One that does not depend upon the measure of other attributes • Software quality metric – A function whose inputs are software measures and whose output is a single numerical value that can be interpreted as the degree to which software possesses a given attribute that affects its quality

10. Types of Metrics (cont’d) • Process metric – A metric used to measure characteristics of the methods, techniques, and tools employed in developing, implementing, and maintaining a software system • Product metric – A metric used to measure that characteristic of any product of the software development process

11. IEEE Software Metrics Methodology • Establish software quality requirements • Identify software quality metrics • Implement the software quality metrics • Analyze the software metrics results • Validate the software quality metrics

12. Establish Software Quality Requirements • What group is empowered to define software quality requirements? • How should customers provide input? • How are requirements conflicts resolved?

13. Identify Software Quality Metrics • Specify important quality factors and subfactors • Identify direct metrics • Name • Costs • Target value • Tools • Application • Data items • Computation

14. Example of Documenting a Metric

15. Implement the Collection of Data

16. Analyze Software Quality Metric Results • Results need to be analyzed within the context of the project’s overall software quality requirements • Any metrics that fall outside of their respective targets should be identified for further analysis

17. Validate the Software Quality Metrics • Assess the statistical significance of the metrics to the quality factors they represent • See the IEEE Standard 1061-1998 for a thorough description of this process

18. Attributes of a Measurement Program – according to Humphrey • The measures should be robust • The measures should suggest a norm • The measures should relate to specific product and process properties • The measures should suggest an improvement strategy

19. Attributes of a Measurement Program – according to Humphrey (cont’d) • The measures should be a natural result of the software development process • The measures should be simple • The measures should be predictable and trackable • The measures should not be used as part of a person’s performance evaluation

20. Template for Software Quality Goal Definition • Purpose: To (characterize, evaluate, predict, monitor, etc.) the (process, product, model, metric, etc.) in order to (understand, plan, assess, manage, control, engineer, learn, improve, etc.) it. • Example: To evaluate the maintenance process in order to improve it.

21. Template for Software Quality Goal Definition (cont’d) • Perspective: Examine the (cost, effectiveness, correctness, defects, changes, product measures, etc.) from the viewpoint of the (developer, manager, customer, etc.) • Example: Examine the effectiveness from the viewpoint of the customer.

22. Template for Software Quality Goal Definition (cont’d) • Environment: The environment consists of the following: process factors, people factors, methods, tools, constraints, etc. • Example: The maintenance staff are poorly motivated programmers who have limited access to tools.

23. Determining Metrics