Software Quality : The Elusive Target

1. Software Quality : The Elusive Target January, 1996 IEEE Software Kitchenham B. & Pfleeger S. L. Zhenyu Wu CS5391 Spring 2003

2. Introduction Questions: • What do you really mean by software quality (SQ)? • Is your definition adequate? • How to evaluate your product’s SQ ? • How to measure SQ? • How to achieve SQ? CS5391 Spring 2003

3. Meaning of SQ • A good definition should let us measure quality in a meaningful way • Measurements let us know • if techniques improve quality • how process quality affects product quality • how quality methods affect product quality during use • Is the investment in SQ methods profitable? CS5391 Spring 2003

4. Meaning of SQ… • Depends on how you approach software quality • Different groups involved have different views • Product-based • Process-based • People believe quality is important and can be improved CS5391 Spring 2003

5. Views of SQ • SQ can be perceived in various domains (philosophy, economics, marketing and operations management) • SQ is complex and multifaceted concept and can be described from five different perspectives • Transcendental view • User view • Manufacturing view • Product view • Value-based view CS5391 Spring 2003

6. Transcendental View • See quality as something that can be recognized but not defined • Quality as something toward which we strive as an ideal, but may never implement completely • Abstract sense CS5391 Spring 2003

7. User View • More concrete, grounded in the product characteristics that meet user’s needs • Evaluate product in a task context can be personalized view • Asses product behavior with respect to operational profiles • Related to product usability and reliability CS5391 Spring 2003

8. Manufacturing View • Focus on product quality during production and after delivery • Concerns whether or not the product was constructed “right the first time” • Can lead to quality assessment independent of the product • Adapted by ISO 9001 and CMM, advocates conformance to process rather than to specification CS5391 Spring 2003

9. Product View • Consider the product’s inherent characteristics, looks inside • Assumption : measuring and controlling internal product properties will result in improved external product behavior • More research needed to conform this assumption CS5391 Spring 2003

10. Value-Based View • Different views can be held by different groups involved in software development • user’s requirement for a useful product conflict with manufacturer’s goal of minimizing rework • consider the trade-offs between cost and quality • manage conflict  design to cost • compare with potential benefits • This view tries to bring in compromise between different groups and views CS5391 Spring 2003

11. Measuring Quality • We need to measure quality so we can: • Establish baselines • Predict likely quality • Monitor improvement • Product attributes contribute to user satisfaction are a mixture of: • product’s functions • product’s nonfunctional qualities • constraints • ISO definition of quality: the totality of characteristics of an entity that bear on its ability to satifsfy stated and implied needs. CS5391 Spring 2003

12. Measuring user's view • Reliability: how long the product functions properly between failures • Usability: including ease of installation, learning and use • Tom Gilb’s technique: characteristics can be measured directly • Example: Learning time = average elapsed time (in hours) for a typical user to achieve stated level of competence CS5391 Spring 2003

13. Measuring user's view… • This technique can be generialized to any quality feature • The quality concept is broken down into component parts until each can be stated in terms of directly measurable attributes CS5391 Spring 2003

14. Measuring the Manufacturer's View • Defects count • Count the number of known defects recorded during development and use • Can compare modules, products or projects (with same way and same time) • Relationship between defects count and operational failures is unclear • Can indicate test efficiency and identify process improvement areas CS5391 Spring 2003

15. Measuring the Manufacturer's View… • Rework costs • Any additional effort required to find and fix problems after documents and code are formally signed-off as part of configuration management • Count debugging effort during integration and testing • Not count end-phase verification and validation • Pre-release rework: measure manufacturing efficient and process improvement • Post-release rework: measure of deliverd quality CS5391 Spring 2003

16. Modeling Quality • Several models have been built to understand and measure quality • Old Models • McCall’s model • ISO 9126 • New model • Dromey’s model CS5391 Spring 2003

17. McCall's Model • Defines product qualities as hierarchy of factors, criteria and metrics: • Quality factor represents behavior characteristic of the system • Quality criterion is an attribute of a quality factor that is related to software production and design • Quality metric is a measure that captures some aspect of a quality criterion. CS5391 Spring 2003

18. McCall's quality Model… • The metrics have to be answered with ‘yes’ or ‘no’ answer • The final quality is assessed by dividing the ‘yes’ answers by the total number of questions chosen to describe the quality CS5391 Spring 2003

19. McCall’sModel… CS5391 Spring 2003

20. ISO 9126 • Six characteristics, completely hierarchical • Quality charactertistics are divided into sub-characteristics which can be measured using metrics • Recommends measuring the char-acteristics directly, but does not indicate clearly how to do it CS5391 Spring 2003

21. CS5391 Spring 2003

22. CS5391 Spring 2003

23. ISO 9126… Suitability Accurecy Interoperability Secuity Functionality Materity Fault Telerance Recoverability Reliebility Understandability Learnability Operability Usability Characteristics Sub-characteristics CS5391 Spring 2003

24. ISO 9126… Time behavior Resource behavior Effiency Analyzability Changeability Stability Testability Maintainability Adaptability Installability Conformance Replaceability Portability Characteristics Sub-characteristics CS5391 Spring 2003

25. Mccall’s with different quality frame quality factor not hierarchical ISO 9126 work and terminology quality characteristic and subcharacteristics completely hierarchical Main Differences between Mccall’s and ISO 9126 model CS5391 Spring 2003

26. Model Problems • Lack rationale for determining which factor should be included in the quality definition • Lack rationale for deciding which criteria relate to a particular factor • The measure of metrics can be too subjective • Does not describe how lower level metrics are composed into an overall assessment of higher level quality characteristics CS5391 Spring 2003

27. Dromey's models(new model) • Higher level quality attributes cannot be measured or built directly • Higher level quality can be achieved by building components which exhibit a consistent, harmonious and complete set of product properties that result in manifestations of quality attributes • Can allow us to verify models CS5391 Spring 2003

28. Business Value of Quality • Businesses invest valuable money for obtaining good software, we have to pay them back in good • We should be careful in assessing and providing software with good quality • How much "less than perfect" software are the businesses willing to accept should be determined CS5391 Spring 2003

29. Conclusion • Quality is a complex concept; It means different things to different people • You must define aspects of quality in which you are interested • Quality should be defined in a measurable way, so that it can be understood and verified • Quality must be related to business goals CS5391 Spring 2003