Software Quality Assurance

1. Software Quality Assurance SE 3354

2. Software Quality Assurance • What is “quality”?

3. Software Quality Assurance • What is “quality”? • IEEE Glossary: Degree to which a system, component, or process meets (1) specified requirements, and (2) customer or user needs or expectations

4. Software Quality Assurance • What is “quality”? • IEEE Glossary: Degree to which a system, component, or process meets (1) specified requirements, and (2) customer or user needs or expectations • ISO: the totality of features and characteristics of a product or service that bear on its ability to satisfy specified or implied needs

5. Software Quality Assurance • An alternate view of Quality: • is not absolute • is multidimensional, can be difficult to quantify • has aspects that are not easy to measure • assessment is subject to constraints (e.g., cost) • is about acceptable compromises • criteria are not independent, can conflict

6. Software Quality Assurance • Quality Criteria include: • correctness • efficiency • flexibility • integrity • interoperability • maintainability • portability • reliability • reusability • testability • usability

7. What is Software Quality Assurance (SQA)? • “Set of systematic activities providing evidence of the ability of the software process to produce a software product that is fit to use” • G. Schulmeyer and J. McManus, Software Quality Handbook, Prentice Hall, 1998.

8. What is SQA? • Monitoring processes and products throughout the software development lifecycle to ensure the quality of the delivered product(s) • Monitoring the processes • Provides management with objective feedback regarding process compliance to approved plans, procedures, standards, and analyses • Monitoring the products • Focus on the quality of product within each phase of the SDLC • e.g., requirements, test plan, architecture, etc. • Objective: identify and remove defects throughout the lifecycle, as early as possible

9. Quality of Software developed in-house & COTS components • SQA processes apply when integrating purchased or customer-supplied software products into the developed product • Question. How do you determine the “quality” of COTS components? • Current research problem

10. Process Assessment • Use of standards and process models has a positive impact on the quality of the software product • Disciplined, controlled development process • Examples include: • ISO 9001 • CMM • CMU SEI, 5 levels • SPICE • Developing a standard for software process assessment • ISO joint committee, Europe, Australia • IEEE 1074, IEEE 12207

11. Product Assessment • Reviews, inspections, walkthroughs of Plans, reports, models, standards • Project management, quality assurance, training, test plan(s) • Requirements, analysis, architecture, detailed design model, test cases • Issue or problem reports • Metric reports • Traceability reports • Documentation, coding standards • …

12. Software Reviews • They may include managerial reviews, acquirer-supplier reviews, technical reviews, inspections, walkthroughs, and audits. • Inspection: • A formal evaluation technique in which an artifact (e.g., software requirements, design, or code) is examined in detail by a person or group other than the originator • detect faults, violations of development standards, and other problems. • review members are peers (equals) of the designer or programmer. • data is collected during inspections for later analysis and to assist in future inspections. Note • Introduced by Fagan, 1976.

13. Picture from “Inspections” presentation http://www.math.uaa.alaska.edu/~afkjm/cs470/handouts/inspections.pdf

14. Problem Reporting, Tracking, and Resolving • Describe the practices and procedures to be followed for reporting, tracking, and resolving problems • Who can report a problem? • How is it reported? • How is is tracked? • Who determines if it is a problem that going to be resolved? • How is it assigned for resolution? • How does the person indicate it has been corrected? • Who reviews it to determine if it can be closed? • Problems can be product or process related • e.g. incorrect requirement, incomplete class definition, code defect, ambiguous description in user documentation, process to review detailed design is not clearly defined, etc.

15. Metrics • Metrics for each artifact • e.g., Requirements • Number of requirements • Number of changes per requirement • Called “churn” rate • Characterization of defects • Not testable, ambiguous, inconsistent, incorrect, incomplete redundant, infeasible, … • Major or minor defect • Phase defect detected • Cost to fix

16. Tools, techniques, training • What tools? • e.g., CVS for CM, excel spreadsheet for problem reporting/tracking, ... • What techniques? • e.g., formal peer review for deliverables, checklists for defect detection, ... • What training is needed on tools, techniques?

17. Media Control • Identify the media for each intermediate and deliverable artifact • Documentation required to store the media, including the backup and restore process • Protect computer program physical media from: • unauthorized access • inadvertent damage • degradation

18. References • Fagan, M., “Design and Code Inspections to Reduce Errors in Program Development”, IBM Systems Journal, 15, 3 (1976), pp. 182-211 • Fagan, M., “Advances in Software Inspections”, IEEE Transactions on Software Engineering, 12, 7(July 1986), pp. 744-751 • Schulmeyer G. and McManus, J., Software Quality Handbook, Prentice Hall, 1998. • IEEE Std 730™ 2002, IEEE Standard for Software Quality Assurance Plans, IEEE Computer Society, Sponsored by the Software Engineering Standards Committee • Rosenberg, L.H.; Gallo, A.M., Jr.,“Software quality assurance engineering at NASA”, Proceedings of the IEEE Aerospace Conference, 2002, Volume: 5 , 2002, pp. 5-2569 -5-2575. • “Inspections”, http://www.math.uaa.alaska.edu/~afkjm/cs470/handouts/inspections.pdf