SW Quality Assurance Landscape

1. CS352 – Software Engineering IILecture 17: SW Quality Assurance Landscape Slides by Mohammad El-Ramly, PhD

2. حديث شريف • إن الله يحب إذا عمل أحدكم عملا أن يتقنه • إن الله كتب الإحسان على كل شئ

3. Lecture 17 Outline • Understanding SW Quality Attributes • Understand the value of SW QA • Quality Landscape • SW QA Jobs and Certificates • Effectiveness of QA Techniques Readingsare essentialfor this course. Reading: (7) Code Complete Chapter 20 Accompanying site: http://cc2e.com/

4. What is SW Quality? • Customer’s Perspective • System does not crash • System follows documentation & contract • System is logical and easy to use • Developer’s Perspective • System is easy to change • System is easy to understand • System is pleasant to work on

5. External Quality Characteristic ● Correctness. • The degree to which a system is free from faults in its specification, design, and implementation. ● Usability. • The ease with which users can learn and use a system. ● Efficiency. • Minimal use of system resources, including memory and execution time.

6. External Quality Characteristic ● Reliability • The ability of a system to perform its required functions under stated conditions whenever required—having a long mean time between failures. ● Robustness • The degree to which a system continues to function in the presence of invalid inputs or stressful environmental conditions.

7. External Quality Characteristic ● Integrity • The degree to which a system prevents unauthorized or improper access to its programs and data. • Ensuring that data is accessed properly ● Adaptability • The extent to which a system can be used, without modification, in applications or environments other than those for which it was specifically designed.

9. Internal Quality Characteristic ● Maintainability • The ease with which you can modify a software system to change or add capabilities, improve performance, or correct defects. ● Flexibility • The extent to which you can modify a system for uses or environments other than those for which it was specifically designed.

10. Internal Quality Characteristic ● Reusability • The extent to which and the ease with which you can use parts of the system in other systems. ● Understandability / Readability • The ease with which you can comprehend a system at organizational and detailed levels.

11. Total Quality Management • Factories • Goal is for every item coming off the assembly line to be perfect • Management, production, engineering, QA • Everyone is involved in quality • Develop a reliable, repeatable process • Continuously improve the process

12. The Cost of Poor Quality • Money • Time • Resources • Lives • Reputation • Business

13. Failure Costs • Internal • Rework • Repair • Failure analysis • External • Resolving complaints • Returning and replacing product • Help line

14. Prevention Costs • Prevention • Planning • Managing and collecting information • Reviews • Appraisal • Inspection • Testing

15. The Cost of Poor Quality

17. Techniques for Improving SW Quality • What can we do improve quality ? • SW QA is a wide spectrum of activities conducted through out the development process that focus on: • Product quality • Process quality

18. Techniques for Improving SW Quality (Code Complete) • SW Q Objectives & Planning • Explicit QA Activities • Testing Strategy • SE Guidelines • Formal / Informal Technical Reviews • Change-control Procedures • Measurement of Results • Prototyping

20. Change and Configuration Management Software Quality Landscape Reviews Collaborative Development / Reviews Requirements V & V Guidelines, Standards and Templates Design Review Bug Tracking Surveys Contract Review QA Planning and Budgeting Testing Metrics Debugging Refactoring Deployment & Maintenance Planning Design Testing Requirements Development M a n a g e m e n t

21. 1. SW Quality Objectives • Decide explicit quality characteristics • Inform the developers • They are morelikely to achieve quality objectives if they now them. • Review their work • Study by G M Weinberg, E L Schulman, Goals and Performance in Computer Programming (1974)

24. 2. Explicit QA Activities • QA should be perceived as a primary not secondary goal • QA should be an organizationalculture. • Organization should train developers that quality is a priority.

25. 3. Testing Strategy • Develop a test strategy with product requirements, architecture and design. • Or even consider test-driven development.

26. 4. SE Guidelines • Decide on guidelines, templates, checklists and standards to follow. • Requirements, architecture and design guidelines. • Templates for plans, reports, etc. • Coding style. • File naming conventions. • Apply them and enforce them in every phase.

27. 5. Reviews • Informal Reviews • Desk-checking • Walk through • Formal Technical Reviews • Act as a “quality gate” • Catch problems at the “lowest-value” stage—that is, at the stage in which problems cost the least tocorrect. • Can be a peer review, a customer review, an, inspection, or an externalaudit.

28. 6. Change-control Procedures • Changing hurts quality. • Remember the Vaza • Uncontrolled changes: • Requirements => unstable designs and code • Design => Inconsistency with req. and code • Code => Inconsistency / invalid doc. • Adopt incremental development • Adopt Configuration and Change Management

29. 7. Measurement of Results • Measuring what ? • 3P => Product, Project and Process • Quality attributes. • Measurements tells you if QA plan failed or succeeded. • Can be the basis for improvement.

30. 8. Prototyping • Develop realistic model of the system’s key functionalities. • UI to evaluate usability • Key calculations to see their results, memory requirements and performance • Studies suggest that prototyping can lead to better designs, better satisfaction of user needs and improved maintainability.

32. CS428 SQA Manager • Responsible for SQA policy • Develops testing plan • Checks that plan is being followed • Develops review process • Trains reviewers • Monitors reviews • Develops customer survey plan • …

33. SW QA Engineer Ad. • Nonin Medical, Inc. is a rapidly growing international medical device manufacturer that has been designing and distributing high quality medical monitoring devices since 1986. • Responsible for ensuring software quality within complex medical device products through the development and execution of software test plans by way of verification procedures based upon software and system specifications • http://www.nonin.com/JobPost.aspx?ID=243

34. SW QA Engineer: Duties & Responsibilities • Review product design documentation to ensure that requirements stated are correct, unambiguous, and verifiable. • Consult with product development to evaluate system interfaces, operational requirements, and performance requirements of overall system. • Define test methods and create test plans for new or updated software projects. • Recommend design improvements or corrections to engineers throughout the development process. • Maintain effective communication with the project software engineers..

35. SW QA Engineer: Duties & Responsibilities • Execute test plans and create test reports to describe program evaluation, testing, and correction. • Monitor program performance after implementation to prevent reoccurrence of program operating problems and ensure efficiency of operation. • Conduct compatibility tests with vendor-provided programs.

38. Effectiveness of Quality Techniques

40. Cost of Fixing Defects • Fixing a defect in later phases costs more • One step techniques are cheaper than 2-steps • Inspection: symptoms and cause • Testing: symptoms (more work to get root cause) • MS: 3 hrs to find and fix a bug by inspection • MS: 12 to find and fix a bug with testing

41. SW QA Program must…. • Employ a variety of techniques to achieve higher-than-average quality • Formaldesigninspections of the critical parts of a system. • Modeling or prototyping using a rapid prototyping technique • Code reading or inspections • Execution testing