SOFTWARE QUALITY ASSURANCE & MANAGEMENT CHAPTER # 03 Software Quality Assurance

1. SOFTWARE QUALITY ASSURANCE & MANAGEMENTCHAPTER # 03Software Quality Assurance

2. ALI JAVED Lecturer SOFTWARE ENGINEERING DEPARTMENT U.E.T TAXILA Email:: ali.javed@uettaxila.edu.pk Office Room #:: 7

3. Presentation Outline • Software Quality Assurance • SQA Group • SQA Group Tasks • SQA Plan • Software Reviews • Inspections • Fagan Inspection • Gilb Inspection • Walkthroughs • Desk Checking • Peer Ratings

4. Software Quality Assurance • Software QA involves monitoring and improving the entire software development process, making sure that any agreed-upon standards and procedures are followed, and ensuring that problems are found and dealt with. It is oriented to 'prevention’ • Software quality assurance is an umbrella activity that is applied throughout the software process. • SQA encompasses: • a quality management approach • formal technical reviews • a multi-tiered testing strategy • software development standard • measurement and reporting mechanism

5. SQA Group • Who involves quality assurance activities? • Software engineers, project managers, customers, sales people, SQA group • The SQA group’s role • serves as the customer’s in-house representative • assist the software engineering team in achieving high-quality • The SQA group’s responsibility: • Quality assurance planning • Reviews and Analysis • Record keeping • Reporting

6. SQA Plan • The software quality assurance plan is an outline of quality measures • to ensure quality levels within a software development effort. • The plan is used to compare the actual levels of quality • during development with the planned levels of quality. • If the levels of quality are not within the planned quality levels, • management will respond appropriately as documented within the • plan. • Basic items: • purpose of plan and its scope • Describe the purpose and scope of the document and indicate those software process activities that are covered by quality assurance • All documents noted in the SQA plan are listed

7. SQA Plan • management • organization structure • SQA tasks and activities and their placement throughout the software process • Organizational roles and responsibilities related to product quality • Documentation • Project documents- e.g project plan • Technical documents- e.g specifications, test plans • User documents- e.g help files • standards, practices, and conventions • List all applicable standards and practices that are applied during the software process (e.g document standards, coding standards and review guidelines) • All project and process metrics are listed

8. SQA Plan • reviews and audits • Identifies the reviews and audits to be conducted • Provides an overview of the approach for each review and audit • test • test plan and procedure • problem reporting, and correction actions • Defines procedures for reporting, tracking and resolving errors and defects • Tools • identifies the tools that supports SQA tasks and activities procedures for reporting, tracking and resolving errors and defects, and identifies the organizational responsibilities for these activities

9. SQA Plan • Software configuration management procedures • Required for controlling change • records collection, maintenance, and retention • Training • Identifies training required to meet the needs of the plan • risk management • Defines methods for identifying, assessing, monitoring and controlling risk

10. Software Reviews • What is software review? • A “filter” for the software engineering process. • Purpose of Software Reviews • Serves to uncover errors in analysis, design, coding etc. • Why software reviews? • Humans do mistakes, so some process is required to catch the errors.

11. Formal Technical Reviews • Objectives of FTR: • to uncover errors in function, logic, or implementation • to verify the software under review meets its requirements • to ensure that the software has been represented according to predefined standards • to develop software in a uniform manner • to make projects more manageable • Review meeting’s constraints: • 3-5 people involved in a review • advanced preparation (no more than 2 hours for each person) • the duration of the review meeting should be less than or equal to 2 hours

12. Formal Technical Reviews • People involved in a review meeting: • Producer • review leader • 2 or 3 reviewers (one of them is recorder) • The preparation of a review meeting: • a meeting agenda and schedule (by review leader) • review material and distribution (by the producer) • review in advance (by reviewers)

13. Formal Technical Reviews • Review meeting results: • a review issues list • a simple review summary report (called meeting minutes) • meeting decisions: • - accept the work product w/o further modification • - reject the work product due to errors • - accept the work under conditions (such as change and review) • - sign-off sheet • Review summary report answers the following questions: • what was reviewed? • who reviewed it? • what were the findings and conclusions • Review issues list serves two purposes: • to identify problem areas in the project • to serve as an action item checklist (a follow-up procedure is needed)

14. Review Guidelines • A minimum set of guidelines for FTR: • Review the product, not the producer • Set an agenda and maintain it • Limit debate and rebuttal • Enunciate problem areas, but don’t attempt to solve every problem noted • Take written notes • Limit the number of participants and insist upon advance preparation

15. Review Guidelines • Develop a checklist for each work product that is likely to be reviewed • Allocate resources and time schedule for FTRs • Conduct meaningful training for all reviewers • Review your early reviews

16. Inspection • An inspection is a review of any document (e.g requirements specification, design etc) which typically involves 3-5 people including a moderator, reader, and a recorder to take notes. • The subject of the inspection is typically a document and the purpose is to find problems and see what's missing, not to fix anything. • Attendees should prepare for this type of meeting by reading through the document; most problems will be found during this preparation. • The result of the inspection meeting should be a written report.

17. Generic Inspection Process • Generic process/steps: • Planning and preparation (individual) • 2. Collection (group/meeting) • 3. Repair (follow up)

18. Generic Inspection Process • 1. Planning and preparation (individual) • Inspection planning needs to answer the general question about • the inspection, including: • What are the objectives or goals of the inspection? • What are the software artifacts to be inspected? • Who are performing the inspection? • Who else need to be involved, in what roles, and with what specific responsibilities? • What are the overall process, techniques and follow-up activities of the inspection? • Preparation of these meetings involves the individual inspection • of the work product or artifacts by each member of inspection • Team before the inspection meeting.

19. Generic Inspection Process • 2. Inspection or collection • This step corresponds to the execution of QA activities • This step refers to the inspection meeting alone while all • activities leading up to this session are grouped in the previous step • This step is also referred to as collection or collection meeting • The focus of this step is to detect faults in the software artifacts inspected, and record the inspection results so that these faults can be resolved in the next step

20. Generic Inspection Process • 3. Correction and follow-up • The discovered faults need to be corrected by the people who • are responsible for the specific software artifacts inspected. • For example, in design or code inspection, the responsible • designer or programmer, often labeled as design or code owners in industry, need to fix the design or code. • There must be some follow-up activities to verify the fix. Sometimes new inspection rounds can be planned and carried out, as illustrated by the dotted line leading back from the correction & feedback step to the preparation & planning step as shown in the figure

21. Fagan Inspection • Fagan inspection refers to a structured process of trying to find defects in documents such as programming code, specifications, designs and others during various phases of the software development process. It is named after Michael Fagan who is credited with being the inventor of formal software inspections. • Fagan Inspection is a group review method used to evaluate output of a given process. • In the process of software inspection the defects which are found are categorized in two categories: major and minor defects. The defects which are incorrect or even missing functionality or specifications can be classified as major defects: the software will not function correctly when these defects are not being solved. • In contrast to major defects, minor defects do not threaten the correct functioning of the software, but are mostly small errors like spelling mistakes in documents or optical issues like incorrect positioning of controls in a program interface.

22. Fagan Inspection • Typical operations • In a typical Fagan inspection the inspection process consists of the following operations: • Planning • Preparation of materials • Arrangement of participants • Arrangement of meeting place • Overview • Author-inspectors meeting • Assignment of roles • Preparationor Individual Inspection • Independent analysis/examination • Code as well as other document

23. Fagan Inspection • Individual results: • questions • potential defects • The participants prepare their roles • Inspection meeting • Meeting to collect/consolidate individual inspection results • Defect identification, but not solutions, to ensure inspection effectiveness • Fagan inspection typically involves about 4 people in the inspection team • No more than 2 hours • Inspection report • Rework (performed by the author) • Rework is the step in software inspection in which the defects found during the inspection meeting are resolved by the author, designer or programmer. On the basis of the list of defects the low-level document is corrected until the requirements in the high-level document are met.

24. Fagan Inspection • Follow-up • In the follow-up phase of a Fagan Inspection, defects fixed in the rework phase should be verified. The moderator is usually responsible for verifying rework. Sometimes fixed work can be accepted without being verified, such as when the defect was trivial. In non-trivial cases, a full re-inspection is performed by the inspection team (not only the moderator). • If verification fails, go back to the rework process.

25. Fagan Inspection Roles The participants of the inspection process fulfill different roles within the inspection process: Moderator: responsible for the inspection session, functions as a coach Author/Designer/Coder: the person who wrote the low-level document Reader:Reads the document Tester: reviews the document from a testing standpoint

26. Fagan Inspection • General observations and Findings • Importance of preparation: • Studies and survey have pointed out that most defects detected during inspection performed by the inspectors during their preparation before the meeting • Meetings are mainly used to consolidate defects by eliminating false alarms to confirm true defects alternatives focusing on preparation. • Other important findings: • Team size depends upon the size and complexity of the artifacts to be inspected • Prefer systematic detection techniques to ad-hoc ones • Additional use of inspection feedback/analysis.

27. Gilb Inspection Steps of Gilb Inspection Planning (same to Fagan inspection) Kickoff (Overview) Individual Checking (Preparation) Logging Meeting (Inspection) a) Edit (Rework) b) Process brainstorming Edit Audit (Follow-up) Process brainstorming is added right after the inspection meeting. The focus of this step is root cause analysis aimed at preventive actions and process improvement in the form of reduced defect injections for future development activities The team size is typically about four to six people

28. Code Inspection

29. Code Inspection • A code inspection is a set of procedures and error-detection techniques for group code reading. • An inspection team usually consists of four people. One of the four people plays the role of moderator. The moderator is expected to be a competent programmer, but he or she is not the author of the program. • The duties of the moderator include • Distributing materials for, and scheduling, the inspection session Leading the session • Recording all errors found • Ensuring that the errors are subsequently corrected

30. Code Inspection • The second team member is the programmer. • The remaining team members usually are the program’s designer (if different from the programmer) and a test specialist. • The moderator distributes the program’s listing and design specification to the other participants several days in advance of the inspection session. The participants are expected to familiarize themselves with the material prior to the session.

31. Code Inspection • During the session, two activities occur:1. The programmer narrates, statement by statement, the logic of the program. During the discourse, other participants should raise questions, and they should be pursued to determine whether errors exist. It is likely that the programmer rather than the other team members will find many of the errors found during this narration. 2. The program is analyzed with respect to a checklist of historically common programming errors. • The moderator is responsible for ensuring that the discussions proceed along productive lines and that the participants focus their attention on finding errors, not correcting them. (The programmer corrects errors after the inspection session.)

32. Code Inspection After the session, the programmer is given a list of the errors found. If more than a few errors were found, or if any of the errors requires a substantial correction, the moderator might make arrangements to re-inspect the program after the errors are corrected. This list of errors is also analyzed, categorized, and used to refine the error checklist to improve the effectiveness of future inspections.The time and location of the inspection should be planned to avoid all outside interruptions. The optimal amount of time for the inspection session appears to be from 90 to 120 minutes. Since the session is a mentally taxing experience, longer sessions tend to be less productive. Most inspections proceed at a rate of approximately 150 program statements per hour. For that reason, large programs should be examined in multiple inspections, each inspection dealing with one or several modules or subroutines.

33. Code Walkthroughs The code walkthrough, like the inspection, is a set of procedures and error-detection techniques for group code reading. It shares much in common with the inspection process, but the procedures are slightly different, and a different error-detection technique is employed. Like the inspection, the walkthrough is an uninterrupted meeting of one to two hours in duration. The walkthrough team consists of three to five people. One of these people plays a role similar to that of the moderator in the inspection process, another person plays the role of a secretary (a person who records all errors found), and a third person plays the role of a tester.

34. Code Walkthroughs • Suggestions as to who the three to five people should be vary. • Of course, the programmer is one of those people. Suggestions for the other participants include • a highly experienced programmer • a programming-language expert • A new programmer (to give a fresh, unbiased outlook) • someone from a different project, • someone from the same programming team as the programmer.

35. Code Walkthroughs The initial procedure is identical to that of the inspection process: The participants are given the materials several days in advance to allow them to bone up on the program. However, the procedure in the meeting is different. Rather than simply reading the program or using error checklists, the participants “play computer.” The person designated as the tester comes to the meeting armed with a small set of paper test cases—representative sets of inputs (and expected outputs) for the program or module. During the meeting, each test case is mentally executed. That is, the test data are walked through the logic of the program. The state of the program (i.e., the values of the variables) is monitored on paper or whiteboard. Of course, the test cases must be simple in nature and few in number, because people execute programs at a rate that is many orders of magnitude slower than a machine.

36. Desk Checking • A third human error-detection process is the older practice of desk checking. • A desk check can be viewed as a one-person inspection or walkthrough: A person reads a program, checks it with respect to an error list, and/or walks test data through it. • In other words you can say Manually testing the logic of a program. • For most people, desk checking is relatively unproductive. The reason behind is that it runs counter to a testing principle (“that people are generally ineffective in testing their own programs”). • For this reason, you could deduce that desk checking is best performed by a person other than the author of the program.

37. Peer Ratings Peer rating is a technique of evaluating anonymous programs in terms of their overall quality, maintainability, extensibility, usability, and clarity. The purpose of the technique is to provide programmer self-evaluation. A programmer is selected to serve as an administrator of the process. The administrator, in turn, selects approximately 6 to 20 participants(6 is the minimum). The participants are expected to have similar backgrounds (you shouldn’t group Java application programmers with assembly language system programmers, for example). Each participant is asked to select two of his or her own programs to be reviewed. One program should be representative of what the participant considers to be his or her finest work; the other should be a program that the programmer considers to be poorer in quality.

38. Peer Ratings Once the programs have been collected, they are randomly distributed to the participants. Each participant is given four programs to review. Two of the programs are the “finest” programs and two are" poorer” programs, but the reviewer is not told which is which. Each participant spends 30 minutes with each program and then completes an evaluation form after reviewing the program. After reviewing all four programs, each participant rates the relative quality of the four programs. The evaluation form asks the reviewer to answer, on a scale from 1 to 7 (1 meaning definitely “yes,” 7 meaning definitely “no”), such questions as these: • Was the program easy to understand? • Was the high-level design visible and reasonable? • Was the low-level design visible and reasonable? • Would it be easy for you to modify this program? • Would you be proud to have written this program?

39. Peer Ratings The reviewer also is asked for general comments and suggested improvements. After the review, the participants are given the anonymous evaluation forms for their two contributed programs. The participants also are given a statistical summary showing the overall and detailed ranking of their original programs across the entire set of programs, as well as an analysis of how their ratings of other programs compared with those ratings of other reviewers of the same program. The purpose of the process is to allow programmers to self-assess their programming skills.

40. Assignment # 2 • Q: 1 Explain the following • Active design reviews • Humphrey’s Inspection Process • N-Fold inspections • Two-Person Inspection • Verification-Based Inspection • Phased Inspection • Q: 2 Explain the statistical software quality assurance in detail

41. Any question