Software Testing 4

1. Software Testing 4 Damian Gordon

2. Test Approaches • Static Testing is the testing of a component or system at a specification or implementation level without execution of the software. • Dynamic Testing is the testing of software by executing the software of either a component or system. • Static Testing and Dynamic Testing are complementary methods and tend to find different types of defects effectively and efficiently.

3. Test Approaches • Static Testing detects defects such as deviations from standards, missing requirements, design defects, non-maintainable code and inconsistent interface specifications. • Dynamic Testing detects defects such as checking if outputs from the expected values.

4. Test Approaches • One key approach in Static Testing is the Review Process. • Reviews can find defects, are informational, communicational, and educational. • Participants in the review learn the content of the software systems, the role of their own work, help planning for future stages of the work. • Reviews often represent milestones, and support the establishments of a baseline for the software product.

5. The Review Process

6. Test Approaches • The type and quantity of defects found during the review stage can help focus the testing process. • In some cases customers or users attend the review process and provide feedback to the developers and document authors. • Studies have shown the reviews significantly increase productivity and product quality.

7. The Review Process • They can be either informal or formal. • The formality of the review process is related to factors such as the maturity of the development process, any legal or regulatory requirements or the need for an audit trail. • In practice most reviews are informal. • A two-person team can conduct an informal review, as the developer/author can get a colleague to review the code and documentation.

8. The Review Process • Phases of a Formal Review • Planning • Kick-Off • Preparation • Review Meeting • Rework • Follow-up

9. The Review Process • 1. Planning • The authors/developers request a review. • A moderator is assigned – this is the leader of the review process. • The project planning needs to incorporate time to undertake the review. • The planning must start with deciding an entry criteria to ensure that document is ready for review.

10. The Review Process • 1. Planning • Entry Criteria • The set of generic and specific conditions for permitting a process to go forward with a defined task, e.g. Test phase. The purpose of entry criteria is to prevent a task from starting which would entail more (wasted) effort compared to the effort needed to remove the failed entry criteria.

11. The Review Process • 1. Planning • Minimum set of criteria for performing entry check: • A short check of a product sample by the moderator does not reveal a large number of defects, e.g. After 30 minutes of checking, no more than 3 major defects are found in a single page or fewer than 10 major defects in total in a set of 5 pages. • The document to be reviewed is available with line numbers. • The document has been cleaned up by running any automated checks that apply. • References needed for the inspection are stable and available. • The document author is prepared to join the review team and feels confident with the quality of the document.

12. The Review Process • 1. Planning • The review will focus on a few different things: • Focus on higher-level documents, e.g. Does the design comply to the requirements • Focus on standards, e.g. Internal consistency, clarity, naming conventions, templates • Focus on related documents at the same level, e.g. Interfaces between software functions • Focus on usage, e.g. For testability and maintainablity

13. The Review Process • 2. Kick-Off • The review starts with a kick-off meeting, to make sure everyone is on the same wavelength regarding the document under review. • The meeting consists of a short introduction to the objectives of the review and the documents. • Role assignments, checking rate, the pages to be checked, process changes and possible other questions are discussed at this meeting.

14. The Review Process • 3. Preparation • Participants identify defects, questions, and comments, according to their understanding of the document and their role. • A checking rate is decided, which is the number of pages checked per hour, usually about 5 to 10 pages per hour, depending on complexity.

15. The Review Process • 4. Review Meeting • Usually made up of the following phases: • Logging phase • Discussion phase • Decision phase

16. The Review Process • 4. Review Meeting • Logging Phase: • The issues that have been identified in the Preparation stage are logged. • To ensure progress and efficiency, no real discussion is allowed during the logging phase. • Each defect is logged with a severity: • Critical: defects will cause downstream damage. • Major: defects could cause downstream damage. • Minor: defects are not likely to cause downstream damage.

17. The Review Process • 4. Review Meeting • Discussion Phase: • Each of the defects that require discussion are discussed, with a chairman preventing discussions from getting too personal.

18. The Review Process • 4. Review Meeting • Decision Phase: • At the end of the discussion phase, a decision is taken about the document under review. If the number of defects found per page exceeds a certain level, the document may need to be reworked, and reviewed again.

19. The Review Process • 5. Rework • Based on the defects detects, the author will improve the document under review, step-by-step.

20. The Review Process

21. Specification Terminology Checklist • Always, every, all, none, never, … (absolutely sure?) • Certainly, therefore, clearly, obviously, customarily, most, … (persuasion lingo) • Some, sometimes, often, usually, ordinarily, customarily, most, … (vague) • etc., and so forth, and so on, such as, … (not testable) • Good, fast, cheap, efficient, small, stable, … (unquantifiable) • Handled, processed, rejected, skipped, eliminated… • If … then … (missing else)

22. The Review Process • Roles and Responsibilities • The moderator • The author • The scribe • The reviewers • The manager

23. The Review Process • Roles and Responsibilities • The moderator serves as the review leader, they determine the type of review, approach and the composition of the review team. • The moderator performs the entry check, and the follow-up on the rework. • The moderator also schedules meetings, disseminates documents, leads discussions and stores relevant data.

24. The Review Process • Roles and Responsibilities • The author writes the original document and seeks to improve the quality of the document by working with others.

25. The Review Process • Roles and Responsibilities • The scribe records all of the defects during the logging meetings.

26. The Review Process • Roles and Responsibilities • The reviewers (also called checkers and inspectors) check the documents for defects. • Reviewers are chosen to represent different perspectives in the review.

27. The Review Process • Roles and Responsibilities • The manager decides on the execution of reviews and determines whether review process objectives have been met.

28. The Review Process • Types of Reviews • Walkthrough • Technical Review • Inspection

29. The Review Process • Types of Reviews • Walkthrough • The author guides the review team through the document, to achieve a common understanding and gather feedback. • This means the author does a range of preparation, and the review team don’t need to do a detailed study before the meeting. • A walkthrough is especially useful for higher-level documents, like requirements specifications and architectural documents.

30. The Review Process • Types of Reviews • Technical Review • This approach focuses on developing a consensus about the technical content of the document. • Defects are found by technical experts, who focus on the content of the document (as opposed to considering any relevant legislation and standards, referenced documents, and intended readership).

31. The Review Process • Types of Reviews • Inspection • This approach is the most formal review type. • The document is inspected thoroughly be the reviews before the meeting, comparing the work product with its sources and other referenced documents, and using rules and checklists. • The defects found are logged and any discussion is postponed until the discussion phase. • This makes the inspection meeting a very efficient meeting.

32. The Review Process • Let’s do one.