Chapter 3 STATIC TECHNIQUES

1. Chapter 3STATIC TECHNIQUES

2. 3.1. Static Techniques and The Test Process • Static Testing • Testing of a component or system at specification or implementation level without execution of that software • i.e., reviews and static analysis • Software work products are examined manually, or with a set of tools, but not executed. • Dynamic Testing • Testing that involves the execution of the software or a component • Software is executed using a set of input values and its output is then examined and compared to what is expected • Dynamic testing can only be applied to software code

3. 3.1. Static Techniques and The Test Process • Dynamic testing and static testing are complementary methods • They tend to find different types of defects effectively • Types of defects that are easier to find during static testing • Deviations from standards • Missing requirements • Design defets • Non-maintainable code • Inconsistent interface specification • Compared to dynamic testing, static testing finds defects rather than failures

4. 3.1. Static Techniques and The Test Process • Advantages of static testing • Early feedback on quality issues • Low rework costs • Increase in productivity • Exchange of information • Increased awareness of quality issues

5. 3.2. Review Process • Informal review • A review not based on a formal (documented) procedure • Informal reviews are not documented • Formal review • A review characterized by documented procedures and requirements • i.e., inspection

6. 3.2.1. Phases of a formal review • Planning • Kick-off (optional) • Preparation • Review meeting • Rework • Follow-up

7. Planning • The Foundation Syllabus specifies the following elements: • Define the review criteria • Select the personnel • Allocate roles • Define the entry and exit criteria for more formal review types • Select which parts of documents to review • Check entry criteria

8. Planning • Moderator • The leader and main person responsible for an inspection or other review process • Entry Criteria • The set of generic and specific conditions for permitting a process to go forward with a defined task • i.e., a document containing too many mistakes should not be reviewed • Reviewer (Inspector) • The person involved in the review that identifies and describes anomalies in the product or project under review • Reviewer roles • Type 1: Focus on higher-level documents • i.e., does the design comply to the requirements? • Type 2: Focus on standards • i.e., naming conventions • Type 3: Focus on related documents at the same level • i.e., interfaces between software functions • Type 4: Focus on usage of the document • i.e., maintainability

9. Kick-off (optional) • The Foundation Syllabus specifies the following elements: • Distributing documents • Explaining the objectives, process, and documents to the participants • At customer sites, • «We have measured results of up to 70% more major defects found per page as a result of performing a kick-off» [van Veenendaal and van der Zwan, 2000]

10. Preparation • The Foundation Syllabus specifies the following elements • Preparing for the review meeting by reviewing the documents • Noting potential defects, questions and comments • Participants work individually on the document under review using the related documents, procedures, rules and checklists • A critical success factor for a thorough preparation is the number of pages checked per hour (called checking rate) • A mix of factors such as the type of the document, complexity, experience of the reviewer, the number of related documents • 5-10 pages per hour but may be much less for formal inspection e.g, 1 page per hour

11. Review Meeting • Foundation Syllabus defines the following elements • Discussing or logging, with documented results or minutes (for more formal review types) • Noting defects, making recommendations regarding handling the defects, making decisions about the defects • Examining, evaluating, and recording issues during any physical meetings • Review meeting consists of the following elements • Logging phase: • Discussion phase: • Decision phase:

12. Review Meeting • Every defect and its severity should be logged • The participant who identifies the defect proposes the severity • Severity Classes • Critical • Major • Minor • Spelling errors are not part of defect classification • They are noted and given to the author at the end of the meeting

13. Review Meeting • Logging phase: The focus is on logging as many defects as possible. In a disciplined formal meeting, the logging rate should be between 1-2 defects logged per minute • Discussion phase: Participants take part in the discussion and moderator takes care of people issues. i.e., the moderator prevents discussions from getting too personal, calls for a break to cool down «heated» discussions. The outcome of discussions is documented for future reference • Decision phase: Adecision on the document has to be made by the participants, sometimes based on formal exit criteria. The most important exit criterion is the average number of critical defects found per page (i.e., no more than three critical defects per page). If the number of defects exceeds a certain level, the document must be reviewed again.

14. Rework • The Foundation Syllabus specifies the following elements • Fixing defects found (typically done by the author) • Recording updated status of defects (in formal reviews) • Not every defect that is found leads to rework • Changes that are made to the document should be easy to identify during follow-up. • Therefore, the author has to indicate where changes are made (i.e., using «Track Changes» in Word)

15. Follow-up • The Foundation Syllabus specifies the following elements • Checking that defects have been addressed • Gathering metrics • Checking exit criteria (for more formal review types) • The moderator is responsible for ensuring that satisfactory actions have been taken on all logged defects, process improvement suggestions and change requests • In order to optimize the review process, a number of measurements are collected by the moderator • i.e. Number of defects found, number of defects found per page, time spent checking per page, total review effort

16. 3.2.2. Roles and Responsibilities • Within a review team, 4 types of participants exist • Moderator • Author • Scribe (or recorder) • Moderator • In addition, management needs to play a role in the review

17. 3.2.3. Types of Review • Walkthrough • A step-by-step presentation by the author of a document in order to gather information and to establish a common understanding of content • Especially useful if people from outside the software disciplines are present • Especially useful for higher-level documents such as requirement specs. • Technical review • A discussing meeting that focuses on achieving consensus about the technical content of a document • Compared to inspections, technical reviews are less formal and there is little or no focus on defect identification • Often performed as a peer review without management participation • Peer review: A review of a work product by colleagues of the producer for the purpose of identifyin defects and improvements. • Inspection • Most formal review type • Defects found are logged and any discussion is postponed until the discussion phase

18. Success Factors for Reviews • Find a «champion» • One who will lead the process on a project or organizational level • Pick things that really count • Review the highly critical documents like SRS, and architecture • Pick the right techniques • The correct type of review must be selected • Don’t try to review everything by inspection. Some documents may only warrant an informal review • Train participants • Train participants in review techniques, esp. inspections • Special training is needed for the moderators to prepare them for their critical role in the review process

19. Success Factors for Reviews • Manage people issues • Make the review a positive experience for the author • Managers must commit not to use metrics or other information from reviews for the evaluation of the author • Follow the rules but keep it simple • Do not become too detailed • Checklists and roles are recommended • Continously improve process and tools • Report results • Use testers • Testers are professional pessimists • Just do it!

20. 3.3. Static Analysis by Tools • Static Analysis • An examination of requirements, design or code that differs from more traditional dynamic testing • Differences: • Performed on requirements, design or code without actually executing the software artefact being examined • Ideally performed before the types of formal reviews • Unrelated to dynamic properties of the requirements, design and code • The goal is to find defects, whether or not they may cause failures • Most static analysis tools focus on software code

21. Typical Defects • Static analysis tools find the typical defects: • Referencing a variable with an undefined value (i.e., before the variable has been properly initialized) • Inconsistent interfaces between modules and components, such as the improper use of a function, including the wrong parameters • The improper declaration of variables, or the declaration of variables that are never used • Unreachable («dead») code • Certain types of erroneous logic, such as potentially infinite loops • Highly complex functions • Various types of programming standards violations • Security vulnerabilities such as problems related to buffer overflow • When used to analyze code, static analysis tools are typically used by developers

22. Coding Standards • Coding Standard consists of • A set of programming rules • Example: Always check boundaries on an array when copying to that array • Naming conventions • Example: Classes should start with with capital C • Layout specifications • Example: Indent 4 spaces • It’s recommended that existing standards are adopted • Saves a lot of effort • Available tools for that standard

23. Code Metrics • Many different kinds of structural measures exist • Complexity metrics identify high risk, complex areas • The Cyclomatic Complexity (CC) metric is based on the number of decisions in a program • CC is important for testers • It provides an indication of the amount of testing necessary to detect a sufficient number of defects and to have adequate confidence in the system • Areas of code identified as more complex are candidates for reviews and additional dynamic tests • While there are many ways to calculate cyclomatic complexity, the easiest way is to sum the number of binary decision statements (i.e., if, while, for, etc.) and add 1 to it.

24. Cyclomatic Complexity • Cyclomatic complexity: • The number of independent paths through a program • CC = L – N + 2 L: the number of edges/links in a graph N: the number of nodes in a graph P: the number of disconnected parts of the graph (i.e., a subroutine) Control flow: A sequence of events in the execution through a component or system

25. Example IF A = 354 THEN IF B > C THEN A = B ELSE A = C ENDIF ENDIF PRINT A 8 -7 + 2 = 3 OR 1+ 1 (IF)+ 1 (IF) = 3

26. Example

27. Example CC = 5 1 + 1 (while) + 1 (&&) + 1 (&&) + 1 (if) = 5 Practical Method: Start with 1 and add 1 when you see IF, WHILE, FOR, &&, || and also for each case in a switch statement

28. Example Independent Paths: 1, 8 1, 2, 3, 7b, 1, 8 1, 2, 4, 5, 7a, 7b, 1, 8 1, 2, 4, 6, 7a, 7b, 1, 8

29. Industrial Threshold Value for CC SEI, Carnegie Mellon University www.sei.cmu.edu/str/descriptions/cyclomatic_body.html NASA  20

30. Code Structure • Several aspects of code structure to consider • Control Flow Structure • Addresses the sequence in which the instructions are executed • Reflects the iterations and loops in a program’s design • Can be used to identify unreachable (dead) code • Many code metrics such as CC relate to control flow structure • Data Flow Structure • Follows the trail of data item as it is accessed and modified by code • Some defects can be found with this structure • Variables that are never used • Referencing a variable with an undefined value • Data Structure • Sometimes a program is complex because it has a complex data structure, rather than because of complex control or data flow