Chapter 9: Path Testing

1. Chapter 9: Path Testing Csci 565 Fall 2012

2. Objectives • Decision-to-decision path (DD-Paths) • Test Coverage Metrics • Basis Path Testing • Observation on McCabe Basis Path Method • Essential Complexity

9. f t t f f f f

10. Test Selection Criteria (white box) • Test Selection coverage Criteria • Help to measure the adequacy of a test suite • E.g., if we use statement coverage to test 40% of the code, it means 60% of code was never executed • Help to decide when to STOP testing • E.g., use other criteria to cover 85% • Coverage? • refers to the extent by which a given verification activity has satisfied its objectives • Coverage • a measure, not a method or a test • expressed as the percentage of an activity that is accomplished.

11. Types of Structural Coverage • Typically structural coverage criteria are divided into two types: • Data flow • control flow. • Data flow criteria? • measure the flow of data between variable assignments and references to the variables • Control flow criteria? • measure the flow of control between statements and sequences of statements

12. Overview of the families of test selection criteria • Families of test selection criteria include • Structural model coverage criteria • All statements (C0) • All Decisions (C1) (IBM) • All paths (C) • C  C1  C0 • Condition coverage (CCC) • A test set achieves CCC when each individual condition in the decision is tested for both true and false results • Multiple condition coverage (CMCC) • A test set achieves CMCC if it exercises all possible combinations of condition outcomes in each decision • For example, for N conditions, it requires 2N tests (truth table) • Dependent pairs Cd (Define/Usage paths)

13. Statement testing (C0) • Let T be a test suite for a Program P, then T satisfies the statement criterion for P, iff, for each statement S of P, there exists at least one test case in T that causes the execution of S. • In terms of Flow graph model of program P, it is the same as visiting each single node on some execution path exercised by a test case in T • C0 = number of executed statements/ total number of statements

14. Statement testing: Small Java Program

15. Statement testing: Flow Graph C0: path ace T1: A=2, B=0, X=3

16. Decision testing (DD-path C1) • Let T be a test suite for a program P. T satisfies the decision adequacy criterion for P, iff, • for each branch B (or predicate P), there exists at least test case in T that causes execution of B. • This is the same as stating that every edge in the flow graph model of program P belongs to some execution path exercised by a test case in T • C1 = number of executed branches/ total number of branches • T satisfies the branch adequacy criterion if C1 =1

17. Decision Testing: Flow Graph C1: path1: acd and path2: abe T1: A=3, B=0, X=1 T2: A=2, B=1, X=1

18. Condition testing C1P • decision coverage is good for exercising faults in the way a computation has been decomposed into cases. • Condition coverage takes this decomposition in more detail, forcing execution of not only possible outcome of a Boolean expression but also of different combinations of the individual conditions in compound Boolean expression • A test suite T for a program P covers all C1P iff • for each atomic condition in P, it has at least two test cases in T : one forcing P to have true out come and the other one forcing P to have a false outcome • C1P = total # of truth values for all basic condition/ 2*number of atomic conditions • T satisfies the branch adequacy criterion if C1 =1

19. Condition Testing: Flow Graph Four conditions: A>1, B=0, and A=2, X>1 Need test cases to force when 1) A>1, and A<=1 2) B=0, and B<>1 3) A=2, and A<> 2 4) X>1, and X<=1 Test cases: T1: A=2, B=0, X=4 T2: A=1, B=1, X=1 How about using the following test cases? T3: A =1, B=0, X>3 T4:A=2, B=1, X=1 They cover all condition but not all decision

20. Multiple (compound) condition coverage CMCC • A more complete extension that includes both the basic condition and branch adequacy criteria is called the multiple condition coverage CMCC • CMCC requires a test case for each possible evaluation of compound conditions. • For N basic conditions, we need 2N combinations of N basic conditions • Short-circuit evaluation is effective in reducing the above number to a more manageable number.

21. Statement testing: Multiple Conditions Test Case Input values:

22. Example 1: Compound Condition

23. Example 2: Compound Condition

24. Condition/decision adequacy criterion CCD • Branch and condition criterion refers to the extension that include both condition and branch adequacy criteria. • A test suite T satisfies the CCD criterion iff • its satisfies both the decision criterion and the condition criterion

25. Statement testing: Machine Code

26. Modified Condition/Decision Coverage (MC/DC) • MC/DC • Requires that each basic condition be shown to independently affect the outcome of each decision • a condition has independent effect when that condition alone determines the outcome of the decision (AKA unique-cause approach) • For each basic condition C, select two test cases such that the truth values of all evaluated conditions except C are the same, and the compound condition as a whole evaluates to True for one of those test cases and False for the other • MC/DC is mandated by RTCA/Do-178B

27. Note about: Condition, Decision, and MC/DC • Condition? • A Boolean expression containing no Boolean operators. • E.g., A>B • Decision? • A Boolean expression composed of conditions and zero or more Boolean operators. (AND, OR) • (A>B OR (C=B AND D=H)) • A decision without a Boolean operator is a condition. • If a condition appears more than once in a decision, each occurrence is a distinct condition.

28. Example 3: MC/DC

29. Example 4: MC/DC Z:= (A or B) AND (C or D)

30. Path testing C (or P) • A test suite T for a program P satisfies the path adequacy criterion iff • for each path pi of P, there exists at least one test case in T that causes the execution of pi • This is the same as stating that every path in the flow graph model of program P is exercised by at least a test case in T. • C = # of executed paths/total number of paths • Cannot be achieved because the number of paths in a program having loop is unbounded, so this criterion cannot be satisfied for any but the most trivial programs.

31. Basis Path Testing

33. ex1= p2+p3-p1 ex1= (1,0,1,2,0,0,0,0,1,0) +(1,0,0,0,1,0,0,1,0,1)-(1,0,0,1,0,0,0,0,1,0) ex1 = (1,0,1,1,1,0,0,1,0,1)=A,B,C,B,E,F,G ex2=2p2-p1 ex2=(2,0,2,4,0,0,0,0,2,0) – (1,0,0,1,0,0,0,0,1,0)= (1,0,2,3,0,0,0,0,1,0)=A,B,C,B,C,B,C,G

34. Linearly independent paths

35. McCabe’s Algorithmic Procedure • McCable’s Algorithmic Procedure can be used to identify a set of basis paths • Select a path with highest number of decision nodes • Retrace each decision in baseline path (i.e., normal path that can be arbitrary path) • Flip each decision to create a new path • E.g. • p1: A,B,C,B,E,F,G (baseline, flip A) P2: A,D,E,F,G (flip p1 at D) P3:A,D,F,G (take D) P4:A,B,E,F,G (flip p1 at B) P5:A,B,C,G (flip p1 at C) -The basis paths is different from the one in table 2 -A basis path is not required to be unique

36. Linearly independent paths

37. Observations on McCabe’s Basis Path Is testing Basis path enough? Infeasible paths

38. f t t f f f f

39. Loop boundary adequacy criterion (LC) • A test suite T for a program P satisfies the LC criterion iff for each loop l in P: • There is at least one execution in which control reaches the loop, and then the loop control condition evaluates to False the first time it is evaluated (zero time) • There is at least on e execution in which control reaches the loop and then the body of the loop is executed exactly once before control leaves the loop (one time) • There is at least on e execution in which the body of the loop is repeated more than once (many times)

40. Essential Complexity V(G)=1wellStr(P) V(G) >=10 requires substantial testing Use the characteristics of code to select appropriate criteria

43. More on White-box testing • The purpose of structural coverage analysis with the associated structural coverage analysis resolution is to complement requirements-based testing as follows: • Provide evidence that the code structure was verified to the degree required for the applicable software level; • Provide a means to support demonstration of absence of unintended functions; • no information about whether the code is doing what it is supposed to be doing as specified in the requirements • Establish the thoroughness of requirements-based testing.

44. References • Material in this presentation is taken from the following resources: • Paul C. Jorgensen. Software Testing: A Craftsman’s Approach, 3rd Edition. Auerbach Publication, 2008 • Mauro Pezze and Michal Young. Software Testing And Analysis. Wiley 2008 • Kelly j. Hayhurst et al. A Practical Tutoring on Modified Condition/Decision Coverage NASA/TM-2001-210876, May 2001.