Chapter 14 Testing Tactics

1. Chapter 14Testing Tactics

2. V-Cycle 软件任务 交付软件 验证与确认 系统需求 系统测试 系统需求 验证 测试后的软件 验证与确认 软件需求 合格性测试 软件需求 验证 集成的软件 验证 概要设计 集成测试 概要设计 验证 测试后的单元 验证 详细设计 单元测试 详细设计 验证 编译后的单元 编码

3. Three Examples 防不胜防的软件错误 —— 例1：1963年, 美国, 飞往火星的火箭爆炸, 损失$ 10 million. 原因: FORTRAN循环 DO 5 I = 1, 3 误写为 DO 5 I = 1.3

4. 例2：1996年，美国Washington Post 报导 Dallas, Aug. 23 — 美国航空公司一架直升飞机撞上山崖，造成163人死亡。 原因：飞行目的地代码输入错误。 驾驶Boeing 757的机长认为他输入了目的地Cali的单字符代码，而在南美的航空资料中， Cali的单字符代码和Bogota的一样，所以系统将目的地处理成了Bogota。但在大多数计算机数据库中Cali和Bogota的代码是不同的。

5. 例3：[摘自Pfleeger 书中的“REAL-TIME EXAMPLE”] European Space Agency (ESA) 的Ariane-5航空运载火箭在June 4, 1996她第一次飞行中远程失控，造成火箭本身和4棵卫星损失，$500 million 原因： 其前身Ariane-4声誉良好，在开发Ariane-5时，忽略了一个需求细节， Ariane-5与Ariane-4的轨道不同。

6. Outline • The Concepts of Testing • Software Testing • Testing Objectives • Testing Principles • Testability • Test Case Design • Whiter-box Testing • Basis Path Testing • Control Structure Testing • Black-box Testing • GUI Testing

7. The Concepts of Testing

8. The Relationship Between Testing and Development

9. What is Software Testing? Testing is the process of exercising a program with the specific intent of finding errors prior to delivery to the end user. Software testing is a critical element of software quality assurance and represents the ultimate review of specification, design, and code generation.

10. The Flow of Software Testing

11. What Testing Shows errors requirements conformance performance an indication of quality

12. What is a “good” test? • It has a high probability of finding an error. • It is not redundant. • It should be “best of breed”. • It should be neither too simple, nor too complex.

13. Who Tests the Software? developer independent tester Understands the system Must learn about the system, but, will test "gently" but, will attempt to break it and, is driven by quality and, is driven by "delivery"

14. Testing Objectives • Testing is a process of executing a program with the intent of finding an error. • A good test case is one that has a high probability of finding an as-yet undiscovered error. • A successful test is one that uncovers an as-yet undiscovered error.

15. Testing Principles • All tests should be traceable to customer requirements. • Tests should be planned long before testing begins. • The Pare-to (80% to 20%) principle applies to software testing. • Testing should begin “ in the small” and progress toward testing “ in the large”. • Exhaustive testing is not possible. • To be most effective, testing should be conducted by an independent third party.

16. Testability • Operability—it operates cleanly • Observables—the results of each test case are readily observed • Controllability—the degree to which testing can be automated and optimized • Decomposability—testing can be targeted • Simplicity—reduce complex architecture and logic to simplify tests • Stability—few changes are requested during testing • Understandability—of the design

17. Exhaustive Testing loop < 20 X 14 There are 10 possible paths! If we execute one test per millisecond, it would take 3,170 years to test this program!!

18. Selective Testing Selected path loop < 20 X

19. Software Testing black-box methods white-box methods Methods Strategies

20. Methods of Testing • 动态测试 • 白盒测试 • 基本路径测试 • 控制结构测试 • 黑盒测试 • 功能分解 • 等价类划分 • 边值分析 • 因果图 • 随机测试 • 猜错法 • 静态测试 • 静态分析 • 代码审查(Code Reading Review) • 代码走查(Walkthroughs) • 技术评审(Technical Review) • 桌面检查(Desk Checking)

21. 静态测试 • 代码审查：小组集体阅读讨论检查代码 • 代码走查：小组集体用“脑”执行并检查代码 • 桌面检查：由程序员阅读自己编写的程序 • 技术评审：会议形式讨论检查代码 • 静态分析：对代码的机械性、程式化的特性分析方法，包括控制流分析、数据流分析、接口分析、表达式分析

22. Test Case Design

23. Test Case Design "Bugs lurk in corners and congregate at boundaries ..." Boris Beizer OBJECTIVE to uncover errors CRITERIA in a complete manner CONSTRAINT with a minimum of effort and time

24. Whiter-Box Testing

25. What is White-Box Testing? white-box , or open-box, clear-box testing: Use the structure of the program to test. —— Structural testing ... our goal is to ensure that all statements and conditions havebeen executed at least once ...

26. Why Cover? logic errors and incorrect assumptions are inversely proportional to a path's execution probability we often believe that a path is not likely to be executed; in fact, reality is often counter intuitive typographical errors are random; it's likely that untested paths will contain some

27. Basis Path Testing

28. What is Basis Path Testing? • It is a white-box testing technique first proposed by Tom McCabe. • It enables the test case designer to derive a logical complexity measure of a procedural design and use this measure as a guide for defining a basis set of execution paths. • Test cases derived to exercise the basis set are guaranteed to execute every statement in the program at least one time during testing.

29. Flow Graph Notation • The structured constructs in flow graph form: • Sequence; • If; • While; • Until; • Case; • Where each circle represents one or more non branching PDL or source code statements.

30. Flow Graph

31. Flow Chart and Flow Graph

32. An Example of Flow Graph

33. Cyclomatic Complexity （秩复杂性） • Cyclomatic complexity is a software metric that provides a quantitative measure of the logical complexity of a program. • An independent path is any path through the program that introduces at least one new set of processing statements or a new condition.

34. Computing Complexity • There are three ways: • The number of regions of the flow graph correspond to the cyclomatic complexity; • Cyclomatic complexity, V(g), for a flow graph G is defined as: • V(G) = E - N + 2 • where E is the number of flow graph edges, N is the number of flow graph nodes; • Cyclomatic complexity, V(G), for a flow graph G is also defined as: • V (G) = P + 1 • where P is the number of predicate nodes contained in the flow graph G.

35. An Example First, we compute the cyclomatic complexity: V(G) = 11 edges - 9 nodes + 2 = 4 or V(G) = 3 predicate nodes + 1 = 4 In this case, V(G) = 4

36. A Conclusion A number of industry studies have indicated that the higher V(G), the higher the probability or errors. modules V(G) modules in this range are more error prone

37. Deriving Test Cases • The basis path testing method can be applied to a procedural design or to source code. • A series of steps: • 1. Using the design or code as a foundation, draw a corresponding flow graph. • 2. Determine the cyclomatic complexity of the resultant flow graph. • 3. Determine a basis set of linearly independent paths. • 4. Prepare test cases that will force execution of each path in the basis set.

38. 1 2 3 4 5 6 7 8 An Example Next, we derive the independent paths: Since V(G) = 4, there are four paths Path 1: 1,2,3,6,7,8 Path 2: 1,2,3,5,7,8 Path 3: 1,2,4,7,8 Path 4: 1,2,4,7,2,4,...7,8 Finally, we derive test cases to exercise these paths.

39. you don't need a flow chart, but the picture will help when you trace program paths count each simple logical test, compound tests count as 2 or more basis path testing should be applied to critical modules Basis Path Testing Notes

40. 1 2 3 4 5 a 1 2 3 4 5 b d f c e g Graph Matrices It is a square matrix whose size is equal to the number of nodes on the flow graph. 1 a 1-1=0 3 e b 2-1=1 f 4 5 d 2-1=1 c g 2 2-1=1 3+1=4 Cyclomatic complexity

41. Control Structure Testing

42. Control Structure Testing • There are other broaden testing coverage and improve quality of white-box testing: • Condition testing; • Data flow testing; • Loop testing;

43. Condition Testing • Condition testing is a test case design method that exercises the logical conditions contained in a program module. • Condition testing focuses on testing each condition in the program. The errors include the following: • Boolean operator error; • Boolean variable error; • Boolean parenthesis error; • relational operator error; • arithmetic expression error;

44. Condition Testing Strategies • Branch testing; • Domain testing; • Branch and Relational Operator (BRO);

45. Data Flow Testing • The data flow testing method selects test paths of a program according to the locations of definitions and uses of variables in the program. • Data flow testing strategies are useful for selecting test paths of a program containing nested if and loop statements.

46. Loop Testing Simple loop Nested Loops Concatenated Loops Unstructured Loops

47. Loop Testing: Simple Loops Minimum conditions—Simple Loops 1. skip the loop entirely 2. only one pass through the loop 3. two passes through the loop 4. m passes through the loop m < n 5. (n-1), n, and (n+1) passes through the loop where n is the maximum number of allowable passes

48. Loop Testing: Nested Loops Nested Loops Start at the innermost loop. Set all outer loops to their minimum iteration parameter values. Test the min+1, typical, max-1 and max for the innermost loop, while holding the outer loops at their minimum values. Move out one loop and set it up as in step 2, holding all other loops at typical values. Continue this step until the outermost loop has been tested. Concatenated Loops If the loops are independent of one another then treat each as a simple loop else* treat as nested loops endif* for example, the final loop counter value of loop 1 is used to initialize loop 2.

49. a A>1 and B=0 YES c NO X=X/A b A=2 or x>1 YES e X=X+1 NO d Example func(int A,B,X) { if((A>1)&&(B=0)) {X:=X/A}; if((A=2)||(X>1)) {X:=X+1}; }

50. a A>1 and B=0 YES c NO X=X/A b A=2 or x>1 YES e X=X+1 NO d Sentences Coverage • 取A=2，B=0，X=3