1 / 33

System and White-Box Testing

This article explores the limitations of test-driven development (TDD) and highlights the importance of system and white-box testing. It discusses the roles of developers and testers in system testing and provides tips for effective testing. The article also introduces different coverage measures, such as statement coverage, condition coverage, and path coverage, and explains how to choose test cases based on these measures.

dpriest
Download Presentation

System and White-Box Testing

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. http://flic.kr/p/edctkJ System and White-Box Testing

  2. So you’re doing test-driven development –is it enough?

  3. From the news!

  4. So you’re doing test-driven development –is it enough? Probably not. For one, TDD emphasizes unit tests, but what happens when you stick the units together?

  5. System Testing • Hook everything together • Test it “end to end” • Treat system like a black box • Use real-world scenarios • Focus on functionality • What the customer asked for!

  6. Developers typically do unit testing(as in TDD)Who should do system testing?The developers?

  7. Developer Testing • Developers know too much • Can’t put themselves in the users shoes • Especially never system test your own code • It’s your baby

  8. Tester Testing • Dedicated testers bring fresh perspective • More motivated to find bugs

  9. System Testing + Iterative Development • Two parallel iterations • Bugs found by testers logged, scheduled in future iterations • Moving target makes testing a challenge • Communication key!

  10. Principle: The key to most problems you’ll run into in software development is COMMUNICATION.When in doubt, TALK to your team, other teams, and your customer.

  11. Other possible ways to coordinate testing Test iteration at the end Alternate dev/testing iterations What are pros/cons of these?

  12. Tips for Effective Testing • Document tests • Do tests the same each time • Establish success criteria • When can system go live? • Automate where possible • Involve the customer • Users always find problems the best testers miss

  13. Recall: Common approaches forchoosing test cases • Blackbox testing: Choose based on module’s possible inputs and outputs • Do not use code • Often test boundary cases • White-box testing: Uses internal logic to choose tests • Different levels of code coverage • Aka glass box testing, clear box testing • Regression testing: Keep tests that reveal old bugs • Rationale: “Fixed” bugs come back!

  14. Criteria for choosing test cases:Coverage Measures • Degree to which the source code of a program is tested by a test suite • Examples: • Statement coverage • Condition coverage • Path coverage Some examples will clarify, but first…

  15. Control Flow Graphs –Frequently used to calculate coverage Control Flow Graph int foo(int x, int y) { int z = 0; if ((x>0) && (y>0)) { z = x; } return z; } int z = 0; if ((x>0) && (y>0)) { true z = x; false return z; Basic blocks: straight-line pieces of code without any jumps or jump targets Jumps: control branches

  16. Statement Coverage • Set of test cases such that…Each program statement (line or basic block) is executed at least once

  17. Define a test suite that provides statement coverage for this code int foo(int x, int y) { int z = 0; if ((x>0) && (y>0)) { z = x; } return z; } ✔ Control Flow Graph int z = 0; if ((x>0) && (y>0)) { ✔ true z = x; ✔ false return z;

  18. Now try this code… public int bar(int x) { int result=0; if (x < 69) { if (x % 2 == 0) { result = x/2; } else { result = x*2; } } else if (x > 6969) { x = 69; } else { x = 6969; } return result; } Control Flow Graph int result=0; result = x*2; F T x < 69 x % 2 == 0 T F result = x/2; T x > 6969 x = 69; F return result; x = 6969;

  19. Now try this code… Control Flow Graph ✔ ✔ int result=0; result = x*2; ✔ F ✔ T x < 69 x % 2 == 0 T ✔ F result = x/2; ✔ ✔ T x > 6969 x = 69; ✔ F return result; ✔ x = 6969;

  20. Condition Coverage • Set of test cases such that…Each boolean expression (in control structures) evaluates to true at least once and to false at least once

  21. Define a test suite that provides condition coverage for this code int foo(int x, int y) { int z = 0; if ((x>0) && (y>0)) { z = x; } return z; } Control Flow Graph ✔ int z = 0; if ((x>0) && (y>0)) { ✔ true z = x; false return z;

  22. Now try this code… public int bar(int x) { int result=0; if (x < 69) { if (x % 2 == 0) { result = x/2; } else { result = x*2; } } else if (x > 6969) { x = 69; } else { x = 6969; } return result; } Control Flow Graph int result=0; result = x*2; F T x < 69 x % 2 == 0 T F result = x/2; T x > 6969 x = 69; F return result; x = 6969;

  23. Now try this code… Control Flow Graph int result=0; result = x*2; ✔ ✔ F T x < 69 x % 2 == 0 ✔ T ✔ F result = x/2; ✔ T x > 6969 x = 69; ✔ F return result; x = 6969;

  24. Path Coverage • Set of test cases such that…Each possible path through a program’s control flow graph is taken at least once

  25. Define a test suite that provides path coverage for this code int foo(int x, int y) { int z = 0; if ((x>0) && (y>0)) { z = x; } return z; } Control Flow Graph int z = 0; if ((x>0) && (y>0)) { (c) (a) true (b) z = x; false return z; • Paths: • a , b • c ✔ ✔

  26. Now try this code… public int bar(int x) { int result=0; if (x < 69) { if (x % 2 == 0) { result = x/2; } else { result = x*2; } } else if (x > 6969) { x = 69; } else { x = 6969; } return result; } Control Flow Graph int result=0; result = x*2; F T x < 69 x % 2 == 0 T F result = x/2; T x > 6969 x = 69; F return result; x = 6969;

  27. Now try this code… Control Flow Graph int result=0; result = x*2; (a) (f) F T • Paths: • a , d , f , h • a , d , g , i • a , b , e , j • a , b , c , k x < 69 (d) x % 2 == 0 ✔ (h) T (g) ✔ (b) F ✔ result = x/2; ✔ (i) T x > 6969 x = 69; (e) (j) F (c) return result; (k) x = 6969;

  28. Now for a bonus verification techniquethat is not testing

  29. Code Reviews • Code walkthrough: Developer leads a review team through code • Informal, focus on code • Code inspection: Review team checks against a list of concerns • Team members prepare offline in many cases • Team moderator usually leads http://flic.kr/p/75HG5c

  30. More on Code Reviews • Some experiments show removal of 67-85% of defects via inspections • Some consider XP’s pair programming as a kind of “code review” process, but it’s not quite the same • Why? • Can review/walkthrough requirements and design documents, not just code!

  31. BONUS QUESTION

  32. Given this code • Draw a control flow graph • Define test suites that provide: • Statement coverage • Condition coverage • Path coverage int myF(int x, int y) { while (x > 10) { x = x – 10; if (x == 10) { break; } } if (y < 20 && x%2 == 0) { y = y + 20; } else { y = y – 20; } return 2*x + y; }

  33. int myF(int x, int y) { while (x > 10) { x = x – 10; if (x == 10) { break; } } if (y < 20 && x%2 == 0) { y = y + 20; } else { y = y – 20; } return 2*x + y; } while (x > 10) F T x = x – 10; if (x == 10) F T break; if (y < 20 && x%2 == 0) F T y = y – 20; y = y + 20; return 2*x + y; Here’s the CFG

More Related