1 / 20

Lecture 2 May 21, 2013 Software Testing Research Dr. Sergiy Vilkomir

2013 REU Program at ECU Software Testing - Foundations, Tools, and Applications. Lecture 2 May 21, 2013 Software Testing Research Dr. Sergiy Vilkomir. Testing can show that defects are present, but cannot prove that there are no defects.

haley-reeves
Download Presentation

Lecture 2 May 21, 2013 Software Testing Research Dr. Sergiy Vilkomir

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. 2013 REU Program at ECUSoftware Testing - Foundations, Tools, and Applications • Lecture 2 May 21, 2013 • Software Testing Research • Dr. SergiyVilkomir

  2. Testing can show that defects are present, but cannot prove that there are no defects. A restricted number of test cases should be selected There are different ways (testing methods/strategies/criteria/approaches) to select test cases There are no method that guarantee that software is 100% correct Exhaustive testing is impossible

  3. Pair-Wise Pair-Wise: A value of each level for each factor must be combined with a value from every levels for each other factor. In other words, we mustcover all possible pairs of values. Pair-wise testing provides a small set of test cases. It is a practical alternative to testing all combinations. Why? One test case covers several pairs

  4. Pair-Wise Pair-Wise: A value of each level for each factor must be combined with a value from every levels for each other factor. Exercise: Choose one pair and find a test case which covers this pair

  5. Pairwise designs: Example Suppose that a program to be tested requires 3 inputs, one corresponding to each input variable. Each variable can take only one of two distinct values. Considering each input variable as a factor, the total number of factor combinations is 23. Let X, Y, and Z denote the three input variables and {X1, X2}, {Y1, Y2}, {Z1, Z2} their respective sets of values. All possible combinations of these three factors follow.

  6. Pairwise designs: Reducing the combinations Now suppose we want to generate tests such that each pair appears in at least one test. There are 12 such pairs: (X1, Y1), (X1, Y2), (X1, Z1), (X1, Z2), (X2, Y1), (X2, Y2), (X2, Z1), (X2, Z2), (Y1, Z1), (Y1, Z2), (Y2, Z1), and (Y2, Z2). The following four combinations cover all pairs: The above design is also known as a pairwise design. It is a balanced design because each value occurs exactly the same number of times. There are several sets of four combinations that cover all 12 pairs.

  7. Pairwise combinatorial testing • Pairwise combination (instead of exhaustive) • Generate combinations that efficiently cover all pairs • Rationale: most failures are triggered by single values or combinations of a few values. Covering pairs reduces the number of test cases, but reveals most faults

  8. Example: Display Control The total number of combinations – 432 (3x4x3x4x3) test cases

  9. Pairwise combinations: 17 test cases Exercise: Choose one pair and find a test case which covers this pair

  10. Number of test cases

  11. In-class exercise 4 x 3 x 2 = 24 Dest - M: 4 x 3 = 12 Dest – Direct: 4 x 2 = 8 M – Direct: 3 x 2 = 6 Total: 26 Start with factors with max numbers of levels: Dest - M: 4 x 3 = 12 We need min 12 test cases Consider software with input parameters: • Dest: London, Paris, Chicago, NY • M: 1000, 2000, 3000 • Direct: Y, N • Create test cases according to pair-wise approach. • Number of all possible combinations? • Number of pairs? • Minimal number of test cases?

  12. In-class exercise • Dest: London, Paris, Chicago, NY • M: 1000, 2000, 3000 • Direct: Y, N

  13. In-class exercise • Dest: London, Paris, Chicago, NY • M: 1000, 2000, 3000 • Direct: Y, N

  14. In-class exercise • Dest: London, Paris, Chicago, NY • M: 1000, 2000, 3000 • Direct: Y, N

  15. t-Wise t-Wise: A value of each level for each group of t factors must be combined. Sometimes: t-way (3-way, 4-way, etc) A natural extension is to require combinations of tvalues instead of 2 t-wise is expensive and benefits are not clear Example: 4-way 30 parameters 5 values each 3,800 test cases (could be too many)

  16. Combinatorial Methods in Software Testing Presentation by Rick Kuhn, (NIST) at ECU, March 22, 2012 http://core.ecu.edu/STRG/seminars.html National Institute of Standards and Technology (NIST) http://csrc.nist.gov/groups/SNS/acts/index.html

  17. Software Testing Study Software Testing is a part of ECU MSc SE program • SENG 6265 Foundations of Software Testing • SENG 6270 Software Verification and Validation

  18. Software Testing Research New Scientific approach: investigation, analysis, comparison, justification, etc. Publications Research - ?

  19. http://core.ecu.edu/STRG/

  20. Project: Testing of Mobile Applications Android Development Tools Monkeyrunner tools from Android SDK Case study (testing) Robotium Calabash-Android Comparison, analysis Pair-wise approach for mobile testing Cloud Testing of Mobile Systems (CTOMS) framework

More Related