1 / 17

Testing Basics

Testing Basics. Detection probability. Methodology to evaluate testing criterion. Pick sample program, sample faults, sample operational profile, two criteria. For each criterion, randomly generate test suites that satisfy the criterion.

kellie-cotton
Download Presentation

Testing Basics

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Testing Basics Detection probability

  2. Methodology to evaluate testing criterion • Pick sample program, sample faults, sample operational profile, two criteria. • For each criterion, randomly generate test suites that satisfy the criterion. • Calculate the percentage of the suites that detect each of the faults. • The criterion that has the higher percentage is better.

  3. Triangle Example cin >> a >> b >> c ; type = “scalene”; if (a == b || a == c || b == c) type= “isosceles”; if (a == b && a == c) type = “equilateral”; if (a >= b+c || b >= a+c || c >= a+b) type=“not a triangle”; if (a <= 0 || b <= 0 || c <= 0) type=“bad input”; cout<< type;

  4. Control Flow Graph Operational profile 3,3,3 abcdegi equi 3,3,4 abcegi isos 3,3,5 abcegi isos 3,3,6 abcefgi not 3,4,3 abcegi isos 3,4,4 abcegi isos 3,4,5 acegi scal 3,4,6 acegi scal All inputs are equally likely

  5. What are the failure probability for each color (separately)? cin >> a >> b >> c ; type = “scalene”; if (a == b || a == c && b == c) type= “isosceles”; if (a == b || a == c) type = “equilateral”; if (a >= b+c || b >= a+c || c> a+b) type=“not a triangle”; if (a <= 0 || b <= 0 || c <= 0) type=“bad input”; cout<< type; BlueGreen Red

  6. TTYP – probability of detection • What is the probability of detection with one randomly chosen test case? • What is the probability of detection with two randomly chosen test cases?

  7. TTYP – per path • What is the probability of detection with one randomly chosen test case per path? • What is the probability of detection with an equal number of randomly chosen test cases?

  8. TTYP – smaller subdomains • What might be better smaller subdomains?

  9. TTYP - subdomains • Are paths the best subdomains? • Would a functional decomposition be better? • Should we re-define the term subdomain?

  10. Comparing Criteria

  11. Solving this choice • What are the assumptions? • Analogy with dice?

  12. TTYP – smaller subdomains • What might be better smaller subdomains? • Would MCC (multiple condition coverage) be better subdomains

  13. TTYP2 – C0 and C1 coverage • How do we deal with C0 and C1 coverage since they are not subdomain testing methodologies?

  14. TTYP3 • How could you estimate the det prob of C0 or C1 testing?

  15. Marble Problem • Assume that there is a bag of marbles from which marbles are drawn with replacement. • What is the maximum likelihood estimate of p (the probability of drawing a purple) marble if you draw exactly n purple marbles in a row? • ? P such that (p)n(1-p) is max or • ? P such that (p)n = .5

  16. For Tuesday, Sep 4– study through section 2.3 Evaluating Testing Methods by Delivered Reliability Frankl, Hamlet, Littlewood, Strigini IEEE TOSE Aug98

More Related