1 / 28

Exploiting Weights of Test Cases to Enhance Fault Localization

Exploiting Weights of Test Cases to Enhance Fault Localization. Yihan Li,Chao Liu, Zi Yuan Beihang University, Beijing,China. Background. Developers more or less introduce bugs during software development

cicely
Download Presentation

Exploiting Weights of Test Cases to Enhance Fault Localization

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Exploiting Weights of Test Cases to Enhance Fault Localization YihanLi,Chao Liu, Zi Yuan Beihang University, Beijing,China

  2. Background • Developers more or less introduce bugs during software development • Software debugging is employed to remove bugs so as to improve quality of software • Inference on fault location • Fault repair • Verify repair • Fault localization is one of the most expensive tasks in debugging.

  3. Background • Spectrum based fault localization(SFL)technique are proposed • The number of failing tests • The number of passing tests • statistical formula • Exploit how program entities are correlated with program failure

  4. Background • Four coverage statistics are collected for each program entity (statement,branch,function,etc) • aef(s): the number of failed test cases that covered statement s • anf(s):the number of failed test cases that do not coverstatement s • anp(s): the number of successful test cases that cover statement s • aep(s): the number of successful test cases that do not cover statement s

  5. Statistical Formula

  6. Motivation • Two statements that covered by the same number of failed and passed tests can’t be distinguished. • The number of failed tests is often relatively smaller than that passed tests Only use the number of passing and failing tests

  7. Our work • Propose an approach to quantify weights for different tests and incorporated into existing techniques • Compare the effectiveness of the refinement techniques with original ones

  8. Approach Four weights with respect to each statement are defined • Nef(s): the total sum of weights of failed test cases that cover statement s • Nnf(s): the total sum of weights of failed test cases that do not cover statement s • Nep(s): the total sum of weights of successful test cases that cover statement s • Nnp(s): the total sum of weights of successful test cases that do not cover statement s

  9. Weights of Failed Tests ForNef(s) and Nnf(s), two factors are considered: • Weight of coverage of failed test with respect to statement • Weight of status of tests with respect to statement

  10. Weight of Coverage of test • For a failed test ti that cover statement s, the following weight is defined about coverage: where m is the total number of basic block in program and bri is value in basic block vector<b1i,b2i,b3i,…,bmi>. The vector record the block information for test ti, where block bj is 1 if it is covered by tiotherwise 0.

  11. Weight of Coverage of test • Similar, for a failed test ti that dose not cover statement s, the following weight is defined about coverage:

  12. Weight of status of tests • In extensive testing, if a few tests failed, these failed tests provide more information for error diagnosis. • On the other hand, if a few tests passed, these passed tests provide more information for error diagnosis. • In information theory, information quantity is used to measure how much information an event contains. • An even that occurs with a small probability has large information quantity.

  13. Weight of status of tests • For a failed testti that cover statement s and statement s is executed by h passed tests and k failed tests, the following weight of status of test is computed: Rci(s) = -log(k/(h+ak))= log((h+ak)/k) • Similarly, if a statement is not covered by a failed test ti, thefollowing weight of status of test is computed: Rui(s) = -log((|Tf|-k)/(| Tp|-h+a*(|Tf|-k)) = log((|Tp|-h+a*(|Tf|-k))/(|Tf|-k)) Where Tp is a set of all passing tests in program and Tf is a set of all failing tests in program

  14. Compution of Nef and Nnf For each failing test pi and a statement s • If statement s is covered by test pi, Wpi(s) = Eci(s) * Rci(s) • If statement s is not covered by test pi Wpi( s ) = Eui( s ) * Rui ( s ) • Nef(s) = Σ Wpi(s) = ΣEci(s) * Rci(s) • Nnf(s) = Σ Wpi( s ) = ΣEui(s) * Rui(s)

  15. Weight of Passed tests • Since no definiteconclusion can be made about activation of fault, weight of every passed testis 1. • Nep(s) = |{ j | test tj is a passing test and cover statement s}| • Nnp(s) = |{j | test tj is a passing test and do not cover statement s}|

  16. Revised Formula

  17. Experiments Setup

  18. Evaluation Metric

  19. Comparison • Expense of original techniques • Expense of techniques proposed by Lee Naish • Weight of failed test is inversely proportional to the number of suspect statements executed in the test minus one • Expense of revised techniques

  20. Experiments for Tarantula

  21. Experiments for Jaccard

  22. Experiments for Ochiai

  23. Experiments for SBI

  24. Mean Expense on each program

  25. Conclusion • The revised techniques consistently outperform the original techniques • On average, the revised techniques can locate 20.5% more faults than original ones by examined no more than 5% of all the code.

  26. Related Works • Wong et. al proposed some heuristic strategy to assign different weights for passed and failed tests respectively. • Lee Naish et. al also proposed a weighting strategy for failed tests. The rationale behind the idea is that failed tests that cover few statements provide more information than other failed tests. • Bandyopadhyay et.al proposed a proximity based weighting method for passing test cases. The weights of passing test cases are assigned based on the measure of proximity.

  27. Future work • Conduct more empirical studies by using larger scale programs and multiple-faults versions • Explore other factors that may impact on weights of test cases • Apply weighting techniques to other fault localization techniques • Compare with other fault localization techniques

  28. Thanks

More Related