1 / 17

Test Coverage with SOTA

Humboldt University Berlin Department Of Computer Science Chair Of Software Engineering. Test Coverage with SOTA. Michael Hildebrandt. agenda. motivation testing is expensive purposes of SOTA static program analysis structure oriented program test approach source code parsing

fmaxwell
Download Presentation

Test Coverage with SOTA

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Humboldt University Berlin Department Of Computer Science Chair Of Software Engineering Test Coverage with SOTA Michael Hildebrandt

  2. agenda DAAD Workshop Neum, 2009 • motivation • testing is expensive • purposes of SOTA • static program analysis • structure oriented program test • approach • source code parsing • instrumenting • workflow • source code preparation • test execution • evaluation

  3. Software Engineering II, Prof. Dr. Holger Schlingloff motivation DAAD Workshop Neum, 2009 • given: function overlaps • parameter: • (time) interval A (startA, endA) • (time) interval B (startB, endB) • result: • true if A and B overlap, otherwise false • How many test cases exist?

  4. motivation (cont.) DAAD Workshop Neum, 2009 test cases for overlapping intervals test cases for non-overlapping intervals test cases for A is before B and vice versa test cases for A  B and B  A test cases for the same starting/ending point of A and B test case for A = B

  5. motivation (cont.) DAAD Workshop Neum, 2009 test cases for intervals consisting of one point test cases for identical intervals consisting of point test cases for intervals containing integer / real numbers test cases for invalid intervals test cases for intervals with values at the borders of number range (maxint, maxreal)

  6. motivation (cont.) DAAD Workshop Neum, 2009 about 20 test cases for this simple example imagine a project with 1.5 MLOC conclusion:testing every possible input is expensive or even impossible problem:Which subset of the possible test cases has the highest probability to detect the most possible failures? possible solution:structure oriented tests (using SOTA)

  7. SOTA author: Ronny Treyße(diploma thesis) DAAD Workshop Neum, 2009 • Structure Oriented Test and Analysis • static program analysis • metrics • control flow graph (CFG) • structure oriented tests • evaluation of test cases by code coverage • development of additional test cases

  8. static program analysis: metrics DAAD Workshop Neum, 2009 • calculated by parsing the source code • provides several metrics • lines of code • number of statements, branches, atoms, conditions, … • cyclomatic complexity • essential complexity

  9. static program analysis: CFG publicstaticint fac(int n) { int result = 1; if (n == 0 | n == 1) { return result; } else { for (int i = n; i >= 2; i--) { result *= i; } return result; } } DAAD Workshop Neum, 2009

  10. structure oriented tests DAAD Workshop Neum, 2009 • calculated by instrumenting and test execution

  11. approach DAAD Workshop Neum, 2009 • instrumentation • enrichment of the original source code by • adding logging statements • breaking up conditions, transforming loops, … • source code grows bigger (up to ten times) • logging information during test execution • values of condition, atoms • used function entries / exits and branches of control structures (reconstruction of the control flow)

  12. instrumentation: levels DAAD Workshop Neum, 2009 • level 0 • no instrumentation • to mark structures for no instrumentation • level 1 • all entries and exists of a method, branching control structures logged • all coverage measures based on the control flow computable • level 2 • additional to level 1 the assignment of every atom is logged • all coverage measures computable • level 3 • complete instrumentation: additional to level 2 the execution of every statement is logged • allows control flow analysis of abnormalterminating programs

  13. instrumentation (cont.) if (n == 0 | n == 1) { return result; } DAAD Workshop Neum, 2009

  14. instrumentation (cont.) for (int i = n; i >= 2; i--) { result *= i; } DAAD Workshop Neum, 2009

  15. modes of usage DAAD Workshop Neum, 2009 • 3 modes of usage • manual program testing • pre procession: SOTA • test execution: by hand • post procession: SOTA • testing with an external test system (e.g. ATOSj) • pre procession: SOTA • test execution: external test system • post procession: SOTA • testing with an automatic test system • usage of non-gui functionality by SOTA library • accessible via ASTManager object or command line

  16. workflow DAAD Workshop Neum, 2009 • phase 1: preparation • read in and parsing the source code • configure the instrumentation • instrumentation of source code • phase 2: test execution • compiling of the instrumented source code • test execution including the logging of information • phase 3: test evaluation • restoration of the original code • read In the log files • calculation of the coverage measure • visualization of the results • creation of report

  17. Thankyou. DAAD Workshop Neum, 2009 Questions ? Comments?

More Related