1 / 64

CONCOLIC TESTING

CONCOLIC TESTING. Pınar Sağlam. U nit Testing. In unit testing , A program is decomposed into units which are collections of functions. A part of unit can be testing by generating inputs for a single entry function. Concolic Testing. CONCRETE EXECUTION UNIT TESTING

angeni
Download Presentation

CONCOLIC TESTING

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. CONCOLIC TESTING Pınar Sağlam

  2. Unit Testing • In unit testing, A program is decomposed into units which are collections of functions. • A part of unit can be testing by generating inputs for a single entry function.

  3. Concolic Testing CONCRETE EXECUTION UNIT TESTING SYMBOLIC EXECUTION

  4. Overview • Concrete Execution • Symbolic Execution • Concolic Execution • The goal and achievements of CONCOLIC Testing • General Concolic execution behaviours • Founded bugs in SGLIB Library & Sun Microsystems’ Java Collection Framework

  5. Concrete Execution • The inputs of a unit which is tested are concrete values. • e.x: Randomly choosing The domain of potential inputs are randomly choosen. redundant probability of reaching an error can be astronomically less

  6. Existing Approach l • Random Testing • generate random inputs • execute the program on generated inputs test_me(int x){ if(x==94389){ ERROR; } } • Probability of hitting ERROR = 1/2^32

  7. Symbolic Execution • Symbolic Execution • use symbolic values for input variables • execute the program symbolically on symbolic input values • collect symbolic path constraints • use theorem prover to check if a branch can be taken • Does not scale for large programs

  8. Symbolic Execution use symbolic values for input variables execute the program symbolically on symbolic input values collect symbolic path constraints use theorem prover to check if a branch can be taken Does not scale for large programs test_me(int x){ if((x%10)*4!=17){ ERROR; } else { ERROR; } } Symbolic execution will say both branches are reachable: False positive Existing Approach II

  9. 1 0 1 0 1 0 0 1 1 1 0 1 Execution Paths of a Program • Can be seen as a binary tree with possibly infinite depth • Computation tree • Each node represents the execution of a “if then else” statement • Each edge represents the execution of a sequence of non-conditional statements • Each path in the tree represents an equivalence class of inputs

  10. Concrete Execution Path (example)

  11. Symbolic Execution Path (example)

  12. Concolic Execution • Automated Scalable Unit Testing of real-world C Programs • Generate test inputs • Execute unit under test on generated test inputs • so that all reachable statements are executed • Any assertion violation gets caught • Our Approach: • Explore all execution paths of an Unit for all possible inputs • Exploring all execution paths ensure that all reachable statements are executed

  13. Concolic Execution • Combine concrete and symbolic execution for unit testing • Concrete + Symbolic = Concolic • In a nutshell • Use concrete execution over a concrete input to guide symbolic execution • Concrete execution helps Symbolic execution to simplify complex and unmanageable symbolic expressions • by replacing symbolic values by concrete values • Achieves Scalability • Higher branch coverage than random testing • No false positives or scalability issue like in symbolic execution based testing

  14. Concolic Execution Process • CUTE generates concrete inputs one by one. • The program is executed on that input simultaneously (both concretely & symbolically) • The symb. ex. follows the path taken by the concrete ex. maintains a symb. state, generates path constraints. • The info. collected by the concrete ex. guided symb. ex. is then used to generate inputs for the next ex. • This process continues until all different feasible paths are executed once.

  15. Example typedef struct cell { int v; struct cell *next; } cell; int f(int v) { return 2*v + 1; } int testme(cell *p, int x) { if (x > 0) if (p != NULL) if (f(x) == p->v) if (p->next == p) abort(); return 0; } • Random Test Driver: • random memory graph reachable from p • random value for x • Probability of reaching abort( ) is extremely low

  16. concrete state symbolic state constraints p , x=236 NULL p=p0, x=x0 CUTE Approach Concrete Execution Symbolic Execution typedef struct cell { int v; struct cell *next; } cell; int f(int v) { return 2*v + 1; } int testme(cell *p, int x) { if (x > 0) if (p != NULL) if (f(x) == p->v) if (p->next == p) abort(); return 0; }

  17. concrete state symbolic state constraints p , x=236 NULL p=p0, x=x0 CUTE Approach Concrete Execution Symbolic Execution typedef struct cell { int v; struct cell *next; } cell; int f(int v) { return 2*v + 1; } int testme(cell *p, int x) { if (x > 0) if (p != NULL) if (f(x) == p->v) if (p->next == p) abort(); return 0; }

  18. concrete state symbolic state constraints p , x=236 NULL p=p0, x=x0 CUTE Approach Concrete Execution Symbolic Execution typedef struct cell { int v; struct cell *next; } cell; int f(int v) { return 2*v + 1; } int testme(cell *p, int x) { if (x > 0) if (p != NULL) if (f(x) == p->v) if (p->next == p) abort(); return 0; } x0>0

  19. concrete state symbolic state constraints p , x=236 NULL p=p0, x=x0 CUTE Approach Concrete Execution Symbolic Execution typedef struct cell { int v; struct cell *next; } cell; int f(int v) { return 2*v + 1; } int testme(cell *p, int x) { if (x > 0) if (p != NULL) if (f(x) == p->v) if (p->next == p) abort(); return 0; } x0>0 !(p0!=NULL)

  20. concrete state symbolic state constraints p , x=236 NULL p=p0, x=x0 CUTE Approach Concrete Execution Symbolic Execution typedef struct cell { int v; struct cell *next; } cell; int f(int v) { return 2*v + 1; } int testme(cell *p, int x) { if (x > 0) if (p != NULL) if (f(x) == p->v) if (p->next == p) abort(); return 0; } solve: x0>0 and p0NULL x0>0 p0=NULL

  21. concrete state symbolic state constraints p , x=236 NULL p=p0, x=x0 634 CUTE Approach Concrete Execution Symbolic Execution typedef struct cell { int v; struct cell *next; } cell; int f(int v) { return 2*v + 1; } int testme(cell *p, int x) { if (x > 0) if (p != NULL) if (f(x) == p->v) if (p->next == p) abort(); return 0; } solve: x0>0 and p0NULL x0=236, p0 NULL x0>0 p0=NULL

  22. concrete state symbolic state constraints p , x=236 NULL p=p0, x=x0,p->v =v0,p->next=n0 634 CUTE Approach Concrete Execution Symbolic Execution typedef struct cell { int v; struct cell *next; } cell; int f(int v) { return 2*v + 1; } int testme(cell *p, int x) { if (x > 0) if (p != NULL) if (f(x) == p->v) if (p->next == p) abort(); return 0; }

  23. concrete state symbolic state constraints p , x=236 NULL p=p0, x=x0,p->v =v0,p->next=n0 634 CUTE Approach Concrete Execution Symbolic Execution typedef struct cell { int v; struct cell *next; } cell; int f(int v) { return 2*v + 1; } int testme(cell *p, int x) { if (x > 0) if (p != NULL) if (f(x) == p->v) if (p->next == p) abort(); return 0; } x0>0

  24. concrete state symbolic state constraints p , x=236 NULL p=p0, x=x0,p->v =v0,p->next=n0 634 CUTE Approach Concrete Execution Symbolic Execution typedef struct cell { int v; struct cell *next; } cell; int f(int v) { return 2*v + 1; } int testme(cell *p, int x) { if (x > 0) if (p != NULL) if (f(x) == p->v) if (p->next == p) abort(); return 0; } x0>0 p0NULL

  25. concrete state symbolic state constraints p , x=236 NULL p=p0, x=x0,p->v =v0,p->next=n0 634 CUTE Approach Concrete Execution Symbolic Execution typedef struct cell { int v; struct cell *next; } cell; int f(int v) { return 2*v + 1; } int testme(cell *p, int x) { if (x > 0) if (p != NULL) if (f(x) == p->v) if (p->next == p) abort(); return 0; } x0>0 p0NULL 2x0+1v0

  26. concrete state symbolic state constraints p , x=236 NULL p=p0, x=x0,p->v =v0,p->next=n0 634 CUTE Approach Concrete Execution Symbolic Execution typedef struct cell { int v; struct cell *next; } cell; int f(int v) { return 2*v + 1; } int testme(cell *p, int x) { if (x > 0) if (p != NULL) if (f(x) == p->v) if (p->next == p) abort(); return 0; } x0>0 p0NULL 2x0+1v0

  27. concrete state symbolic state constraints p , x=236 NULL p=p0, x=x0,p->v =v0,p->next=n0 634 CUTE Approach Concrete Execution Symbolic Execution typedef struct cell { int v; struct cell *next; } cell; int f(int v) { return 2*v + 1; } int testme(cell *p, int x) { if (x > 0) if (p != NULL) if (f(x) == p->v) if (p->next == p) abort(); return 0; } solve: x0>0 and p0NULL and 2x0+1=v0 x0>0 p0NULL 2x0+1v0

  28. concrete state symbolic state constraints p , x=236 NULL p=p0, x=x0,p->v =v0,p->next=n0 634 3 CUTE Approach Concrete Execution Symbolic Execution typedef struct cell { int v; struct cell *next; } cell; int f(int v) { return 2*v + 1; } int testme(cell *p, int x) { if (x > 0) if (p != NULL) if (f(x) == p->v) if (p->next == p) abort(); return 0; } solve: x0>0 and p0NULL and 2x0+1=v0 x0=1, p0 NULL x0>0 p0NULL 2x0+1v0

  29. concrete state symbolic state constraints p , x=1 NULL p=p0, x=x0,p->v =v0,p->next=n0 3 CUTE Approach Concrete Execution Symbolic Execution typedef struct cell { int v; struct cell *next; } cell; int f(int v) { return 2*v + 1; } int testme(cell *p, int x) { if (x > 0) if (p != NULL) if (f(x) == p->v) if (p->next == p) abort(); return 0; }

  30. concrete state symbolic state constraints p , x=1 NULL p=p0, x=x0,p->v =v0,p->next=n0 3 CUTE Approach Concrete Execution Symbolic Execution typedef struct cell { int v; struct cell *next; } cell; int f(int v) { return 2*v + 1; } int testme(cell *p, int x) { if (x > 0) if (p != NULL) if (f(x) == p->v) if (p->next == p) abort(); return 0; } x0>0

  31. concrete state symbolic state constraints p , x=1 NULL p=p0, x=x0,p->v =v0,p->next=n0 3 CUTE Approach Concrete Execution Symbolic Execution typedef struct cell { int v; struct cell *next; } cell; int f(int v) { return 2*v + 1; } int testme(cell *p, int x) { if (x > 0) if (p != NULL) if (f(x) == p->v) if (p->next == p) abort(); return 0; } x0>0 p0NULL

  32. concrete state symbolic state constraints p , x=1 NULL p=p0, x=x0,p->v =v0,p->next=n0 3 CUTE Approach Concrete Execution Symbolic Execution typedef struct cell { int v; struct cell *next; } cell; int f(int v) { return 2*v + 1; } int testme(cell *p, int x) { if (x > 0) if (p != NULL) if (f(x) == p->v) if (p->next == p) abort(); return 0; } x0>0 p0NULL 2x0+1=v0

  33. concrete state symbolic state constraints p , x=1 NULL p=p0, x=x0,p->v =v0,p->next=n0 3 CUTE Approach Concrete Execution Symbolic Execution typedef struct cell { int v; struct cell *next; } cell; int f(int v) { return 2*v + 1; } int testme(cell *p, int x) { if (x > 0) if (p != NULL) if (f(x) == p->v) if (p->next == p) abort(); return 0; } x0>0 p0NULL 2x0+1=v0 n0p0

  34. concrete state symbolic state constraints p , x=1 NULL p=p0, x=x0,p->v =v0,p->next=n0 3 CUTE Approach Concrete Execution Symbolic Execution typedef struct cell { int v; struct cell *next; } cell; int f(int v) { return 2*v + 1; } int testme(cell *p, int x) { if (x > 0) if (p != NULL) if (f(x) == p->v) if (p->next == p) abort(); return 0; } x0>0 p0NULL 2x0+1=v0 n0p0

  35. concrete state symbolic state constraints p , x=1 NULL p=p0, x=x0,p->v =v0,p->next=n0 3 CUTE Approach Concrete Execution Symbolic Execution typedef struct cell { int v; struct cell *next; } cell; int f(int v) { return 2*v + 1; } int testme(cell *p, int x) { if (x > 0) if (p != NULL) if (f(x) == p->v) if (p->next == p) abort(); return 0; } solve: x0>0 and p0NULL and 2x0+1=v0 and n0=p0 . x0>0 p0NULL 2x0+1=v0 n0p0

  36. concrete state symbolic state constraints p , x=1 NULL p=p0, x=x0,p->v =v0,p->next=n0 3 3 CUTE Approach Concrete Execution Symbolic Execution typedef struct cell { int v; struct cell *next; } cell; int f(int v) { return 2*v + 1; } int testme(cell *p, int x) { if (x > 0) if (p != NULL) if (f(x) == p->v) if (p->next == p) abort(); return 0; } solve: x0>0 and p0NULL and 2x0+1=v0 and n0=p0 x0=1, p0 x0>0 p0NULL 2x0+1=v0 n0p0

  37. concrete state symbolic state constraints 3 CUTE Approach Concrete Execution Symbolic Execution typedef struct cell { int v; struct cell *next; } cell; int f(int v) { return 2*v + 1; } int testme(cell *p, int x) { if (x > 0) if (p != NULL) if (f(x) == p->v) if (p->next == p) abort(); return 0; } p=p0, x=x0,p->v =v0,p->next=n0 p , x=1

  38. concrete state symbolic state constraints 3 p , x=1 p=p0, x=x0,p->v =v0,p->next=n0 CUTE Approach Concrete Execution Symbolic Execution typedef struct cell { int v; struct cell *next; } cell; int f(int v) { return 2*v + 1; } int testme(cell *p, int x) { if (x > 0) if (p != NULL) if (f(x) == p->v) if (p->next == p) abort(); return 0; } x0>0

  39. concrete state symbolic state constraints 3 p , x=1 p=p0, x=x0,p->v =v0,p->next=n0 CUTE Approach Concrete Execution Symbolic Execution typedef struct cell { int v; struct cell *next; } cell; int f(int v) { return 2*v + 1; } int testme(cell *p, int x) { if (x > 0) if (p != NULL) if (f(x) == p->v) if (p->next == p) abort(); return 0; } x0>0 p0NULL

  40. concrete state symbolic state constraints 3 p , x=1 p=p0, x=x0,p->v =v0,p->next=n0 CUTE Approach Concrete Execution Symbolic Execution typedef struct cell { int v; struct cell *next; } cell; int f(int v) { return 2*v + 1; } int testme(cell *p, int x) { if (x > 0) if (p != NULL) if (f(x) == p->v) if (p->next == p) abort(); return 0; } x0>0 p0NULL 2x0+1=v0

  41. concrete state symbolic state constraints 3 p , x=1 p=p0, x=x0,p->v =v0,p->next=n0 CUTE Approach Concrete Execution Symbolic Execution typedef struct cell { int v; struct cell *next; } cell; int f(int v) { return 2*v + 1; } int testme(cell *p, int x) { if (x > 0) if (p != NULL) if (f(x) == p->v) if (p->next == p) abort(); return 0; } Program Error x0>0 p0NULL 2x0+1=v0 n0=p0

  42. CUTE in a Nutshell • Generate concrete inputs one by one

  43. CUTE in a Nutshell • Generate concrete inputs one by one • On each input execute program both concretely and symbolically

  44. CUTE in a Nutshell • Generate concrete inputs one by one • On each input execute program both concretely and symbolically • Both cooperate with each other

  45. CUTE in a Nutshell • Generate concrete inputs one by one • On each input execute program both concretely and symbolically • Both cooperate with each other • concrete execution guides the symbolic execution

  46. CUTE in a Nutshell • Generate concrete inputs one by one • On each input execute program both concretely and symbolically • Both cooperate with each other • concrete execution guides the symbolic execution • concrete execution enables symbolic execution to overcome incompleteness of theorem prover e.x:replace symbolic expressions by concrete values if symbolic expressions become complex

  47. CUTE in a Nutshell • Generate concrete inputs one by one • On each input execute program both concretely and symbolically • Both cooperate with each other • concrete execution guides the symbolic execution • concrete execution enables symbolic execution to overcome incompleteness of theorem prover e.x:replace symbolic expressions by concrete values if symbolic expressions become complex • symbolic execution helps to generate concrete input for next execution • increases coverage

  48. void again_test_me(int x,int y){ z = x*x*x + 3*x*x + 9; if(z != y){ printf(“Good branch”); } else { printf(“Bad branch”); abort(); } } Let initially x = -3 and y = 7 generated by random test-driver Simultaneous Symbolic & Concrete Execution

  49. void again_test_me(int x,int y){ z = x*x*x + 3*x*x + 9; if(z != y){ printf(“Good branch”); } else { printf(“Bad branch”); abort(); } } Let initially x = -3 and y = 7 generated by random test-driver concrete z = 9 symbolic z = x*x*x + 3*x*x+9 take then branch with constraint x*x*x+ 3*x*x+9 != y Simultaneous Symbolic & Concrete Execution

  50. void again_test_me(int x,int y){ z = x*x*x + 3*x*x + 9; if(z != y){ printf(“Good branch”); } else { printf(“Bad branch”); abort(); } } Let initially x = -3 and y = 7 generated by random test-driver concrete z = 9 symbolic z = x*x*x + 3*x*x+9 take then branch with constraint x*x*x+ 3*x*x+9 != y solve x*x*x+ 3*x*x+9 = y to take else branch Don’t know how to solve !! Stuck ? Simultaneous Symbolic & Concrete Execution

More Related