1 / 46

Improving Error Discovery using Guided Search

Improving Error Discovery using Guided Search. Neha Rungta & Eric Mercer Computer Science Department Brigham Young University, Provo UT. Software Model Checking. Motivation Ariane 5 Comair debacle Verifying Software Models A transition graph for the model is created

lisle
Download Presentation

Improving Error Discovery using Guided Search

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Improving Error Discovery using Guided Search Neha Rungta & Eric Mercer Computer Science Department Brigham Young University, Provo UT

  2. Software Model Checking • Motivation • Ariane 5 • Comair debacle • Verifying Software Models • A transition graph for the model is created • A predefined property is verified ex. Reachability • Problem • Number of behaviors is exponential with every increment • This causes a state explosion problem Verification and Validation, CS Dept, BYU

  3. Approaches • Traditional techniques to counter it • Parallel or Distributed Model Checking • Predicate Abstraction • Disk based Algorithm • Heuristics for Guided search • Heuristics • Find a counterexample before memory runs out • Property based heuristics • Structure based heuristics • Structure of program can be use to guide the search Verification and Validation, CS Dept, BYU

  4. Current Structural heuristics • Stefan Edelkamp and Tilman Mehler • Finds a short and easy to understand Error trail • Minimal operations to reach g from s is FSM distance • This distance is admissible and consistent • Build control flow graph (CFG) with just PC values • Willem Visser and Alex Groce • Specific only to Java Verification and Validation, CS Dept, BYU

  5. Underestimation Example 01 main: 01: ldx #1 02: call foo 03: add x,1 04: call foo 05: check for error foo: 06: pshx 07: pulx 08: rts main Verification and Validation, CS Dept, BYU

  6. Underestimation Example 01 main: 01: ldx #1 02: call foo 03: add x,1 04: call foo 05: check for error foo: 06: pshx 07: pulx 08: rts main 02 Verification and Validation, CS Dept, BYU

  7. Underestimation Example 01 main: 01: ldx #1 02: call foo 03: add x,1 04: call foo 05: check for error foo: 06: pshx 07: pulx 08: rts main 02 06 foo Verification and Validation, CS Dept, BYU

  8. Underestimation Example 01 main: 01: ldx #1 02: call foo 03: add x,1 04: call foo 05: check for error foo: 06: pshx 07: pulx 08: rts main 02 06 foo 03 07 04 08 05 error Verification and Validation, CS Dept, BYU

  9. Underestimation Example 01 main: 01: ldx #1 02: call foo 03: add x,1 04: call foo 05: check for error foo: 06: pshx 07: pulx 08: rts main 02 06 foo 03 07 04 08 Edelkamp’s FSM Heuristic: Shortest Distance from current state to checking for error in the CFG 05 error error Verification and Validation, CS Dept, BYU

  10. Underestimation Example 01 main: 01: ldx #1 02: call foo 03: add x,1 04: call foo 05: check for error foo: 06: pshx 07: pulx 08: rts main 02 06 foo 03 07 04 08 Edelkamp’s FSM Heuristic: Shortest Distance from current state to checking for error in the CFG 05 error error Verification and Validation, CS Dept, BYU

  11. Underestimation Example 01 main: 01: ldx #1 02: call foo 03: add x,1 04: call foo 05: check for error foo: 06: pshx 07: pulx 08: rts main 02 06 foo 03 07 04 08 Edelkamp’s FSM Heuristic: Shortest Distance from current state to checking for error in the CFG 05 error error Verification and Validation, CS Dept, BYU

  12. Underestimation Example 01 main: 01: ldx #1 02: call foo 03: add x,1 04: call foo 05: check for error foo: 06: pshx 07: pulx 08: rts main 02 06 foo 03 07 04 08 Edelkamp’s FSM Heuristic: Shortest Distance from current state to checking for error in the CFG 05 error error Verification and Validation, CS Dept, BYU

  13. Underestimation Example 01 main: 01: ldx #1 02: call foo 03: add x,1 04: call foo 05: check for error foo: 06: pshx 07: pulx 08: rts main 02 06 foo foo 03 07 04 08 Edelkamp’s FSM Heuristic: Shortest Distance from current state to checking for error in the CFG 05 3 steps error error Verification and Validation, CS Dept, BYU

  14. True Distance should be …. 01 main: 01: ldx #1 02: call foo 03: add x,1 04: call foo 05: check for error foo: 06: pshx 07: pulx 08: rts main 02 06 foo 03 07 04 08 05 error error Verification and Validation, CS Dept, BYU

  15. True Distance should be …. 01 main: 01: ldx #1 02: call foo 03: add x,1 04: call foo 05: check for error foo: 06: pshx 07: pulx 08: rts main 02 06 foo 03 07 04 08 05 error error error Verification and Validation, CS Dept, BYU

  16. True Distance should be …. 01 main: 01: ldx #1 02: call foo 03: add x,1 04: call foo 05: check for error foo: 06: pshx 07: pulx 08: rts main 02 06 foo 03 07 04 08 05 error error error Verification and Validation, CS Dept, BYU

  17. True Distance should be …. 01 main: 01: ldx #1 02: call foo 03: add x,1 04: call foo 05: check for error foo: 06: pshx 07: pulx 08: rts main 02 06 foo 03 07 04 08 05 error error error Verification and Validation, CS Dept, BYU

  18. Underestimation Example 01 main: 01: ldx #1 02: call foo 03: add x,1 04: call foo 05: check for error foo: 06: pshx 07: pulx 08: rts main 02 06 foo 03 07 04 08 05 error error error Verification and Validation, CS Dept, BYU

  19. True Distance should be …. 01 main: 01: ldx #1 02: call foo 03: add x,1 04: call foo 05: check for error foo: 06: pshx 07: pulx 08: rts main 02 06 foo 03 07 04 08 05 error error error Verification and Validation, CS Dept, BYU

  20. True Distance should be …. 01 main: 01: ldx #1 02: call foo 03: add x,1 04: call foo 05: check for error foo: 06: pshx 07: pulx 08: rts main 02 06 foo 03 07 04 08 05 error error error Verification and Validation, CS Dept, BYU

  21. True Distance should be …. 01 main: 01: ldx #1 02: call foo 03: add x,1 04: call foo 05: check for error foo: 06: pshx 07: pulx 08: rts main 02 06 foo 03 07 04 08 05 8 steps error error error Verification and Validation, CS Dept, BYU

  22. Solution: Interprocedural CFG • All the nodes in the ICFG that are part of a subroutine will be indexed on two things • PC Value • Return address to where the subroutine will return when it encounters a return statement Verification and Validation, CS Dept, BYU

  23. 01 (init) main: 01: ldx #1 02: call foo 03: add x,1 04: call foo 05: check for error foo: 06: pshx 07: pulx 08: rts Verification and Validation, CS Dept, BYU

  24. 01 (init) 02 (init) main: 01: ldx #1 02: call foo 03: add x,1 04: call foo 05: check for error foo: 06: pshx 07: pulx 08: rts Verification and Validation, CS Dept, BYU

  25. 01 (init) 02 (init) 06 (03) main: 01: ldx #1 02: call foo 03: add x,1 04: call foo 05: check for error foo: 06: pshx 07: pulx 08: rts Verification and Validation, CS Dept, BYU

  26. 01 (init) 02 (init) 06 (03) main: 01: ldx #1 02: call foo 03: add x,1 04: call foo 05: check for error foo: 06: pshx 07: pulx 08: rts Verification and Validation, CS Dept, BYU

  27. 01 (init) 02 (init) 06 (03) 07 (03) 08 (03) main: 01: ldx #1 02: call foo 03: add x,1 04: call foo 05: check for error foo: 06: pshx 07: pulx 08: rts 03 (init) Verification and Validation, CS Dept, BYU

  28. 01 (init) 02 (init) 06 (03) 07 (03) 08 (03) main: 01: ldx #1 02: call foo 03: add x,1 04: call foo 05: check for error foo: 06: pshx 07: pulx 08: rts 03 (init) 04 (init) 06 (05) 07 (05) 08 (05) 05 (init) Verification and Validation, CS Dept, BYU

  29. 01 (init) 02 (init) 06 (03) 07 (03) 08 (03) main: 01: ldx #1 02: call foo 03: add x,1 04: call foo 05: check for error foo: 06: pshx 07: pulx 08: rts 03 (init) 04 (init) 06 (05) 07 (05) 08 (05) 05 (init) Verification and Validation, CS Dept, BYU

  30. 01 (init) 02 (init) 06 (03) 07 (03) 08 (03) main: 01: ldx #1 02: call foo 03: add x,1 04: call foo 05: check for error foo: 06: pshx 07: pulx 08: rts 03 (init) 04 (init) 06 (05) 07 (05) 08 (05) 05 (init) 8 steps Verification and Validation, CS Dept, BYU

  31. Nested Function Calls x x f f g g • x → f → g • y → f → g • Same problem as before main 1:call x (init) 3:call f (2) 7:call g (4) 9 (8) f y 5:call g (error) 7:call g (6) a:rts (8) main: 1 call x 2 call y error f: 7 call g 8 rts 2:call y (init) 4:rts (2) 8:rts (4) x: 3 call f 4 rts y: 5 call f 6 rts g: 9 xyz a rts 6:rts (error) error 8:rts (6) Verification and Validation, CS Dept, BYU

  32. Improved ICFG Algorithm ICFG_Algorithm(state S) sa0 = icfgState(S) <sa1,sa2,….,san> // are abstracted states in the call stack of S d = 0 for i = 0 to n do srtn = rtn(sai) if FSM(sai, error) < FSM(sai,srtn) then d = d + FSM(sai,error) return d d = d + FSM(sai, srtn) + 1 return d Verification and Validation, CS Dept, BYU

  33. Improved ICFG Algorithm ICFG_Algorithm(state S) sa0 = icfgState(S) <sa1,sa2,….,san> // are abstracted states in the call stack of S d = 0 for i = 0 to n do srtn = rtn(sai) if FSM(sai, error) < FSM(sai,srtn) then d = d + FSM(sai,error) return d d = d + FSM(sai, srtn) + 1 return d Verification and Validation, CS Dept, BYU

  34. Abstract states from the stack 0a (08) PC: 09 abstract states generated from the stack sa0 02 04 08 (04) 08 sa1 04 (02) sa2 02 (init) sa3 Verification and Validation, CS Dept, BYU

  35. Improved ICFG Algorithm ICFG_Algorithm(state S) sa0 = icfgState(S) <sa1,sa2,….,san> // are abstracted states in the call stack of S d = 0 for i = 0 to n do srtn = rtn(sai) if FSM(sai, error) < FSM(sai,srtn) then d = d + FSM(sai,error) return d d = d + FSM(sai, srtn) + 1 return d Verification and Validation, CS Dept, BYU

  36. Improved ICFG Algorithm ICFG_Algorithm(state S) sa0 = icfgState(S) <sa1,sa2,….,san> // are abstracted states in the call stack of S d = 0 for i = 0 to n do srtn = rtn(sai) if FSM(sai, error) < FSM(sai,srtn) then d = d + FSM(sai,error) return d d = d + FSM(sai, srtn) + 1 return d Verification and Validation, CS Dept, BYU

  37. Improved ICFG Algorithm ICFG_Algorithm(state S) sa0 = icfgState(S) <sa1,sa2,….,san> // are abstracted states in the call stack of S d = 0 for i = 0 to n do srtn = rtn(sai) if FSM(sai, error) < FSM(sai,srtn) then d = d + FSM(sai,error) return d d = d + FSM(sai, srtn) + 1 return d Verification and Validation, CS Dept, BYU

  38. Marking returns statically foo prologue beq epilogue return Verification and Validation, CS Dept, BYU

  39. Improved ICFG Algorithm ICFG_Algorithm(state S) sa0 = icfgState(S) <sa1,sa2,….,san> // are abstracted states in the call stack of S d = 0 for i = 0 to n do srtn = rtn(sai) if FSM(sai, error) < FSM(sai,srtn) then d = d + FSM(sai,error) return d d = d + FSM(sai, srtn) + 1 return d Verification and Validation, CS Dept, BYU

  40. Improved ICFG Algorithm x x f f g g PC: 09 main 1:call x (init) 3:call f (2) 7:call g (4) 9 (8) 02 04 f y 08 5:call g (error) 7:call g (6) a:rts (8) Calculating the Heuristic: D = 0 FSM ((a,8),error) = 4 2:call y (init) 4:rts (2) 8:rts (4) 6:rts (error) error 8:rts (6) Verification and Validation, CS Dept, BYU

  41. Improved ICFG Algorithm x f g PC: 09 main 1:call x (init) 3:call f (2) 7:call g (4) 9 (8) 02 04 f y 08 5:call g (error) 7:call g (6) a:rts (8) Calculating the Heuristic: D = 0 FSM ((a,8),error) = 4 FSM ((a,8), (rts,8) = 1 1 < 4 D += 1 2:call y (init) 4:rts (2) 8:rts (4) 6:rts (error) error 8:rts (6) Verification and Validation, CS Dept, BYU

  42. Improved ICFG Algorithm ICFG_Algorithm(state S) sa0 = icfgState(S) <sa1,sa2,….,san> // are abstracted states in the call stack of S d = 0 for i = 0 to n do srtn = rtn(sai) if FSM(sai, error) < FSM(sai,srtn) then d = d + FSM(sai,error) return d d = d + FSM(sai, srtn) + 1 return d Verification and Validation, CS Dept, BYU

  43. Improved ICFG Algorithm x f g PC: 0a main 1:call x (init) 3:call f (2) 7:call g (4) 9 (8) f y 5:call g (error) 7:call g (6) a:rts (8) D = 11 2:call y (init) 4:rts (2) 8:rts (4) 6:rts (error) error 8:rts (6) Verification and Validation, CS Dept, BYU

  44. Results: Number of states generated Verification and Validation, CS Dept, BYU

  45. Conclusions • Small overhead allowed use of more static information • The Dynamic call stack with static analysis gave a better estimate • Testing shows an significant improvement in FSM distance • The Improved ICFG algorithm can be used on any graph • The algorithm is admissible and consistent Verification and Validation, CS Dept, BYU

  46. QUESTIONS Verification and Validation, CS Dept, BYU

More Related