1 / 54

Guided Test Visualization : Making Sense of Errors in Concurrent Programs

Guided Test Visualization : Making Sense of Errors in Concurrent Programs . Saint Wesonga & Neha Rungta & Eric Mercer Brigham Young University & NASA AMES, USA. State of the Art. Stress Testing Dynamic Analysis Runtime monitoring Static Analysis Model Checking Symbolic Execution

apollo
Download Presentation

Guided Test Visualization : Making Sense of Errors in Concurrent Programs

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Guided Test Visualization:Making Sense of Errors in Concurrent Programs Saint Wesonga & NehaRungta & Eric Mercer Brigham Young University & NASA AMES, USA

  2. State of the Art • Stress Testing • Dynamic Analysis • Runtime monitoring • Static Analysis • Model Checking • Symbolic Execution • Software Model Checking Guided-test [SPIN 2009] Slice And Dice [ICSE-NIER 2010]

  3. Belief Guided by belief Sea of choices Guided-test and Slice-and-dice

  4. Guided-test wants to find real errors Find an error or you do not have much to say

  5. Our Solution Possible errors locations from imprecise analysis tools Add inter-thread dependenceinformation Guided Symbolic Execution Sound errordetection Generate backward slicesfrom possible error locations Heuristics Runtime Environment

  6. Input Possible errors locations from imprecise analysis tools Add inter-thread dependenceinformation Guided Symbolic Execution Sound errordetection Generate backward slicesfrom possible error locations Heuristics Runtime Environment

  7. Small example Thread A { void run (Element elem) { lock(elem) check(elem) unlock(elem) } void check(Elementelem) { if(elem.e > 9) Throw exception } } Thread B { void run (Element elem) { intx; // input variable if( x > 18) { lock(elem) elem.reset(); unlock(elem) } } } Object Element { inte; Element() { e = 1; } void reset() { e = 11; } }

  8. Small Example Thread A { void run (Element elem) { lock(elem) check(elem) unlock(elem) } void check(Elementelem) { if(elem.e > 9) Throw exception } } Thread B { void run (Element elem) { intx; // input variable if( x > 18) { lock(elem) elem.reset(); unlock(elem) } } } Object Element { inte; Element() { e = 1; } void reset() { e = 11; } }

  9. Small Example Thread A { void run (Element elem) { lock(elem) check(elem) unlock(elem) } void check(Elementelem) { if(elem.e > 9) Throw exception } } Thread B { void run (Element elem) { intx; // input variable if( x > 18) { lock(elem) elem.reset(); unlock(elem) } } } Object Element { inte; Element() { e = 1; } void reset() { e = 11; } }

  10. Small Example Thread A { void run (Element elem) { lock(elem) check(elem) unlock(elem) } void check(Elementelem) { if(elem.e > 9) Throw exception } } Thread B { void run (Element elem) { intx; // input variable if( x > 18) { lock(elem) elem.reset(); unlock(elem) } } } Object Element { inte; Element() { e = 1; } void reset() { e = 11; } }

  11. Small Example Thread A { void run (Element elem) { lock(elem) check(elem) unlock(elem) } void check(Elementelem) { if(elem.e > 9) Throw exception } } Thread B { void run (Element elem) { intx; // input variable if( x > 18) { lock(elem) elem.reset(); unlock(elem) } } } Object Element { inte; Element() { e = 1; } void reset() { e = 11; } }

  12. Small Example Thread A { void run (Element elem) { lock(elem) check(elem) unlock(elem) } void check(Elementelem) { if(elem.e > 9) Throw exception } } Thread B { void run (Element elem) { intx; // input variable if( x > 18) { lock(elem) elem.reset(); unlock(elem) } } } Object Element { inte; Element() { e = 1; } void reset() { e = 11; } }

  13. Small Example Thread A { void run (Element elem) { lock(elem) check(elem) unlock(elem) } void check(Elementelem) { if(elem.e > 9) Throw exception } } Thread B { void run (Element elem) { intx; // input variable if( x > 18) { lock(elem) elem.reset(); unlock(elem) } } } Object Element { inte; Element() { e = 1; } void reset() { e = 11; } }

  14. Could it really be? Thread A { void run (Element elem) { lock(elem) check(elem) unlock(elem) } void check(Elementelem) { if(elem.e > 9) Throw exception } } Thread B { void run (Element elem) { intx; // input variable if( x > 18) { lock(elem) elem.reset(); unlock(elem) } } } Object Element { inte; Element() { e = 1; } void reset() { e = 11; } } Is it possible to arrive at this location on some execution?

  15. Initial Abstraction to Follow (not explore) Possible errors locations from imprecise analysis tools Add inter-thread dependenceinformation Guided Symbolic Execution Sound errordetection Generate backward slicesto possible error locations Heuristics Runtime Environment

  16. Slice along sequential execution Thread A { void run (Element elem) { lock(elem) check(elem) unlock(elem) } void check(Elementelem) { if(elem.e > 9) Throw exception } } Object Element { inte; Element() { e = 1; } void reset() { e = 11; } }

  17. Target Locations & Start of Programs Thread A { void run (Element elem) { lock(elem) check(elem) unlock(elem) } void check(Elementelem) { if(elem.e > 9) Throw exception } } A.runlstart Object Element { inte; Element() { e = 1; } void reset() { e = 11; } } Exception

  18. Call Sites and Start Locs of the callee Thread A { void run (Element elem) { lock(elem) check(elem) unlock(elem) } void check(Elementelem) { if(elem.e > 9) Throw exception } } A.runlstart check(elem) Object Element { inte; Element() { e = 1; } void reset() { e = 11; } } A.checklstart Exception

  19. Conditional Statements Thread A { void run (Element elem) { lock(elem) check(elem) unlock(elem) } void check(Elementelem) { if(elem.e > 9) Throw exception } } A.runlstart check(elem) Object Element { inte; Element() { e = 1; } void reset() { e = 11; } } A.checklstart if(elem.e > 9) Exception

  20. Data Definitions Thread A { void run (Element elem) { lock(elem) check(elem) unlock(elem) } void check(Elementelem) { if(elem.e > 9) Throw exception } } A.runlstart check(elem) Object Element { inte; Element() { e = 1; } void reset() { e = 11; } } A.checklstart if(elem.e > 9) Exception

  21. Synchronization Operations Thread A { void run (Element elem) { lock(elem) check(elem) unlock(elem) } void check(Elementelem) { if(elem.e > 9) Throw exception } } A.runlstart lock(elem) check(elem) Object Element { inte; Element() { e = 1; } void reset() { e = 11; } } unlock(elem) A.checklstart if(elem.e > 9) Exception

  22. Abstract System Thread A { void run (Element elem) { lock(elem) check(elem) unlock(elem) } void check(Elementelem) { if(elem.e > 9) Throw exception } } A.runlstart lock(elem) check(elem) unlock(elem) Object Element { inte; Element() { e = 1; } void reset() { e = 11; } } A.checklstart if(elem.e > 9) Exception

  23. Abstract Trace Thread A { void run (Element elem) { lock(elem) check(elem) unlock(elem) } void check(Elementelem) { if(elem.e > 9) Throw exception } } A.runlstart lock(elem) check(elem) Object Element { inte; Element() { e = 1; } void reset() { e = 11; } } unlock(elem) A.checklstart if(elem.e > 9) Exception

  24. Abstract Trace Thread A { void run (Element elem) { lock(elem) check(elem) unlock(elem) } void check(Elementelem) { if(elem.e > 9) Throw exception } } A.runlstart lock(elem) One trace of many we could have selected: fair coin or farm in parallel check(elem) A.checklstart Object Element { inte; Element() { e = 1; } void reset() { e = 11; } } if(elem.e > 9) Exception

  25. Our Solution Possible errors locations from imprecise analysis tools Add inter-thread dependenceinformation Guided Symbolic Execution Sound errordetection Generate backward slicesto possible error locations Heuristics Runtime Environment

  26. Map Concrete state to a location Goal is to match a concrete state to the next program location in our trace Follow the trace! Thread-0StackLocalVars Thread-1StackLocalVars Thread-nStackLocalVars HeapGlobalVars s1 Data Input Variables Symbolic Representations ϕpath Constraint Extract current program location of themost recently executed thread l1

  27. Guidance Strategy l1 s0 s3 s1 s2 l2 ln

  28. Guidance Strategy l1 s0 s3 s1 s2 l2 Flip a fair coin and move forward ln

  29. Guidance Strategy l1 s0 s3 s1 s2 l2 s4 s5 s6 Distance heuristic to choose nearest to next location on the current trace ln

  30. Our Solution Possible errors locations from imprecise analysis tools Add inter-thread dependenceinformation (refine) Guided Symbolic Execution Sound errordetection Generate backward slicesto possible error locations Heuristics Runtime Environment

  31. Followed trace to predicate… A.runlstart lock(elem) check(elem) A.checklstart False in all successors (zut!) if(elem.e > 9) Exception

  32. Where else is elem.e defined? Thread A { void run (Element elem) { lock(elem) check(elem) unlock(elem) } void check(Elementelem) { if(elem.e > 9) Throw exception } } Thread B { void run (Element elem) { intx; // input variable if( x > 18) { lock(elem) elem.reset(); unlock(elem) } } } Object Element { inte; Element() { e = 1; } void reset() { e = 11; } }

  33. Redefine a global variable B.runlstart A.runlstart lock(elem) check(elem) unlock(elem) A.checklstart if(elem.e > 9) e := 11 Exception

  34. Generate a new backward slice Thread B { void run (Element elem) { intx; // input variable if( x > 18) { lock(elem) elem.reset(); unlock(elem) } } } B.runlstart if(x > 18) lock(elem) elem.reset() Object Element { inte; Element() { e = 1; } void reset() { e = 11; } } Element.resetlstart unlock(elem) e := 11

  35. Updated Abstract System B.runlstart A.runlstart if(x > 18) lock(elem) lock(elem) check(elem) elem.reset() unlock(elem) A.checklstart Element.resetlstart unlock(elem) if(elem.e > 9) e := 11 Exception

  36. Update Abstract Trace A.runlstart B.runlstart lock(elem) if(x > 18) check(elem) lock(elem) elem.reset() A.checklstart Element.resetlstart if(elem.e > 9) Exception e := 11

  37. Check Lock Dependencies A.runlstart B.runlstart lock(elem) if(x > 18) check(elem) lock(elem) elem.reset() A.checklstart Element.resetlstart if(elem.e > 9) Exception e := 11

  38. Update Abstract Trace A.runlstart B.runlstart lock(elem) if(x > 18) Thread B { void run (Element elem) { intx; // input variable if( x > 18) { lock(elem) elem.reset(); unlock(elem) } } } check(elem) lock(elem) elem.reset() A.checklstart Element.resetlstart if(elem.e > 9) Exception unlock(elem) e := 11

  39. Update Abstract State A.runlstart B.runlstart lock(elem) if(x > 18) check(elem) lock(elem) unlock(elem) elem.reset() A.checklstart Element.resetlstart if(elem.e > 9) Restart with this new trace from the initial state Exception e := 11

  40. Slice and dice wants to find real errors and prove correct… Don’t find an error yet have much to say

  41. Abstract Trace Thread A { void run (Element elem) { lock(elem) check(elem) unlock(elem) } void check(Elementelem) { if(elem.e > 9) Throw exception } } A.runlstart lock(elem) check(elem) Object Element { inte; Element() { e = 1; } void reset() { e = 11; } } unlock(elem) A.checklstart if(elem.e > 9) Exception

  42. Slice and Dice • Keep entire abstraction • Schedule only on node in the abstraction A.runlstart lock(elem) check(elem) unlock(elem) A.checklstart if(elem.e > 9) Exception

  43. When it is time to refine… A.runlstart B.runlstart lock(elem) if(x > 18) check(elem) lock(elem) unlock(elem) elem.reset() A.checklstart Element.resetlstart if(elem.e > 9) …rather than create a single trace… Exception e := 11

  44. Keep the entire abstraction B.runlstart A.runlstart if(x > 18) lock(elem) lock(elem) check(elem) elem.reset() unlock(elem) A.checklstart Element.resetlstart unlock(elem) if(elem.e > 9) e := 11 Exception

  45. Visualization Demonstration

  46. Belief Guided by belief Sea of choices Guided-test and Slice-and-dice

  47. jpf-guided-test Verification & Validation Lab Brigham Young University Provo, UT 84606, USA http://vv.cs.byu.edu/ VV Lab, Brigham Young University, USA

  48. This slide left intentionally blank

  49. Experimental Results • A new GuidedSymbolic extension in Java Pathfinder • Uses the engine from the symbolic extension (Pasareanuet. al. ISSTA 2008) • Dynamic partial order reduction turned on • Abstraction, refinement and heuristic computation all performed on Java bytecode • Libraries are part of the multi-threaded system

  50. Multi-threaded Programs • Reorder BenchmarkSLOC: 44, Reachability • Airline BenchmarkSLOC: 31, Reachability • VecDeadlock Real JDK 1.4 concurrent librarySLOC: 7267, Deadlock in Vector class • VecDeadlock Real JDK 1.4 concurrent librarySLOC: 7169, Deadlock in Vector class • VecRace Real JDK 1.4 concurrent librarySLOC: 7151, Race in ArrayList class

More Related