1 / 12

Applying Dynamic Analysis to Test Corner Cases First

Applying Dynamic Analysis to Test Corner Cases First. Penka Vassileva Markova Madanlal Musuvathi. Motivation. Many Systems can be modeled as state machines that interact with each other e.g. Networking Protocols Subtle bugs in such systems involve sequences of rare corner cases:

cheryl
Download Presentation

Applying Dynamic Analysis to Test Corner Cases First

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Applying Dynamic Analysis to Test Corner Cases First Penka Vassileva Markova Madanlal Musuvathi

  2. Motivation • Many Systems can be modeled as state machines that interact with each other • e.g. Networking Protocols • Subtle bugs in such systems involve sequences of rare corner cases: • TCP Impl in FIN-WAIT state, on receiving a retransmitted FIN packet, sends an ACK with a corrupted checksum. • AODV on receiving a RERR packet out of order, just after a link failure, leads to a routing loop. • Conventional testing misses these errors • Such errors are difficult to detect using Static Analyses

  3. Explicit State Enumeration • Explicit State Enumeration is a promising approach to test such systems. • It systematically explores the entire state space of the system. • Better than “brute force” testing as it visits each system state only once • Usually runs out of resources before it terminates • The trick is to guide the search to interesting parts of state space • Our Proposal: • Use dynamic analyses to detect corner cases • Search corner cases first

  4. CMC – A C Model Checker • CMC is an explicit state enumerator • Runs C Code directly • State consists of variables in the program • Current Implementation restricts state to Global and Heap variables. • State transitions are performed by transferring control to different points in the C code. • Current Implementation requires each transition to be performed by a handler function in the code • By executing code, CMC is able to handle implementation errors as well as protocol errors. • It is Sound: No false positives

  5. Client Server Client Connect SynSent SynRcv Closed Listen SynSent Listen SYN-ACK empty SYN Network Retransmit Pkt Recv Pkt Loss SynSent Listen empty Retransmit Pkt Loss Pkt Recv An Example State Transitions are performed by the Actual C Code

  6. Fundamental Problem • Real systems have infinite or large state space • Memory (not time) is the key bottleneck • Need to maintain a queue of unexplored states • Each state can be as huge as 10K • Partial solutions are available • However, prioritizing state search essential • Visit interesting states before running out of resources • Can flush out low priority states to the disk

  7. Prioritizing the State Search Idea • Gather information from the runs that have been executed • Use this information to rank how interesting the state generated from the current run are Techniques: • Track values and simple invariants • Look for complex “invariants” • Path profiling

  8. Values and simple invariants Tracking values and simple invariants: • Inspiration: DIDUCE • On-line detection • Look at simple invariants, relax after a counterexample • One heuristic: at exit point track the variables that are in the state • Observation so far: get saturated very fast • Use knowledge of the code or algorithm with heuristics to choose other expressions of interest

  9. Complex Invariants More complex relationships between variables • a>b may be more important for the output than the exact values of a or b. Inspiration: Daikon • Overhead management • analyze the collected information in intervals • Observation: the number of bugs found goes down with time • so, after the first stages it is worth spending the time to analyze such information • Only look at some of the possible invariants at a time • Use static analysis or heuristics to suggest more interesting invariants to gather information on • After invalidating most of the current invariants, can add a new set to track and give it a lower confidence degree • Side effect: can suggest real invariants • But should reflect intervals on confidence level

  10. Profiling Path/Edge Profiling • Observation: code coverage matters • Keep track about which branches/paths of the program has been taken more often then others • Use existing profiling techniques

  11. Challenges and Future Work Challenges • Memory management • Scaling • Picking the right heuristics • Evaluating which heuristics are more useful Future work • Look into more complicated protocols • Can we guide the generation of the test set? • Intuition: executing the code at least once is probably good • Unless it was the assert(false) case  • Try to find a counterexample and test case such that to get to this code?

  12. Related Work • Diduce • Daikon • JavaPathfinder • Uses thread-interleaving heuristic to model-check • A lot of work on Profiling • A lot of Model Checking work

More Related