1 / 29

CHESS: A Systematic Testing Tool for Concurrent Software

CHESS: A Systematic Testing Tool for Concurrent Software. CSCI6900 George. Outline. Introduction Using CHESS Capturing nondeterminism Exploring nondeterminism Evaluation Related work. Introduction. Concurrent programming is difficult Testing concurrent program is difficult

devlin
Download Presentation

CHESS: A Systematic Testing Tool for Concurrent Software

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. CHESS: A Systematic Testing Tool for Concurrent Software CSCI6900 George

  2. Outline • Introduction • Using CHESS • Capturing nondeterminism • Exploring nondeterminism • Evaluation • Related work

  3. Introduction • Concurrent programming is difficult • Testing concurrent program is difficult • Famous concurrency bug: “Heisenbugs” • Most strategy used in finding concurrency bugs • Stress testing

  4. Introduction • What is “Concurrency testing” • Concurrency scenario testing • Model checking techniques • Three key challengers by applying model checking • Perturbation problem • Rich and complex concurrency APIs in system • State-space explosion.

  5. Introduction • What is “CHESS” • Concurrency scenario testing • Model checking techniques • Scale to large concurrent program • Reproduce an erroneous execution • Been integrated into the test framework inside Microsoft

  6. Introduction • How does CHESS solve the previous key problems? • Testing methodology of CHESS reduce the perturbation • Only one perturbation in CHESS • Wrappers to provides enough hooks • Variety of techniques to address the state-explosion

  7. Contributions of this paper • The first system for integrating model checking into concurrent systems and test frameworks with minimal perturbation; • Techniques for systematic exploration of systems code for fine-grained concurrency with shared memory and multithreading;

  8. Contributions of this paper • Validation of the CHESS tool and its accompany testing and wrapper methodology on three different platforms; • A substantial number of previously unknown bugs, even in well-tested systems; • The ability to consistently reproduce crashing bugs with unknown cause.

  9. Using CHESS • Traditional testing methodology • Disadvantages of this testing methodology

  10. Using CHESS • CHESS architecture

  11. Using CHESS • Testing methodology in CHESS

  12. Using CHESS • To guarantee the previous advantages, CHESS has to control the scheduling of the tasks • Two ways to control it • Modify the scheduler • Severely limited the deployment

  13. Using CHESS • Testing methodology in CHESS

  14. Using CHESS • Two assumption of about the testing program • Quiescence • Definition of “Quiescence State” • How does CHESS handle “Quiescence State”

  15. Capturing nondeterminism • Tasks in CHESS • Threads • Timers • Asynchronous I/Os • Wrappers in CHESS • Goal of wrappers

  16. Capturing nondeterminism • Design of Wrappers in CHESS

  17. Capturing nondeterminism • Synchronization wrappers

  18. Capturing nondeterminism • Synchronization wrappers

  19. Capturing nondeterminism • Hooking the wrappers • use various mechanisms to dynamically intercept calls to the real API functions and forward them to the wrappers. • Programs in this paper • Win32 • .NET • Singularity

  20. Exploring nondeterminism • How CHESS systematically drives the test • Basic scheduler operation • Allows only one thread to execute at a time • Repeatedly executes the same test driving each iteration of the test through a different schedule • Three phases in each iteration

  21. Exploring nondeterminism • Replay phase • Replays a sequence of scheduling choices from a trace file • Record phase • Schedules a thread till the thread yields the processor • Search phase • Uses the enabled information at each schedule point to determine the scheduler for the next iteration

  22. Exploring nondeterminism • Imperfect replay • Not rely on perfect replay capability • Common sources of nondeterminism • Lazy-initialization • Interference from environment • Nondeterministic calls

  23. Exploring nondeterminism • Ensuring fair schedules • Can not enumerate all fair schedules • State-explosion • Definition of State-explosion • Inserting preemptions • Capturing states

  24. Evaluation • Systems on which CHESS has been run on

  25. Evaluation • Test scenarios and findings

  26. Evaluation • Validation CHESS against stress-testing • Common objection of the CHESS • Failure and bug in CHESS • Succeeded in reproducing every stress-test failure

  27. Evaluation • Description of two bugs • PLINQ bug

  28. Evaluation • Description of two bugs • Singularity bug

  29. Related work • Repeatable deterministic testing • Systematic generation of thread schedules • Applying state exploration directly to executing concurrent programs • Replay a concurrent execution

More Related