210 likes | 338 Views
Practical Software Model Checking via Dynamic Interface Reduction. Huayang Guo 1,2 , Ming Wu 1 , Lidong Zhou 1 , Gang Hu 1,2 , Junfeng Yang 2 , Lintao Zhang 1 1 Microsoft Research Asia 2 Columbia University. Building reliable distributed systems is hard Machine failure Message lost
E N D
Practical Software Model Checking via Dynamic Interface Reduction Huayang Guo1,2, Ming Wu1, Lidong Zhou1, Gang Hu1,2, Junfeng Yang2, Lintao Zhang1 1Microsoft Research Asia 2Columbia University
Building reliable distributed systems is hard • Machine failure • Message lost • Message reorder • Thread interleaving • Non-determinism leads to tricky bugs Crash Async I/O Thr1 Thr2
Implementation-level software model checkers • MaceMC (NSDI’07), MoDist (NSDI’09) • Directly check implementations • No need to construct abstract model beforehand State Space Explorer Crash Async I/O … Thr1 Thr2
State space explosion • MPS: Product-level Paxos • Never fully explored 3 nodes • 34 years for MoDist …
Dynamic Interface Reduction (DIR) • Effective • 34 years 18 hours (Fully explored MPS-3) • Exponential Reduction: • 100K : 1 states for MPS and Berkeley DB w/ replication • Automatic, no manual efforts required • Provably sound and complete • Easy to integrate with legacy MCs • DeMeter: DIR with MoDist and MaceMC • MC specific modifications: ≤ 1k loc
Outline • Insight • Challenges • Dynamic Interface Reduction • Evaluation • Related work • Conclusion
Insight • Distributed systems: componentized • Local non-determinism isolated • Empirically, 99.9% do not propagate (Berkeley DB) • Previous work: • Check components together • |m1|*|m2|*|m3| • DIR: • Check components separately • |m1|+|m2|+|m3| Async I/O m1 Interface behavior m2 Thr1 Thr3 Thr2 Thr4 m3
Challenges and Solutions • How to discover/construct interface behavior of component? • Manually or statically construct interface process • Impractical for complex software system • How to guarantee • Completeness: find all bugs • Soundness: no false positives • Our solution: • Dynamically discover interface behaviors • Combine discovered interface behaviors • Track dependencies
DIR Overview Global Explorer Explore global interface behaviors Interface behavior Interface behavior Interface behavior Interface behavior Interface behavior Interface behavior Local Explorers Component3 Component2 Component1 Explore local states Explore local states Explore local states
Example Primary Secondary Client ClientPrimary/Secondary //Main thread //Checkpoint thread if (Choose(2)==0){ while (n=Recv()) { Lock(); Send(P,1); Lock(); Log(total); Send(P,2); total+=n; Unlock(); } else { Unlock(); Send(P,1); if (isPrimary) Send(P,3); Send(S,n); } } Ckpt Sum
Produce initial global trace Client(Cli)Primary/Secondary(Pri/Sec) //Main thread//Checkpoint thread if (Choose(2)==0){ while(n=Recv()) { Lock(); Send(P,1); Lock(); Log(total); Ckpt Send(P,2); total+=n; Sum Unlock(); } else { Unlock(); Send(P,1); if(isPrimary) Send(P,3); Send(S,n); } } Cli.Choose(2) = 0 Cli.Send(Pri, 1) Pri.Recv(Cli, 1) Pri.Ckpt Pri.Sum Pri.Send(Sec, 1) Sec.Recv(Pri, 1) Sec.Ckpt Sec.Sum Cli.Send(Pri, 2) Pri.Recv(Cli, 2) Pri.Sum Pri.Send(Sec, 2) Sec.Recv(Pri, 2) Sec.Sum -- Produce initial global trace. Global explorer
Construct message trace Client(Cli)Primary/Secondary(Pri/Sec) //Main thread//Checkpoint thread if (Choose(2)==0){ while(n=Recv()) { Lock(); Send(P,1); Lock(); Log(total); Ckpt Send(P,2); total+=n; Sum Unlock(); } else { Unlock(); Send(P,1); if(isPrimary) Send(P,3); Send(S,n); } } Cli.Choose(2) = 0 Cli.Send(Pri, 1) Pri.Recv(Cli, 1) Pri.Ckpt Pri.Sum Pri.Send(Sec, 1) Sec.Recv(Pri, 1) Sec.Ckpt Sec.Sum Cli.Send(Pri, 2) Pri.Recv(Cli, 2) Pri.Sum Pri.Send(Sec, 2) Sec.Recv(Pri, 2) Sec.Sum -- Bold statements form the message trace. Global explorer
Project message trace Client(Cli)Primary/Secondary(Pri/Sec) //Main thread//Checkpoint thread if (Choose(2)==0){ while(n=Recv()) { Lock(); Send(P,1); Lock(); Log(total); Ckpt Send(P,2); total+=n; Sum Unlock(); } else { Unlock(); Send(P,1); if(isPrimary) Send(P,3); Send(S,n); } } Cli.Choose(2) = 0 Cli.Send(Pri, 1) Pri.Recv(Cli, 1) Pri.Ckpt Pri.Sum Pri.Send(Sec, 1) Sec.Recv(Pri, 1) Sec.Ckpt Sec.Sum Cli.Send(Pri, 2) Pri.Recv(Cli, 2) Pri.Sum Pri.Send(Sec, 2) Sec.Recv(Pri, 2) Sec.Sum Secondary Primary Client Pri.Recv(Cli, 1) Sec.Recv(Pri, 1) Cli.Send(Pri, 1) Pri.Send(Sec, 1) Sec.Recv(Pri, 2) Cli.Send(Pri, 2) Pri.Recv(Cli, 2) Pri.Send(Sec, 2) -- Project global message trace to components. Global explorer
Local explorer for Primary Client(Cli)Primary/Secondary(Pri/Sec) //Main thread//Checkpoint thread if (Choose(2)==0){ while(n=Recv()) { Lock(); Send(P,1); Lock(); Log(total); Ckpt Send(P,2); total+=n; Sum Unlock(); } else { Unlock(); Send(P,1); if(isPrimary) Send(P,3); Send(S,n); } } Cli.Choose(2) = 0 Cli.Send(Pri, 1) Pri.Recv(Cli, 1) Pri.Ckpt Pri.Sum Pri.Send(Sec, 1) Sec.Recv(Pri, 1) Sec.Ckpt Sec.Sum Cli.Send(Pri, 2) Pri.Recv(Cli, 2) Pri.Sum Pri.Send(Sec, 2) Sec.Recv(Pri, 2) Sec.Sum Local explorer for Primary Pri.Recv(Cli, 1) Pri.Recv(Cli, 1) Pri.Recv(Cli, 1) Pri.Sum Pri.Sum Pri.Ckpt Pri.Sum Pri.Ckpt Pri.Send(Sec, 1) Pri.Send(Sec, 1) Pri.Send(Sec, 1) Pri.Recv(Cli, 2) Pri.Recv(Cli, 2) Pri.Recv(Cli, 2) Pri.Sum Pri.Sum Pri.Sum Pri.Ckpt Global explorer Pri.Send(Sec, 2) Pri.Send(Sec, 2) Pri.Send(Sec, 2)
Local explorer for Client Client(Cli)Primary/Secondary(Pri/Sec) //Main thread//Checkpoint thread if (Choose(2)==0){ while(n=Recv()) { Lock(); Send(P,1); Lock(); Log(total); Ckpt Send(P,2); total+=n; Sum Unlock(); } else { Unlock(); Send(P,1); if(isPrimary) Send(P,3); Send(S,n); } } Cli.Choose(2) = 0 Cli.Send(Pri, 1) Pri.Recv(Cli, 1) Pri.Ckpt Pri.Sum Pri.Send(Sec, 1) Sec.Recv(Pri, 1) Sec.Ckpt Sec.Sum Cli.Send(Pri, 2) Pri.Recv(Cli, 2) Pri.Sum Pri.Send(Sec, 2) Sec.Recv(Pri, 2) Sec.Sum Local explorer for Client Cli.Choose(2) = 0 Cli.Choose(2) = 1 Cli.Send(Pri, 1) Cli.Send(Pri, 1) Cli.Send(Pri, 2) Cli.Send(Pri, 3) Branching Trace Global explorer
Composition Client(Cli)Primary/Secondary(Pri/Sec) //Main thread//Checkpoint thread if (Choose(2)==0){ while(n=Recv()) { Lock(); Send(P,1); Lock(); Log(total); Ckpt Send(P,2); total+=n; Sum Unlock(); } else { Unlock(); Send(P,1); if(isPrimary) Send(P,3); Send(S,n); } } Cli.Choose(2) = 0 Cli.Send(Pri, 1) Pri.Recv(Cli, 1) Pri.Ckpt Pri.Sum Pri.Send(Sec, 1) Sec.Recv(Pri, 1) Sec.Ckpt Sec.Sum Cli.Send(Pri, 2) Pri.Recv(Cli, 2) Pri.Sum Pri.Send(Sec, 2) Sec.Recv(Pri, 2) Sec.Sum Existing global message trace: Branching local message trace: == Cli.Send(Pri, 1) Cli.Send(Pri, 1) Pri.Recv(Cli, 1) Pri.Send(Sec, 1) Sec.Recv(Pri, 1) Cli.Send(Pri, 3) Cli.Send(Pri, 2) Pri.Recv(Cli, 2) dependence Pri.Send(Sec, 2) Global explorer Sec.Recv(Pri, 2)
Composition Client(Cli)Primary/Secondary(Pri/Sec) //Main thread//Checkpoint thread if (Choose(2)==0){ while(n=Recv()) { Lock(); Send(P,1); Lock(); Log(total); Ckpt Send(P,2); total+=n; Sum Unlock(); } else { Unlock(); Send(P,1); if(isPrimary) Send(P,3); Send(S,n); } } Cli.Choose(2) = 0 Cli.Send(Pri, 1) Pri.Recv(Cli, 1) Pri.Ckpt Pri.Sum Pri.Send(Sec, 1) Sec.Recv(Pri, 1) Sec.Ckpt Sec.Sum Cli.Send(Pri, 2) Pri.Recv(Cli, 2) Pri.Sum Pri.Send(Sec, 2) Sec.Recv(Pri, 2) Sec.Sum New global message trace: Cli.Send(Pri, 1) Pri.Recv(Cli, 1) Pri.Send(Sec, 1) Sec.Recv(Pri, 1) Cli.Send(Pri, 3) Global explorer
Evaluation • Experiment Setup • DeMeter-MoDist: • MPS, an deployed product implementation of Paxos • Berkeley DB (BDB) • DeMeter-MaceMC: • Chord, peer-to-peer DHT implementation
Evaluation • Effectiveness of Dynamic Interface Reduction • App-n : n is the number of distributed nodes • Reduction Ratio: |Mw/o DIR| / |Mw DIR| x1000 x1000 x100
Related Work • Compositional model checking • E.M.Clarke et. al. (Symposium on Logic in Computer Science 1989) • Partial-order reduction • C.Flanagan and P.Godefroid (POPL’05) • Model checking network system • R.Guerraoui and M.Yabandeh (NSDI’11)
Conclusion • Distributed systems componentized • Local non-determinism does not propagate • Dynamic interface reduction • Effective, automatic, easy • Provably sound and complete • DeMeter– enable DIR for legacy MCs