500 likes | 507 Views
Zing: Exploiting Program Structure for Model Checking Concurrent Software. Tony Andrews Shaz Qadeer Sriram K. Rajamani Jakob Rehof Microsoft Research. Yichen Xie Stanford. Outline . Motivation for Zing Zing overview Exploiting structure for efficient model checking Reduction
E N D
Zing: Exploiting Program Structure for Model Checking Concurrent Software Tony Andrews Shaz Qadeer Sriram K. Rajamani Jakob Rehof Microsoft Research Yichen Xie Stanford
Outline • Motivation for Zing • Zing overview • Exploiting structure for efficient model checking • Reduction • Summarization • Compositional conformance checking
Problem • Check if programs written in common programming languages (C, C++, C#, Java) satisfy certain safety properties • Examples of properties: • API usage rules – ordering of calls • Absence of races • Absence of deadlocks • Protocol (state machines) on objects
Approach • Extract abstract “model” from the program that captures all “relevant” portions of the program with respect to property of interest • Systematically explore the state space of the extracted model. • Example: SLAM • Check if a sequential C program uses an interface “correctly” as specified by a safety property, using boolean program models
Software model checking SLAM model checker Data flow analysis implemented using BDDs Finite state machines Push down model Boolean program FSM abstraction C data structures, pointers, procedure calls, parameter passing, scoping,control flow Source code Sequential program in rich programming language (eg. C) Related work: BLAST, MAGIC,…
Zing model checker • 3 core constructs: • Procedure calls with call-stack • Objects with dynamic allocation • Threads with dynamic creation • Inter-process communication: • Shared memory • Channels with blocking-receives, non-blocking sends, FIFO abstraction Source code Concurrent program in rich programming language
Outline • Motivation for Zing • Zing overview • Exploiting structure for efficient model checking • Reduction • Summarization • Compositional conformance checking
Zing: Challenges and Approach • Handling programming language features • Compile Zing to an intermediate “object model” (ZOM) • Build model checker on top of ZOM • State explosion • Expose program structure in ZOM • Exploit program structure to do efficient model checking
Processes Process Process Process Stack … IP Locals Params Heap: complextypes IP Locals Params … … Zing Object Model: Internal StateView State Globals: simple types & refs
Zing Object Model: External State View • Simplified view to query and update state • How many processes? • Is process(i) runnable? • Are two states equal? • Execute process(i) for one atomic step • Can write simple DFS search in 10 lines
privatevoid doDfs(){ while(stateStack.Count > 0){ State s = (State) stateStack.Peek(); bool foundSuccessor = false; // find the next process to execute and execute it for (int p =s.LastProcessExplored + 1; p < s.NumProcesses; p++) { if(s.RunnableProcesses[p] { State newS = s.Execute(p); if (!stateHash.contains(newS)){ stateHash.add(newS); stateStack.push(newS); foundSuccessor = true; break; } } } if(!foundSuccessor) stateStack.Pop(); } } DOESN’T SCALE NEED TO EXPLOIT PROGRAM STRUCTURE !
Outline • Motivation for Zing • Zing overview • Exploiting structure for efficient model checking • Reduction • Summarization • Compositional conformance checking
Racy program: need to explore all interleavings! local int y = 0; x := x + 1; x := x + 1; x := x + 1; x := x +1; assert(x div 4); y = y+1; y = y+1; //initialize int x :=0; local int z = 0; x := x + 1; x := x + 1; x := x + 1; x := x +1; assert(x div 4); z = z+1; z = z+1;
Race-free program: need to explore two interleavings! local int y; acquire (m); x := x + 1; x := x + 1; x := x + 1; x := x +1; assert(x div 4); release (m); y = y+1; y = y+1; //initialize int x :=0; mutex m; local int z; acquire (m); x := x + 1; x := x + 1; x := x + 1; x := x +1; assert(x div 4); release (m); z = z+1; z = z+1;
x r=bal S2 S3 S4 r=bal x S2 T3 S4 z rel(this) r=bal y acq(this) x S5 S6 S7 S2 S3 S4 S0 S1 S2 rel(this) x acq(this) z y r=bal S2 S0 S5 T1 T6 S7 S2 T3 S4 Four atomicities • R: right movers • lock acquire • L: left movers • lock release • B: both right + left movers • variable access holding lock • N: non-movers • access unprotected variable
R* . x . N . Y . L* S0 S5 R* x . . . . Y N L* S0 S5 Transaction Lipton ‘75: any sequence (R+B)*; (N+); (L+B)* is a transaction Other threads need not be scheduled in the middle of a transaction
Recall example:each thread has one transaction! local int y; acquire (m); x := x + 1; x := x + 1; x := x + 1; x := x +1; assert(x div 4); release (m); y = y+1; y = y+1; //initialize int x :=0; mutex m; local int z; acquire (m); x := x + 1; x := x + 1; x := x + 1; x := x +1; assert(x div 4); release (m); z = z+1; z = z+1;
Transaction-based reduction • ZOM extended to expose “mover-ness” of each action • Model checker maintains a state machine to track the “phase” of each transaction • Continues scheduling one thread as long as it is inside a transaction! • Current implementation: • Classifies all heap accesses as non-movers • Can improve the scalability using better analysis (ownership?)
Outline • Motivation for Zing • Zing overview • Exploiting structure for efficient model checking • Reduction • Summarization • Compositional conformance checking
Summarization for sequential programs • Procedure summarization (Sharir-Pnueli 81, Reps-Horwitz-Sagiv 95) is the key to efficiency int x; void incr_by_2() { x++; x++; } void main() { … x = 0; incr_by_2(); … x = 0; incr_by_2(); … } • Bebop, ESP, Moped, MC, Prefix, …
Assertion checking for sequential programs • Boolean program with: • g = number of global vars • m = max. number of local vars in any scope • k = size of the CFG of the program • Complexity is O( k 2 O(g+m) ),linear in the size of CFG • Summarization enables termination in the presence of recursion
Assertion checking for concurrent programs There is no algorithm for assertion checking of concurrent boolean programs, even with only two threads [Ramalingam 00]
Our approach • Precise semi-algorithm for verifying properties of concurrent programs • based on model checking • procedure summarization for efficiency • Termination for a large class of concurrent programs with recursion and shared variables • Generalization of precise interprocedural dataflow analysis for sequential programs
int x; void incr_by_2() { x++; x++; } void main() { … x = 0; incr_by_2(); … x = 0; incr_by_2(); … x = 1; incr_by_2(); … } What is a summary in sequential programs? • Summary of a procedure P = Set of all (pre-state post-state) pairs obtained by invocations of P x x’ 0 2 1 3
What is a summary in concurrent programs? • Unarticulated so far • Naïve extension of summaries for sequential programs do not work
If a procedure body is a single transaction, summarize as in a sequential program bool available[N]; mutex m; int getResource() { int i = 0; L0: acquire(m); L1: while (i < N) { L2: if (available[i]) { L3: available[i] = false; L4: release(m); L5: return i; } L6: i++; } L7: release(m); L8: return i; } Choose N = 2 Summaries: m, (a[0],a[1]) i’, m’, (a[0]’,a[1]’) 0, (0, 0) 2, 0, (0,0) 0, (0, 1) 1, 0, (0,0) 0, (1, 0) 0, 0, (0,0) 0, (1, 1) 0, 0, (0,1)
Transactional procedures • In the Atomizer benchmarks (Flanagan-Freund 04), a majority of procedures are transactional
What if a procedure body comprises multiple transactions? bool available[N]; mutex m[N]; int getResource() { int i = 0; L0: while (i < N) { L1: acquire(m[i]); L2: if (available[i]) { L3: available[i] = false; L4: release(m[i]); L5: return i; } else { L6: release(m[i]); } L7: i++; } L8: return i; } Choose N = 2 Summaries: pc,i,(m[0],m[1]),(a[0],a[1]) pc’,i’,(m[0]’,m[1]’),(a[0]’,a[1]’) L0, 0, (0,*), (0,*) L1, 1, (0,*), (0,*) L0, 0, (0,*), (1,*) L5, 0, (0,*), (0,*) L1, 1, (*,0), (*,0) L8, 2, (*,0), (*,0) L1, 1, (*,0), (*,1) L5, 1, (*,0), (*,0)
int x; mutex m; void foo() { acquire(m); x++; bar(); x--; release(m); } void bar() { release(m); acquire(m); } 1 2 • What if a transaction • starts in caller and ends in callee? • starts in callee and ends in caller?
What if a transaction • starts in caller and ends in callee? • starts in callee and ends in caller? int x; mutex m; void foo() { acquire(m); x++; bar(); x--; release(m); } void bar() { release(m); acquire(m); } 1 2 • Solution: • Split the summary into pieces • Annotate each piece to indicate whether • transaction continues past it
Two-level model checking • Top level performs state exploration • Bottom level performs summarization • Top level uses summaries to explore reduced set of interleavings, and reuse work • Maintains a stack for each thread • Pushes a stack frame if annotated summary edge ends in a call • Pops a stack frame if annotated summary edge ends in a return
Termination • Theorem: • If all recursive functions are transactional, then our algorithm terminates. • The algorithm reports an error iff there is an error in the program. [Qadeer-Rajamani-Rehof POPL 2004]
Summarization-based reduction • ZOM extended to expose procedure boundaries • Summarization implemented over transactions • In progress: • Benchmarking • Publication of implementation details
Outline • Motivation for Zing • Zing overview • Exploiting structure for efficient model checking • Reduction • Summarization • Compositional conformance checking
Goal • Check if all message-passing interactions are well-formed • No deadlocks • No unreceived messages • This requirement is called stuck-freeness • Exploit interface specifications • Check this compositionally
Interface A Intferface B Interface B Interface C Interface C InterfaceA B A A C Compositional conformance checking Conformance
Interface A Compostional conformance checking Interface B Interface C A Defects Pass
Stuck-free conformance Interface B Stuck-freeness preserved by all environments B Interface B A B A Stuck-free Stuck-free Preserves stuck-freeness
Stuck-free Conformance [Rajamani, Rehof CAV 2002] [Fournet, Hoare, Rajamani, Rehof CAV 2004]
Implementation • Generalize conformance definition from CCS to Zing • Expose sends and receives on external channels as “observable” actions • Disallow shared memory between processes • Can have multiple threads within each process communicating through shared memory • Make all other actions “internal” • Run specification and implementation “in parallel” and check for conformance. Finds several errors: • InventoryReservation: missing timeout specification [C2] • InventoryReservation: repeated input not specified [C1] • ShoppingCart: stuck state • Inventory: input not implemented in service [C2] • InventoryChangeNotification: inputs not available after receipt of Done
Summary • Model checking software • Challenges: richness of programming language, state explosion, environment modeling • Zing: a model checker for concurrent software • Modular architecture • Transaction based reduction • Summaries for procedures in concurrent programs • Compositional stuck-free conformance checking • Zing available for download! • http://research.microsoft.com/zing