Lazy Abstraction

Lazy Abstraction Lecture2: Modular Analyses Ranjit Jhala UC San Diego With: Tom Henzinger, Rupak Majumdar, Ken McMillan, Gregoire Sutre

Program Verification byLazy Abstraction Lecture1 Ranjit Jhala UC San Diego With: Tom Henzinger, Rupak Majumdar, Ken McMillan, Gregoire Sutre

Last lecture … Lazy Abstraction for Sequential Programs • Predicates: • Abstract infinite program states • Counterexample-guided Refinement: • Find predicates tailored to prog, property • Abstraction : Expensive Reachability Tree • Refinement : Find predicates, use locations Proof of unsat of TF + Interpolation

This Lecture: Modular Analyses Procedures - Summaries Concurrency - Thread-Context Reasoning

An example main(){  1: if (flag){ 2: y = inc(x,flag); 3: if (y<=x) ERROR; } else { 4: y = inc(z,flag); 5: if (y>=z) ERROR; }  return; } inc(int a, int sign){ 1: if (sign){ 2: rv = a+1; } else { 3: rv = a-1; } 4: return rv; }

Inline Calls in Reach Tree main(){  1: if (flag){ 2: y = inc(x,flag); 3: if (y<=x) ERROR; } else { 4: y = inc(z,flag); 5: if (y>=z) ERROR; }  return; } Initial 1 4 2 1,4 1,2 2,4 3,4 2,2 3,2 inc(int a, int sign){ 1: if (sign){ 2: rv = a+1; } else { 3: rv = a-1; } 4: return rv; } 4,4 4,4 4,2 4,2 5 5 3 3

Inline Calls in Reach Tree Problem • Repeated analysis for “inc” • Exploding call contexts Initial 1 4 2 int x; //global f1(){ 1: x = 0; 2: if(*) f2(); 3: else f2(); 4: if (x<0) ERROR; return; } 1,4 1,2 f2(){ 1: if(*) f3(); 2: else f3(); return; } f3(){ 1: if(*) f4(); 2: else f4(); return; } 2,4 3,4 2,2 3,2 f4(){ 1: if(*) f5(); 2: else f5(); return; } fn(){ 1: x ++; return; } 4,4 4,4 4,2 4,2 5 5 3 3 2nnodes in Reach Tree

Inline Calls in Reach Tree Problem • Repeated analysis for “inc” • Exploding call contexts • Cyclic call graph (Recursion) • Infinite Tree! Initial 1 4 2 1,4 1,2 2,4 3,4 2,2 3,2 4,4 4,4 4,2 4,2 5 5 3 3

Solution : Procedure Summaries Summaries: Input/Output behavior • Plug summaries in at each callsite … instead of inlining entire procedure [Sharir-Pnueli 81, Reps-Horwitz-Sagiv 95] • Summary = set of (F  F’) • F : Precondition formula describing input state • F’ : Postcondition formula describing output state

Solution : Procedure Summaries Summaries: Input/Output behavior • Plug summaries in at each callsite … instead of inlining entire procedure [Sharir-Pnueli 81, Reps-Horwitz-Sagiv 95, Ball-Rajamani 01] • Summary = set of (F  F’) • F : Precondition formula describing input state • F’ : Postcondition formula describing output state inc(int a, int sign){ 1: if (sign){ 2: rv = a+1; } else { 3: rv = a-1; } 4: return rv; } • (: sign=0 rv > a) • (sign = 0 rv < a) Q. How to compute, use summaries ?

Lazy Abstraction + Procedure Summaries Yes Safe Abstract CProgram Refine No Property Trace Q. How to compute, use summaries ?

Abstraction with Summaries main main(){  1: if (flag){ 2: y = inc(x,flag); 3: if (y<=x) ERROR; } else { 4: y = inc(z,flag); 5: if (y>=z) ERROR; }  return; } 1 [flag!=0] 2 : flag=0 a=x sign=flag : sign=0 inc(int a, int sign){ 1: if (sign){ 2: rv = a+1; } else { 3: rv = a-1; } 4: return rv; } Predicates:flag=0 , y>x , y<z sign=0 , rv>a , rv<a

Abstraction with Summaries inc main main(){  1: if (flag){ 2: y = inc(x,flag); 3: if (y<=x) ERROR; } else { 4: y = inc(z,flag); 5: if (y>=z) ERROR; }  return; } 1 1 : sign=0 [sign!=0] 2 : flag=0 2 : sign=0 a=x sign=flag rv=a+1 : sign=0 4 rv>a inc(int a, int sign){ 1: if (sign){ 2: rv = a+1; } else { 3: rv = a-1; } 4: return rv; } Summary:(: sign=0  rv>a), Predicates:flag=0 , y>x , y<z sign=0 , rv>a , rv<a

Summary Successor inc main main(){  1: if (flag){ 2: y = inc(x,flag); 3: if (y<=x) ERROR; } else { 4: y = inc(z,flag); 5: if (y>=z) ERROR; }  return; } 1 1 : sign=0 a=x sign=flag 2 : flag=0 2 assume rv>a 3 y>x y=rv 4 rv>a inc(int a, int sign){ 1: if (sign){ 2: rv = a+1; } else { 3: rv = a-1; } 4: return rv; } Summary:(: sign=0  rv>a), Predicates:flag=0 , y>x , y<z sign=0 , rv>a , rv<a

Abstraction with Summaries inc main main(){  1: if (flag){ 2: y = inc(x,flag); 3: if (y<=x) ERROR; } else { 4: y = inc(z,flag); 5: if (y>=z) ERROR; }  return; } 1 1 : sign=0 [flag==0] [sign=0] 4 2 flag=0 : flag=0 3 2 3 sign=0 y>x 4 rv>a [y<=x] a=x sign=flag inc(int a, int sign){ 1: if (sign){ 2: rv = a+1; } else { 3: rv = a-1; } 4: return rv; } Summary:(: sign=0  rv>a), Predicates:flag=0 , y>x , y<z sign=0 , rv>a , rv<a

Abstraction with Summaries inc main main(){  1: if (flag){ 2: y = inc(x,flag); 3: if (y<=x) ERROR; } else { 4: y = inc(z,flag); 5: if (y>=z) ERROR; }  return; } 1 1 1 sign=0 : sign=0 4 2 flag=0 : flag=0 3 2 3 2 3 sign=0 y>x 4 4 rv>a rv<a a=x sign=flag inc(int a, int sign){ 1: if (sign){ 2: rv = a+1; } else { 3: rv = a-1; } 4: return rv; } Summary:(: sign=0  rv>a), (sign=0  rv<a) Predicates:flag=0 , y>x , y<z sign=0 , rv>a , rv<a

Summary Successor inc main main(){  1: if (flag){ 2: y = inc(x,flag); 3: if (y<=x) ERROR; } else { 4: y = inc(z,flag); 5: if (y>=z) ERROR; }  return; } 1 1 1 sign=0 : sign=0 4 2 flag=0 : flag=0 3 2 3 2 3 3 y>x y<z 4 4 rv>a rv<a a=x sign=flag inc(int a, int sign){ 1: if (sign){ 2: rv = a+1; } else { 3: rv = a-1; } 4: return rv; } assume rv<a y=rv Summary:(: sign=0  rv>a), (sign=0  rv<a) Predicates:flag=0 , y>x , y<z sign=0 , rv>a , rv<a

Abstraction with Summaries inc main main(){  1: if (flag){ 2: y = inc(x,flag); 3: if (y<=x) ERROR; } else { 4: y = inc(z,flag); 5: if (y>=z) ERROR; }  return; } 1 1 1 sign=0 : sign=0 4 2 flag=0 : flag=0 3 2 3 2 3 3 y<z y>x 4 4 rv>a rv<a [y>=z] inc(int a, int sign){ 1: if (sign){ 2: rv = a+1; } else { 3: rv = a-1; } 4: return rv; } Summary:(: sign=0  rv>a), (sign=0  rv<a) Predicates:flag=0 , y>x , y<z sign=0 , rv>a , rv<a

Another Call … inc main main(){  1: if (flag){ 2: y = inc(x,flag); 3: if (y<=x) ERROR; } else { 4: y = inc(z,flag); 5: if (y>=z) ERROR; } 6: y1 = inc(z1,1); 7: if (y1<=z1) ERROR; return; } 1 1 1 sign=0 : sign=0 4 2 flag=0 : flag=0 3 2 3 2 3 3 y<z y>x 4 4 rv>a rv<a inc(int a, int sign){ 1: if (sign){ 2: rv = a+1; } else { 3: rv = a-1; } 4: return rv; } 6 6 a=z1 sign=1 : sign=0 Predicates:flag=0 ,y>x,y<z, y1>z1 sign=0 , rv>a , rv<a Summary:(: sign=0  rv>a), (sign=0  rv<a)

Another Call … inc main main(){  1: if (flag){ 2: y = inc(x,flag); 3: if (y<=x) ERROR; } else { 4: y = inc(z,flag); 5: if (y>=z) ERROR; } 6: y1 = inc(z1,1); 7: if (y1<=z1) ERROR; return; } 1 1 1 sign=0 : sign=0 4 2 flag=0 : flag=0 3 2 3 2 3 3 y<z y>x 4 4 rv>a rv<a Note: Predicates are well-scoped … inc(int a, int sign){ 1: if (sign){ 2: rv = a+1; } else { 3: rv = a-1; } 4: return rv; } 6 6 SAFE a=z1 sign=1 7 y1>z1 assume rv>a y1=rv Summary:(: sign=0  rv>a), (sign=0  rv<a) Predicates:flag=0 ,y>x,y<z, y1>z1 sign=0 , rv>a , rv<a

Lazy Abstraction + Procedure Summaries Yes Safe Abstract CProgram Refine No Property Trace Q. How to find scoped predicates ?

Traces with Procedure Calls Trace Trace Formula pc1: x1 = 3 pc2: assume (x1>0) pc3: x3 = f1(x1) pc4: y2 = y1 pc5: y3 = f2(y2) pc6: z2 = z1+1 pc7: z3 = 2*z2 pc8: return z3 pc9: return y3 pc10: x4 = x3+1 pc11: x5 = f3(x4) pc12: assume(w1<5) pc13: return w1 pc14: assume x4>5 pc15: assume(x1=x3+2) pc1: x1 = 3 pc2: assume (x1>0) pc3: x3 = f1(x1) pc4: y2 = y1 pc5: y3 = f2(y2) pc6: z2 = z1+1 pc7: z3 = 2*z2 pc8: return z3 pc9: return y3 pc10: x4 = x3+1 pc11: x5 = f3(x4) pc12: assume(w1<5) pc13: return w1 pc14: assume x4>5 pc15: assume (x1=x3+2) Find predicate needed at point i i i

Interprocedural Analysis Trace Trace Formula NO Find predicate needed at point i YES i i NO Require at each point i: Scopedpredicates YES: Variables visibleat i NO: Caller’s local variables

Problems with Cutting Trace Trace Formula - i i + Caller variablescommon to - and + • Unsuitable interpolant: not well-scoped

Scoped Cuts Trace Trace Formula Call begins i i

Scoped Cuts Trace Trace Formula Call begins + - i i Predicate at pci= Interpolant from cut i

Common Variables Trace Trace Formula Common Variables Formals + - Formals Current locals i i Well-scoped Predicate at pci= Interpolant from i-cut

Example Trace main(){  1: if (flag){ 2: y = inc(x,flag); 3: if (y<=x) ERROR; } else { 4: y = inc(z,flag); 5: if (y>=z) ERROR; }  return; } m1: assume(flag!=0) m2: y=inc(x,flag) i1: assume(sign!=0) i2: rv = a+1 i4: return rv m3: assume(y<=x) ERROR: inc(int a, int sign){ 1: if (sign){ 2: rv = a+1; } else { 3: rv = a-1; } 4: return rv; }

Trace Formula m1: assume(flag!=0) m2: y=inc(x,flag) i1: assume(sign!=0) i2: rv = a+1 i4: return rv m3: assume(y<=x) m1: assume(flag0!=0) m2: a0=x0, sign0=flag0 i1: assume(sign0!=0) i2: rv0=a0+1 i4: y0=rv0 m3: assume(y0 <= x0) SSA Trace Trace

Trace Formula m1: assume(flag!=0) m2: y=inc(x,flag) i1: assume(sign!=0) i2: rv = a+1 i4: return rv m3: assume(y<=x) Æ flag0 0 Æ a0=x0Æ sign0= flag0 Æ sign0 0 Æ rv0=a0+1 Æ y0=rv0 Æ y0 <= x0 Call begins i i Trace Trace Formula

Trace Formula m1: assume(flag!=0) m2: y=inc(x,flag) i1: assume(sign!=0) i2: rv = a+1 i4: return rv m3: assume(y<=x) Æ flag0 0 Æ a0=x0Æ sign0= flag0 Æ sign0 0 Æ rv0=a0+1 Æ y0=rv0 Æ y0 <= x0 + - i i Trace Trace Formula

Trace Formula m1: assume(flag!=0) m2: y=inc(x,flag) i1: assume(sign!=0) i2: rv = a+1 i4: return rv m3: assume(y<=x) Æ flag0 0 Æ a0=x0Æ sign0= flag0 Æ sign0 0 Æ rv0=a0+1 Æ y0=rv0 Æ y0 <= x0 a0 Interpolate + - a0 rv0 i i rv0 rv0> a0 Trace Trace Formula

Lazy Abstraction + Procedure Summaries Yes Safe Abstract CProgram Refine No Property Trace Q. How to find scoped predicates ? Solution: Scoped Cuts + Interpolation

Review : Procedures Modular Analysis viaSummaries • Summary = (In,Out) vertices’ formulas Requires scoped predicates • Scoped cuts + Interpolation

This Lecture: Modular Analyses Procedures - Summaries Concurrency - Thread-Context Reasoning

OS, WebServers, Databases, Embedded Systems Curse of Interleaving Non-deterministic scheduling Exponentially many behaviors: hard to detect, reproduce errors Testing exercises a fraction of possible behaviors Multithreaded Programs Thread Thread x Shared Memory

A data race on x is a state where: Two threads can access x One of the accesses is a write Unpredictable, undesirable Synchronization: Must hold lock when accessing x Data Races  x:= x+1   x:= x-5  lock(l) unlock(l) lock(l) unlock(l) x

[Dinning-Schonberg 90] [Savage et al. 97] [Cheng et al. 98] [Choi et al. 02] Dynamic LockSet [Flanagan-Freund 00] [Bacon et al. 00] [Boyapati et al. 02] Type-based Static LockSet [Sterling 93], [Engler-Ashcraft 03] [von Praun-Gross 03] Object Usage Graph Previous Work : Locks  x:= x+1  lock(l) unlock(l) • Infer some lock(s) that protect x • Check lock(s)held when accessing x • Report error if lock(s) not held Scalable Restricted to locking

Other Synchronization Idioms atomic{ old:= state; if(state==0){ state:=1; } }  if(old==0){ x++; state:=0; } x Producer-Consumer Interrupt-toggling State-based

Previous Work : Model Checking atomic{ old:= state; if(state==0){ state:=1; } }  if(old==0){ x++; state:=0; } x Producer-Consumer Interrupt-toggling State-based [Godefroid 97] [Holzmann][Havelund-Visser] [Dwyer-Hatcliff][Avrunin-Clarke] [Musuvathi-Dill-Engler 02] [Yahav 01] Any Synch. Idiom Fixed #threads Manual Abstraction State Explosion Model Checking (State Exploration)

Race Checking by State Exploration Initial Shared Memory Is there a path from Initial to Race ? Data Race

Problem: State Explosion Initial Shared Memory Is there a path from Initial to Race ? Data Race

Problem: State Explosion Initial 2. Control k threads, m locations =mk - k=4,m=100, states = 1 billion Unbounded threads ? 1. Data Infinitely many valuations for program variables Data Race

LA for Multithreaded Programs • Data Races • Previous Work • State Explosion • Abstractions • Context Inference

Problem: State Explosion Initial 2. Control k threads, m locations =mk - k=4,m=100, states = 1 billion Unbounded threads ? 1. Data Infinitely many valuations for program variables Data Race

Solution: Abstract Irrelevant Detail Observe - Few relevant variables, relationships - Track predicates (relationships) instead of values 1. Predicate Abstraction 1. Data Infinitely many valuations for program variables Observe - Analyze system as Thread + Context - Context: Summary of all other threads (w.r.t. property) 2. Thread-Context Analysis 2. Control k threads, m locations =mk - k=4,m=100, states = 1 billion Unbounded threads ?

Example Check for races onx: Initially: s is 0 1st thread into atomic : - sets old to 0 (value of s) - sets s to 1 - passes test before access Later threads: - set old to 1 (value set by 1st thread) - fail test before access (until the 1st thread is done) 1: while(1){ atomic{ 2: old := s; 3: if(s==0){ 4: s := 1; } } // do_work()  5: if(old==0){ 6: x++; 7: s:=0;} }

LA for Multithreaded Programs • Data Races • Previous Work • State Explosion • Abstractions • Data: Predicate Abstraction • Control: Thread-Context Reasoning • Verifying Multithreaded Programs

Lazy Abstraction