A Modular Checker for Multithreaded Programs

1. A Modular Checker for Multithreaded Programs Cormac Flanagan HP Systems Research Center Joint work with Shaz Qadeer Sanjit A. Seshia

2. Multithreaded programs • Operating systems, databases, web servers, file systems, web browsers • Large programs, many threads • Correctness problem: • for all inputs and interleavings, program satisfies specification • Testing ? • Frangipani filesystem took 1 year of testing • Linux kernel version 2.2 -> 2.4 • fine-grained locking to support multiprocessors • 2.4.* change log has 36 synchronization bug fixes

3. Verifying multithreaded programs Ashcroft 75 Owicki-Gries 76 Lamport 77 Jones 83 ... • Shared-memory • Many threads • Many procedures • Many synchronization idioms • locks (mutex, reader-writer) • semaphores (binary, counting) • changes in locking discipline • interrupt levels Sequential checkers: ... Extended Static Checking Dijkstra’s weakest precond. Automatic theorem proving ... • Many lines of code • Many language features • dynamic creation heap objects, unbounded # objs • pointers, pointer chasing, pointer smashing, ... • aliasing, arrays, arithmetic • classes, method, inheritance, overriding, ...

4. Contribution • New verification methodology for MT programs • thread-modular reasoning • procedure-modular reasoning • novel procedure specs. that respect module boundaries • generalization of Jones’ rely-guarantee method • Calvin • implements our methodology • automatic checker for multithreaded Java programs • leverages sequential checking technology (ESC)

5. Mutex module: int m := 0 // m = 0 => lock is not held // m = tid => lock held by thread tid // tid is id of current thread (non-zero int) acquire( ) { int t := tid while (t != 0) CAS(m,0,t) } if (m = 0) then m,t := t,m Simple multithreaded program Main module: int x := 0 Thread 1: acquire( ) x := x + 1 assert x > 0 release( ) Thread 2: acquire( ) x := 0 release( ) release( ) { m := 0 }

6. Problems • Many threads • use thread-modular (assume-guarantee) reasoning

7. Other threads int t := 1 while (t != 0) CAS(m,0,t) x := x + 1 assert x > 0 m := 0 * * * * * * Invariant I  m = 0  x >= 0 Env. assumption Atid  I I’ m = tid  m’ = m  x’ = xm,x Thread 1 acquire() x := x + 1 assert x > 0 release() Env assumption A1* int t := 1 A1* while (t != 0) A1* CAS(m,0,t) A1* x := x + 1 A1* assert x > 0 A1* m := 0 Inline body int t := 1 while (t != 0) CAS(m,0,t) x := x + 1 assert x > 0 m := 0 Simplify A1 int t := 1 A1 while (t != 0) A1 CAS(m,0,t) A1 x := x + 1 A1 assert x > 0 A1 m := 0 • Check using ESC • Also check actions of Thread1 satisfies A2 • Similar checks for Thread2 • Program is OK

8. Calvin version 1 • Verifies multithreaded Java programs • Inlines procedures • Thread-modular reasoning • requires environment assumption • reduces correctness of MT program to correctness of sequential programs • Leverages Extended Static Checker for Java • generates VC for sequential programs • uses automatic theorem prover to check VC

9. The Apprentice Challenge • Proposed by Moore and Porter • 50 lines of Java code • requires reasoning about many language features • pointers, aliasing, arithmetic, infinite state • dynamic allocation of new objects & threads • ACL2: 1 week • Calvin: 1 afternoon, 6 annotations

10. Problems • Many threads • use thread-modular (assume-guarantee) reasoning • Many procedures • use procedure-modular reasoning

11. What are the specifications? Main module: int x := 0 Thread 1: acquire( ) x := x + 1 assert x > 0 release( ) Thread 2: acquire( ) x := 0 release( ) Mutex module: int m := 0 acquire( ) { int t := tid while (t != 0) CAS(m,0,t) } release( ) { m := 0 } spec. for acquire( ) spec. for release( )

12. Procedure specification (Jones) precondition environment assumption procedure body guarantee postcondition

13. Simple multithreaded program Main module: int x := 0 Thread 1: acquire( ) x := x + 1 assert x > 0 release( ) Thread 2: acquire( ) x := 0 release( ) Mutex module: int m := 0 precondition: I env_assumption: I  I’  m = tid  m’ = m  x’ = xm,x guarantee: I  I’  m != 0  m’ = mm postcondition: m = tid  x >= 0 acquire( ) { int t := tid while (t != 0) CAS(m,0,t) } release( ) { m := 0 }

14. Procedure specifications (Calvin) precondition environment assumption procedure body guarantee (action) abstraction (program) postcondition

15. Simple multithreaded program Main module: int x := 0 Thread 1: acquire( ) x := x + 1 assert x > 0 release( ) Thread 2: acquire( ) x := 0 release( ) Mutex module: int m := 0 env_assumption: true abstraction: skip* m = 0 m’ = tidm skip* acquire( ) { int t := tid while (t != 0) CAS(m,0,t) } release( ) { m := 0 } env_assumption: true abstraction: m’ = 0m

16. Other threads skip* m=0m’=tidm skip* x := x + 1 assert x > 0 m’ = 0m * * * * * * Invariant I  m = 0  x >= 0 Env. assumption Atid  I I’ m = tid  m’ = m  x’ = xm,x Env assumption A1* skip* A1* m=0m’=tidm A1* skip* A1* x := x + 1 A1* assert x > 0 A1* m’ = 0m Thread 1 acquire() x := x + 1 assert x > 0 release() Inline spec skip* m=0m’=tidm skip* x := x + 1 assert x > 0 m’ = 0m Simplify A1 m=0m’=tidm A1 x := x + 1 A1 assert x > 0 A1 m’ = 0m • Check using ESC • Theorem prover for additional checks • Ditto for Thread2 • Program is OK

17. Verifying Mutex module Implementation: acquire( ) { int t := tid while (t != 0) { CAS(m,0,t) } } env_assumption: true abstraction: skip* m = 0 m’ = tidm skip* Implementation is simulated by abstraction with respect to environment assumption

18. m!=tid m!=0 m=0 m=tid Simulation Witness Abstraction: Implemetation: skip int t := tid t != 0 CAS(m,0,t) m==0  m’==tidm yes no skip end

19. Calvin version 2 • Verifies multithreaded Java programs • Thread-modular reasoning • Procedure-modular reasoning • novel procedure specification mechanism • specifications respect module boundaries • check procedure body is simulated by abstraction • use procedure abstraction at call site • Reduces to correctness of sequential program • leverage ESC

20. Producer || Consumer increasing integers check increasing spec spec int get() put(int n) spec spec acquire( ) { ... } release( ) { ... } Producer-consumer Producer- consumer Queue Mutex 12 11 10 9 8 7 6 5

21. Real multithreaded programs • java.util.Vector • 400 lines of code • 2 environment assumptions • specify locking discipline • Calvin reported 1 race condition • Mercator web crawler • uses reader-writer locks • specified and verified 4 locking primitives • acquire_read(), release_read(), acquire_write(), release_write() • checked 1500 LOC that uses reader-writer locks • verified absence of races on many data structures

22. Daisy file system • Multithreaded • synchronization as in high-performance FS • Verifiable file system • small, 1500 LOC • straightforward data structures • Functional • can mount as NFS partition • Verified absence of race conditions • Verified specifications of many procedures • ialloc is correct • create is atomic

23. interactive theorem provers: PVS, ACL2, ... Scalability Calvin model checkers: JPF, Bandera, SPIN, ... Automation Note: graph is not to scale Conclusion • Checking multithreaded programs requires scalable and automated tools