Verification of Multithreaded Object-Oriented Programs with Invariants

1. Verification of Multithreaded Object-Oriented Programs with Invariants Bart Jacobs, K. Rustan M. Leino, Wolfram Schulte

2. Based On Barnett, DeLine, Fähndrich, Leino, Schulte. Verification of object-oriented programs with invariants. Journal of Object Technology, 2004. Hoare. Monitors: an operating system structuring concept. CACM, 1974.

3. Example in C# class Account { int balance; } Account act = …; int b0 = act.balance; act.balance += 50; int b1 = act.balance; b1 == b0 + 50? Not necessarily!

4. Example in C# lock (act) { b0 = act.balance; act.balance += 50; b1 = act.balance; } b1 == b0 + 50? Not necessarily!

5. Example in Spec# class Account { int balance; } Account act = …; int b0 = act.balance; act.balance += 50; int b1 = act.balance; Rejected by Spec# if act not owned

6. Example in Spec# acquire act; int b0 = act.balance; act.balance += 50; int b1 = act.balance; release act; b1 == b0 + 50? Always!

7. Observations • C# offers synchronization primitives • But their correct usage is not enforced • Spec#’s primitives are very similar • But their correct usage is enforced • This is a must for local reasoning

8. The Bottom Line • Programmers want to reason locally about field accesses • In C# this is unsound due to interference between threads • In Spec# this is sound because Spec# enforces mutual exclusion

9. What Will My Thread Do? • A thread can be influenced by other threads only at inter-thread I/O operations • Inter-thread I/O operations: • In C#: field accesses (= everywhere) • In Spec#: acquire and release operations

10. Spec# • Spec#: conservative extension of C# • Spec# compiler: Run-time checking • Requires no method contracts • Strictest mode guarantees safety, even in presence of malicious code • Boogie: static verification for Spec# • Sound

11. Overview • Methodology is introduced gradually • Per-object exclusion • Aggregate objects • Object invariants • Related work, future work, conclusion

12. Per-Object Exclusion • In each object, we introduce an owner field • owner is either null or a thread • Before thread t accesses field o.f, we check that o.owner == t

13. Per-Object Exclusion • We also introduce two new statements: • acquire o; • release o; • owner field can be modified only by the acquire and release statements

14. Per-Object Exclusion • When thread t creates an object o, we set o.owner = t; • When thread t acquires an object o, we first wait until o.owner == null and then set o.owner = t; • release o; checks that o.owner == t and then sets o.owner = null;

15. Example class Account { int balance; } Account act = new Account(); act.balance = 5; release act; … acquire act; act.balance++; release act;

16. Aggregate Objects • An aggregate object is one that uses other objects for its representation • We support treatment of aggregate objects with their representation as one entity • We support exchange of rep objects

17. Aggregate Objects • We introduce a rep field modifier to indicate fields that hold rep objects • The owner field of a rep object points to its aggregate object • (Thread objects have no rep fields; they denote the thread.)

18. Aggregate Objects • Before a thread can modify an aggregate object, it must unpack it • unpacking causes the rep objects to become owned by the thread • packing transfers ownership back to the aggregate object

19. Aggregate Objects • We introduce into each object a boolean field inv to indicate when it is being modified • This field is set by pack and reset by unpack • While !inv, rep modifiers are void

20. [[ o = new C; ]] o = new C;o.owner = t;o.inv = false; [[ x = o.f; ]] assert o.owner == t;x = o.f; [[ o.f = x; ]] assert o.owner == t;assert !o.inv;o.f = x; [[ acquire o; ]] lock (o) {while (o.owner != null) Monitor.Wait(o); o.owner = t;} [[ release o; ]] assert o.owner == t;assert o.inv;lock (o) { o.owner = null; Monitor.Pulse(o); // (*)} Aggregate Objects

21. [[ pack o; ]] assert o.owner == t;assert !o.inv;for each rep field o.f: if (o.f != null) {assert o.f.owner == t;assert o.f.inv; }for each rep field o.f: if (o.f != null) o.f.owner = o;o.inv = true; [[ unpack o; ]] assert o.owner == t;assert o.inv;for each rep field o.f: if (o.f != null) o.f.owner = t;o.inv = false; Aggregate Objects

22. Example Thread 1 class Agg { rep Rep f; }class Rep {} Agg a = new Agg;Rep r = new Rep;pack r;a.f = r;pack a;release a;

23. Example Thread 1 class Agg { rep Rep f; }class Rep {} Agg a = new Agg;Rep r = new Rep;pack r;a.f = r;pack a;release a; a

24. Example Thread 1 class Agg { rep Rep f; }class Rep {} Agg a = new Agg;Rep r = new Rep;pack r;a.f = r;pack a;release a; a r

25. Example Thread 1 class Agg { rep Rep f; }class Rep {} Agg a = new Agg;Rep r = new Rep;pack r;a.f = r;pack a;release a; a r

26. Example Thread 1 class Agg { rep Rep f; }class Rep {} Agg a = new Agg;Rep r = new Rep;pack r;a.f = r;pack a;release a; a r

27. Example Thread 1 class Agg { rep Rep f; }class Rep {} Agg a = new Agg;Rep r = new Rep;pack r;a.f = r;pack a;release a; a r

28. Example Thread 1 class Agg { rep Rep f; }class Rep {} Agg a = new Agg;Rep r = new Rep;pack r;a.f = r;pack a;release a; a r

29. Object Invariants • Behavior of thread is influenced externally only at acquire operations • Full thread-local reasoning requires knowledge about incoming aggregates • For this reason, we introduce object invariants

30. Object Invariants • Each class may declare an invariant • Invariant can mention only fields this.f1.f2.….fn.g where f1, f2, …, fn are rep fields • pack statement checks invariant • Therefore, invariant holds when inv

31. Object Invariants • release statement checks inv • Therefore, free aggregates satisfy their invariant • Cf. Hoare’s monitor invariants

32. Example: Local Reasoning class A { int p, q; invariant q != 0; } class B { rep A a; invariant a != null; } B b = …; acquire b; unpack b; unpack b.a; // OK, no NRE int n = b.a.p / b.a.q; // OK, no DBZE

33. Summary • Spec# ensures field accesses are thread-local operations • Supports aggregate objects • Object invariants are enforced • Full dynamic enforcement • Sound thread-local static checking

34. Some Related Work • ESC/Modula-3, ESC/Java • Safe Concurrent Java • Vault • Calvin/R • Atomizer • Eraser

35. ESC/Modula-3, ESC/Java • [Detlefs, Flanagan, Leino, Lillibridge, Nelson, Saxe, Stata 1998/2002] • Performs static checking • Each field is protected by a separately, freely chosen lock • Not formalized; no soundness proof

36. Safe Concurrent Java • [Boyapati, Lee, Rinard 2002] • Type system that supports thread-local, aggregate, and read-only objects, and unique pointers • Does not support object invariants • Does not support ownership transfer

37. Vault • [DeLine, Fähndrich 2002] • Uses linear types for mutual exclusion • Pack and unpack are implicit • Does not support general invariants

38. Calvin/R • [Freund, Qadeer 2004] • Specifies and verifies atomic transactions performed by a method • User specifies per-field protection • More flexible, but significantly more complex than Spec#

39. Atomizer • [Flanagan, Freund 2004] • Dynamically checks atomicity of unannotated methods • Enables sequential reasoning • We support sequential reasoning for non-atomic methods as well

40. Eraser • [Savage, Burrows, Nelson, Sobalvarro, Anderson 1997] • Finds data races in a running program • Found to be effective in practice • No guarantees about completeness or soundness

41. Current and Future Work • Assessing efficiency • Adding support for conditions, multiple reader threads, immutable objects, … • Liveness properties • Inference of annotations

42. Conclusion • Spec# ensures field accesses are thread-local operations • Supports aggregate objects • Object invariants are enforced • Full dynamic enforcement • Sound thread-local static checking