The Spec# Programming System: An Overview

1. The Spec# Programming System: An Overview Tutor: Bart Jacobs PhD student at K.U.Leuven, Belgium Spec# contributor Tutorial at FM2005

2. What is Spec#? • A programming language that extends C# with specification constructs • A compiler that emits run-time checks for the specification constructs • A static verifier that modularly proves that the run-time checks never fail • An environment with base class library contracts and a Visual Studio extension Tutorial at FM2005

3. Spec# Goals • Make it easier to record detailed design decisions • Provide tools to enforce these decisions • Help prevent and detect bugs • Reduce cost of software lifecycle Tutorial at FM2005

4. Demo: The Bag Class Demonstrates • Non-null types • Method contracts (including out-of-band contracts for system libraries) • Loop invariants • The object invariant methodology • The static verifier Tutorial at FM2005

5. Tutorial • What is Spec#? • Non-null types • Method contracts • Object invariants • Ownership ―Break― • Inheritance • State Abstraction • Multithreading Tutorial at FM2005

6. Tutorial • What is Spec#? • Non-null types • Method contracts • Object invariants • Ownership ―Break― • Inheritance • State Abstraction • Multithreading Tutorial at FM2005

7. Non-null types • Each reference type T includes the value null • Spec#’s type T! contains only references to objects of type T(notnull). Tutorial at FM2005

8. Types versus Assertions • Without non-null types: Person(string name) requires name != null; • With non-null types: Person(string/*^!^*/ name) [Q] What is the difference? Tutorial at FM2005

9. Non-null types are flow-sensitive • The non-null type of an expression is flow-sensitive. void Foo(T o) { if (o != null) T! p = o; // OK! } That is, it does not follow uniquely from the declared types of the variables and members mentioned in the expression. Tutorial at FM2005

10. class C { public C() { ((D) this).f.Foo(); } } class D : C { T! f; D(T! x) : base() { f = x; } } [Q] Can this be unsound? [A] Yes! Non-null Fields and Object Creation class D : C { T! f; D(T! x) { f = x; base(); } } Tutorial at FM2005

11. Non-nullness of Fields Common coding pattern: if (o.f != null) o.f.Foo(); [Q] How can this go wrong? Tutorial at FM2005

12. Non-nullness of Properties Common coding pattern: if (o.P != null) o.P.Foo(); [Q] How can this go wrong? Tutorial at FM2005

13. Fields and Properties • For the non-null dataflow analysis, we assume that non-nullness of fields and properties is preserved in the absence of intervening heap-modifying operations • Property reads are not considered heap-modifying operations • But we check this at run time because of the possibility of • Data races • Impure property getters Tutorial at FM2005

14. Tutorial • What is Spec#? • Non-null types • Method contracts • Object invariants • Ownership ―Break― • Inheritance • State abstraction • Multithreading Tutorial at FM2005

15. Preconditions staticint Divide(int a, int b) { if (b==0) throw new ArgumentException(); return a / b; } [Q] What’s wrong with this? staticint Divide(int a, int b) /*^requires b != 0;^*/ { return a / b; } • Throws a RequiresException if false. But how can we get full backwards compatibility? staticint Divide(int a, int b) requires b != 0 otherwise ArgumentException; { return a / b; } Tutorial at FM2005

16. Checked Exceptions int Eval(Expr e) throws EvalErrorException; • Alternative: bool TryEval(Expr e, out int value, out string errorMessage) + explicit propagation through recursive calls Tutorial at FM2005

17. Definedness of contracts public static int BinarySearch (int[] a, int value) ensuresresult < a.Length; ensuresa[result] == value; [Q] What if result < 0 ? This contract is ill-defined. In Spec#, an ill-defined contract is considered an error. If at run time, evaluation of a contract clause throws an exception, the exception is wrapped in an InvalidContractException and propagated. For static checking, the program verifier generates an error if it cannot prove that a contract is well-defined. Tutorial at FM2005

18. Tutorial • What is Spec#? • Non-null types • Method contracts • Intermezzo: Inside the Spec# Program Verifier • Object invariants • Ownership ―Break― • Inheritance • State abstraction • Multithreading Tutorial at FM2005

19. From Spec#... static int Abs(int x) ensures 0 <= x ==>result == x; ensures x < 0 ==> result == -x; { if (x < 0) x = -x; return x; } Tutorial at FM2005

20. …via BoogiePL … procedure Abs(x$in: int) returns ($result: int); ensures 0 <= x$in ==> $result == x$in; ensures x$in < 0 ==> $result == -x$in; { var x1, x2: int, b: bool; entry: x1 := x$in; b := x < 0; goto t, f; t: assume b; x := -x; goto end; f: assume !b; goto end; end: $result := x; return; } Tutorial at FM2005

21. …via BoogiePL-DSA … procedure Abs(x$in: int) returns ($result: int); ensures 0 <= x$in ==> $result == x$in; ensures x$in < 0 ==> $result == -x$in; { var x1, x2: int, b: bool; entry: x1 := x$in; b := x1 < 0; goto t, f; t: assume b; x2 := -x1; goto end; f: assume !b; x2 := x1; goto end; end: $result := x2; return; } Tutorial at FM2005

22. …via Passive BoogiePL … procedure Abs(x$in: int) returns ($result: int); ensures 0 <= x$in ==> $result == x$in; ensures x$in < 0 ==> $result == -x$in; { var x1, x2: int, b: bool; entry: assume x1 == x$in; assume b == x1 < 0; goto t, f; t: assume b; assume x2 == -x1; goto end; f: assume !b; assume x2 == x1; goto end; end: assume $result == x2; return; } Tutorial at FM2005

23. … without contracts … procedure Abs(x$in: int) returns ($result: int); ensures 0 <= x$in ==> $result == x$in; ensures x$in < 0 ==> $result == -x$in; { var x1, x2: int, b: bool; entry: assume x1 == x$in; assume b == x1 < 0; goto t, f; t: assume b; assume x2 == -x1; goto end; f: assume !b; assume x2 == x1; goto end; end: assume $result == x2; return; } Tutorial at FM2005

24. … without contracts … procedure Abs(x$in: int) returns ($result: int); { var x1, x2: int, b: bool; entry: assume x1 == x$in; assume b == x1 < 0; goto t, f; t: assume b; assume x2 == -x1; goto end; f: assume !b; assume x2 == x1; goto end; end: assume $result == x2; assert 0 <= x$in ==> $result == x$in; assert x$in < 0 ==> $result == -x$in; return; } Tutorial at FM2005

25. …to Logic [M. Barnett, K. R. M. Leino, in preparation] entry && (entry <== (x1 == x$in ==> b == x1 < 0 ==> t && f)) && (t <== (b ==> x2 == -x1 ==> end)) && (f <== (!b ==> x2 == x1 ==> end)) && (end <== ($result == x2 ==> (0 <= x$in ==> $result == x$in) && (x$in < 0 ==> $result == -x$in) && true)) Tutorial at FM2005

26. Tutorial • What is Spec#? • Non-null types • Method contracts • Object invariants • Ownership ―Break― • Inheritance • State abstraction • Multithreading Tutorial at FM2005

27. Object Invariants class Word { privatestring!line; int start, length; public string ToString() { return line.Substring(start, length); } } [Q] How can we prove ToString? Tutorial at FM2005

28. Object Invariants class Word { private string! line; int start, length; invariant 0 <= start && 0 <= length; invariant start + length <= line.length; public string ToString() { return line.Substring(start, length); } } [Q] How can we prove ToString? Tutorial at FM2005

29. class Subject { … invariant …; Observer d; void Foo(…) { // invariant assumed … if (…) d.Notify(…); … // invariant reestablished } } class Observer { … Subject c; void Notify(…) { … if (…) c.Foo(…); … } } Re-entrancy [Q] What could go wrong here? ==> Let’s drop the assumption that invariants hold on method entry. [Q] But then, when can we assume they hold? Tutorial at FM2005

30. class Subject { … invariantI; Observer d; void Foo(…) requiresI; { // invariant assumed … if (…) d.Notify(…); … // invariant reestablished } } class Observer { … Subject c; void Notify(…) { … if (…) c.Foo(…); … } } Re-entrancy [Q] Can we simply require that the invariant holds? No! This breaks abstraction! Tutorial at FM2005

31. Spec# Object Invariant Methodology • Each object gets a boolean field inv • Regular fields may be modified only when inv is false • inv field may be modified only using special BeginExpose() and EndExpose() calls • BeginExpose() sets inv to false • EndExpose() checks invariant; if it holds, sets inv to true; otherwise, throws ObjectInvariantException • Therefore, if inv is true, the invariant holds Tutorial at FM2005

32. class Subject { … invariant …; Observer d; void Foo(…) requires inv; { … // uses invariant (sound!) BeginExpose(); … // field updates if (…) d.Notify(…); … // restores invariant EndExpose(); } } Exposing objects Tutorial at FM2005

33. class Subject { … invariant …; Observer d; void Foo(…) requires inv; { … // uses invariant (sound!) expose (this) { … // field updates if (…) d.Notify(…); … // restores invariant } } } class Observer { … Subject c; void Notify(…) { … if (…) c.Foo(…); … } } Exposing objects [Q] Did we solve the problem? Yes! This call is not allowed; precondition does not hold! Tutorial at FM2005

34. Object Invariants: Example class Word { privatestring!line; privateint start; public int length; invariant 0 <= start && 0 <= length; invariant start + length <= line.length; public void SelectPart(int start, int length) requires inv; requires 0 <= start && 0 <= length; requires length <= this.length; ensures inv; { expose (this) { this.start += start; this.length = length; } } } A method typically requires and ensures inv Also, the body is typically wrapped in anexpose (this)block Tutorial at FM2005

35. Object Invariants and Object Creation class Word { private string! line; private int start; public int length; invariant 0 <= start && 0 <= length; invariant start + length <= line.Length; public Word(string! line, int start, int length) requires 0 <= start && 0 <= length; requires start + length <= line.Length; ensures inv; { this.line = line; this.start = start; this.length = length; base(); EndExpose(); } } When an object is created, its inv field is false. The constructor typically initializes the fields, establishing the invariant, and then calls EndExpose() to set the inv field. Tutorial at FM2005

36. Object Invariants: Recap • Spec# supports object invariants • To avoid reentrancy problems, we introduce an inv field; it abstracts the invariant and may be used in contracts • BeginExpose() and EndExpose() calls (or expose blocks) toggle inv; they delimit the regions where the invariant does not need to hold and where the object may be modified Tutorial at FM2005

37. Invariants and Exceptions [Q] Should the expose block perform an EndExpose() if the body terminates with a checked exception? Yes – throwing a checked exception does not mean the object is broken. Future calls on the object need to be able to rely on the invariant. Note that if the invariant does not hold, the EndExpose() call replaces the checked exception with an ObjectInvariantException. class EOFException : CheckedException {} byte ReadByte() throws EOFException { expose (this) { … if (…) throw EOFException(); … } } Tutorial at FM2005

38. Invariants and Exceptions [Q] Should the expose block perform an EndExpose() if the body terminates with an unchecked exception? No --- the object is broken. Leaving it exposed will prevent future method calls on the object. Also, performing an EndExpose() might replace the original exception with an ObjectInvariantException, masking the real error. void Foo(int a, int b) { expose (this) { … … a / b … … } } Tutorial at FM2005

39. Tutorial • What is Spec#? • Non-null types • Method contracts • Object invariants • Ownership ―Break― • Inheritance • State abstraction • Multithreading Tutorial at FM2005

40. Inter-object Invariants class Account { List<int> deposits; int balance; invariant balance == Math.Sum(deposits); public List<int> GetDeposits() { return deposits; } public static List<int> GetAllDeposits(Account a1, Account a2) { List<int> ds = a1.GetDeposits(); ds.AddRange(a2.GetDeposits()); return ds; } } [Q] Is this okay? Oops! This modifies a1’s list of deposits. Also, a1’s invariant is now broken. Spec# solution: Allow the list of deposits to be modified only when the account object is exposed. Tutorial at FM2005

41. Inter-object Invariants This is achieved by having the account object own the list of deposits whenever the former is not exposed. class Account { [Owned]List<int> deposits; int balance; invariant balance == Math.Sum(deposits); public List<int> GetDeposits() { return deposits; } public static List<int> GetAllDeposits(Account a1, Account a2) { List<int> ds = a1.GetDeposits(); ds.AddRange(a2.GetDeposits()); return ds; } } [Q] Is this okay? Oops! This modifies a1’s list of deposits. Also, a1’s invariant is now broken. Spec# solution: Allow the list of deposits to be modified only when the account object is exposed. Tutorial at FM2005

42. Inter-object Invariants This is achieved by having the account object own the list of deposits whenever the former is not exposed. class Account { [Owned]List<int> deposits; int balance; invariant balance == Math.Sum(deposits); public List<int> GetDeposits() { return deposits; } public static List<int> GetAllDeposits(Account a1, Account a2) { List<int> ds = a1.GetDeposits(); ds.AddRange(a2.GetDeposits()); return ds; } } [Q] Is this okay? ds.BeginExpose() fails if ds is owned, so AddRange won’t succeed in modifying ds. Oops! This modifies a1’s list of deposits. Also, a1’s invariant is now broken. Spec# solution: Allow the list of deposits to be modified only when the account object is exposed. Tutorial at FM2005

43. Updating Owned Objects class Account { [Owned] List<int> deposits; int balance; invariant balance == Math.Sum(deposits); void Deposit(int amount) { expose (this) { deposits.Add(amount); balance += amount; } } } Objects cannot be modified while they’re owned. But calling BeginExpose() on the owner releases ownership of the owned objects. Calling O.EndExpose() causes O to (re-)gain ownership of the objects pointed to by its [Owned] fields. Tutorial at FM2005

44. // Account a = new Account(); Account a = new Account; List<int> d = new List<int>; d.EndExpose(); a.deposits = d; a.EndExpose(); // a.Deposit(…); a.BeginExpose(); d.BeginExpose(); d.EndExpose(); a.EndExpose(); Object Lifecycle exposedexposableowned Tutorial at FM2005

45. // Account a = new Account(); Account a = new Account; List<int> d = new List<int>; d.EndExpose(); a.deposits = d; a.EndExpose(); // a.Deposit(…); a.BeginExpose(); d.BeginExpose(); d.EndExpose(); a.EndExpose(); Object Lifecycle a inv == false owner == null exposedexposableowned Tutorial at FM2005

46. // Account a = new Account(); Account a = new Account; List<int> d = new List<int>; d.EndExpose(); a.deposits = d; a.EndExpose(); // a.Deposit(…); a.BeginExpose(); d.BeginExpose(); d.EndExpose(); a.EndExpose(); Object Lifecycle d inv == false owner == null a inv == false owner == null exposedexposableowned Tutorial at FM2005

47. // Account a = new Account(); Account a = new Account; List<int> d = new List<int>; d.EndExpose(); a.deposits = d; a.EndExpose(); // a.Deposit(…); a.BeginExpose(); d.BeginExpose(); d.EndExpose(); a.EndExpose(); Object Lifecycle d inv == true owner == null a inv == false owner == null exposedexposableowned Tutorial at FM2005

48. // Account a = new Account(); Account a = new Account; List<int> d = new List<int>; d.EndExpose(); a.deposits = d; a.EndExpose(); // a.Deposit(…); a.BeginExpose(); d.BeginExpose(); d.EndExpose(); a.EndExpose(); Object Lifecycle d inv == true owner == null deposits a inv == false owner == null exposedexposableowned Tutorial at FM2005

49. // Account a = new Account(); Account a = new Account; List<int> d = new List<int>; d.EndExpose(); a.deposits = d; a.EndExpose(); // a.Deposit(…); a.BeginExpose(); d.BeginExpose(); d.EndExpose(); a.EndExpose(); Object Lifecycle d inv == true owner == a deposits a inv == true owner == null exposedexposableowned Tutorial at FM2005

50. // Account a = new Account(); Account a = new Account; List<int> d = new List<int>; d.EndExpose(); a.deposits = d; a.EndExpose(); // a.Deposit(…); a.BeginExpose(); d.BeginExpose(); d.EndExpose(); a.EndExpose(); Object Lifecycle d inv == true owner == null deposits a inv == false owner == null exposedexposableowned Tutorial at FM2005