Technologies for finding errors in object-oriented software

Technologies for finding errorsin object-oriented software K. Rustan M. LeinoMicrosoft Research, Redmond, WA Lecture 3Summer school on Formal Models of Software5 Sep 2003, Tunis, Tunisia

Invariants: straw man Object invariants hold on public method boundaries and are shorthands for pre/post-conditions class T { // field declarations ... invariantJ ; T(...)requires Pmodifies vensures Q { … } method m(...)requires Rmodifies wensures T { … } … class T { // field declarations ... T(...)requires Pmodifies vensures Q∧ J { … } method m(...)requires R ∧ Jmodifies wensures T∧ J { … } …

Invariants, example class T { privateint x, y ;invariant0 ≦ x < y ; public T() { x = 0 ; y = 1 ; } publicmethod m()modifies x, y {assert y-x ≧ 0 ; x = x + 3 ; y = 4 * y ; } … class T { privateint x, y ; public T()ensures0 ≦ x < y { x = 0 ; y = 1 ; } publicmethod m()requires0 ≦ x < ymodifies x, y ensures0 ≦ x < y {assert y-x ≧ 0 ; x = x + 3 ; y = 4 * y ; } …

Invariants, problems class T { privateint x, y ;invariant 0 ≦ x < y ; public T() { x = 0 ; y = 1 ; } publicmethod m()modifies x, y {assert y-x ≧ 0 ; x = x + 3 ; y = 4 * y ; } … class T { privateint x, y ; public T()ensures 0 ≦ x < y { x = 0 ; y = 1 ; } publicmethod m()requires 0 ≦ x < y modifies x, y ensures 0 ≦ x < y {assert y-x ≧ 0 ; x = x + 3 ; y = 4 * y ; } … callers are expected to establish property about internal data structure

Invariants, problems class T { privateint x, y ;invariant 0 ≦ x < y ; public T() { x = 0 ; y = 1 ; } publicmethod m()modifies x, y {assert y-x ≧ 0 ; x = x + 3 ; y = 4 * y ; } … class T { privateint x, y ; public T()ensures 0 ≦ x < y { x = 0 ; y = 1 ; } publicmethod m()requires 0 ≦ x < ymodifies x, y ensures 0 ≦ x < y {assert y-x ≧ 0 ; x = x + 3 ; y = 4 * y ; } … possible solution (?): callers don’t need to be checked for this precondition—it holds automatically!

Invariants, problems class T { privateint x, y ;invariant 0 ≦ x < y ; public T() { x = 0 ; y = 1 ; } publicmethod m()modifies x, y {assert y-x ≧ 0 ; x = x + 3 ; y = 4 * y ; } … class T { privateint x, y ; public T()ensures 0 ≦ x < y { x = 0 ; y = 1 ; } publicmethod m() requires 0 ≦ x < y modifies x, y ensures 0 ≦ x < y {assert y-x ≧ 0 ; x = x + 3 ; y = 4 * y ; } … invariant does not hold here

Invariants, problems class T { privateint x, y ;invariant 0 ≦ x < y ; public T() { x = 0 ; y = 1 ; } public method m()modifies x, y {assert y-x ≧ 0 ; x = x + 3 ; y = 4 * y ; } … class T { privateint x, y ; public T()ensures 0 ≦ x < y { x = 0 ; y = 1 ; } publicmethod m() requires 0 ≦ x < y modifies x, y ensures 0 ≦ x < y {assert y-x ≧ 0 ; x = x + 3 ;p(...) ; y = 4 * y ; } … invariant does not hold here, so what if p calls m?!

The problem of specifying modifications class Kitchen {private Dishes d ; private bool hasGarbage ;private Stove s ;private Light l ; ... publicmethod SpiffUp()modifies hasGarbage, s.foodPieces, s.big.sufaceColor, … { d.Clean() ; hasGarbage = false ; s.Wipe() ; l.TurnOff() ; ... } class Stove {private Burner big ;private Burner small ;privateint foodPieces ;private Knob[] knobs ; ... publicmethod Wipe()modifies foodPieces, big.surfaceColor, … { big.Polish() ; small.Polish() ; foodPieces = 0 ; ... } these lists are long, and they mention private state

The problem of specifying modifications class Kitchen {private Dishes d ; private bool hasGarbage ;private Stove s ;private Light l ; ... publicmethod SpiffUp()modifieshasGarbage, s.foodPieces, s.big.sufaceColor, … { d.Clean() ; hasGarbage = false ; s.Wipe() ; l.TurnOff() ; ... } class Stove {private Burner big ;private Burner small ;privateint foodPieces ;private Knob[] knobs ; ... publicmethod Wipe()modifiesfoodPieces, big.surfaceColor, … { big.Polish() ; small.Polish() ; foodPieces = 0 ; ... } possible solution (?):don’t need to declare modifications of private state—it can’t be observed anyhow

The problem of specifying modifications class Kitchen {private Dishes d ; private bool hasGarbage ;private Stove s ;private Light l ; ... publicmethod SpiffUp()modifies { d.Clean() ; hasGarbage = false ; s.Wipe() ; assert ￢hasGarbage ; l.TurnOff() ; ... } class Stove {private Burner big ;private Burner small ;privateint foodPieces ;private Knob[] knobs ; ... publicmethod Wipe()modifies { big.Polish() ; small.Polish() ; foodPieces = 0 ; ... } SpiffUp treats Wipe as if Wipe modified nothing, so what if Wipe calls a method in Kitchen that sets hasGarbage to true?!

Soundness of verification • Soundness = verification finds all errors • Soundness follows from: • pre- and postconditions are the same for caller and callee • Note: In addition to soundness, we want something usable

Methodology • object invariant declaration • class T { int x, y ;invariant x < y ; • special variable st: {Invalid, Valid} • Idea: program invariant (∀o ･ o.st = Invalid ∨ Inv(o)) • st is changed by commands pack andunpack for any o: T, we writeInv(o)≡ o.x < o.y holds at everyprogram point!

pack and unpack • pack o ≡assert o.st = Invalid ;assert Inv(o) ; o.st := Valid • unpack o ≡assert o.st = Valid ; o.st := Invalid

Example receiver parameter(“this”, “self”, “current”) class T {int x, y ;invariant 0 ≦ x < y ; method init(t)requires t.st = Invalidmodifies t.st, t.x, t.yensures t.st = Valid { t.x := 0 ; t.y := 1 ;pack t } method m(t)requires t.st = Validmodifies t.x, t.y {unpack t ; t.x := t.x + 3 ; t.y := 4 * t.y ;pack t }

Program invariant (∀o ･ o.st = Invalid ∨ Inv(o)) • x := new(T)≡ ... ; assume x.st = Invalid • pack o≡ ... ; assert Inv(o) ; o.st := Valid • unpack o≡ ... ; o.st := Invalid • o.f := E≡ assert o.st = Invalid ; ... • Inv(o) can mention only the fields of o

Methodology, summary • invariant ... • st: {Invalid, Valid} • pack, unpack • modifications of o.f require o.st=Invalid • Inv(o) can mention only the fields of o • (∀o ･ o.st = Invalid ∨ Inv(o))

Methodology, extended • component declarations • class Kitchen {component Stove s ; • st: {Invalid, Valid, Committed} • Idea: program invariant(∀o ･ o.st=Invalid ∨ (Inv(o) ∧ (∀p∈Comp(o) ･ p.st=Committed))) • pack o and unpack o change st for o and o's components for any k: Kitchen, we havek.s ∈ Comp(k)

pack and unpack, extended • pack o ≡assert o.st = Invalid ∧ Inv(o) ;assert (∀p ∈ Comp(o) ･ p.st=Valid) ; o.st := Valid ;change st suchthat (∀p ･ (p ∈ Comp(o) ∧ p.st=Committed) ∨ (p ∈ Comp(o) ∧ p.st=p.st0)) • unpack o ≡assert o.st = Valid ;change st suchthat (∀p ･ (p ∈ Comp(o) ∧ p.st=Valid) ∨ (p ∈ Comp(o) ∧ p.st=p.st0)) ; o.st := Invalid ; ／／

Example class Stove {method Wipe(s)requires s.st=Valid … class Kitchen { method SpiffUp(k)requires k.st=Valid … {unpack k ; k.d.Clean() ; k.s.Wipe() ;pack k } Committed Committed Dishes Committed Committed Stove s d Committed k Kitchen Valid

Example class Stove {method Wipe(s)requires s.st=Valid … class Kitchen { method SpiffUp(k)requires k.st=Valid … {unpack k ; k.d.Clean() ; k.s.Wipe() ;pack k } Committed Committed Dishes Committed Committed Valid Stove s d Committed Valid k Kitchen Valid Invalid

Program invariant (∀o ･ o.st=Invalid ∨ (Inv(o) ∧ (∀p∈Comp(o) ･ p.st=Committed))) • x := new(T)≡ ... ; assume x.st=Invalid • pack o, unpack o • o.f := E≡ assert o.st=Invalid ; ... • Inv(o) can mention only the fields of o andof o.p for any component field p

Extended methodology, summary • invariant ... • component ... • st: {Invalid, Valid, Committed} • pack, unpack • modifications of o.f require o.st=Invalid • Inv(o) can mention only the fields of o and of o.p for any component field p • (∀o ･ o.st=Invalid ∨ (Inv(o) ∧ (∀p∈Comp(o) ･ p.st=Committed)))

Verification system • We let st be used in method specifications (requires, modifies, ensures) • We must address the Problem of Specifying Modifications

An alternative heap model class T { f: U; g: V; ... } • In my previous lecture: • many one-dimensional map variables (one per field) • x := o.f = x := f[o] = x := select(f, o) • o.f := E = f[o] := E = f := store(f, o, E) • Now: • one two-dimensional map variable • x := o.f = x := Heap[o, f] = x := select(Heap, o, f) • o.f := E = Heap[o, f] := E = Heap := store(Heap, o, f, E)

Meaning of modifies • modifies w =modifies Heapensures (∀o,f ･ Heap[o,f] = Heap0[o,f] ∨ (o,f) ∈ w0 ) viewed as set of object/field-name pairs

Meaning of modifies • modifies w =modifies Heapensures (∀o,f ･ Heap[o,f] = Heap0[o,f] ∨ (o,f) ∈ w0 ∨ ￢Heap0[o,alloc] )

Meaning of modifies • modifies w =modifies Heapensures (∀o,f ･ Heap[o,f] = Heap0[o,f] ∨ (o,f) ∈ w0 ∨ ￢Heap0[o,alloc] ∨ Heap0[o,st]=Committed )

Example class Stove {method Wipe(s)requires s.st=Validmodifies s.foodPieces class Kitchen { method SpiffUp(k)requires k.st=Validmodifies k.hasGarbage {unpack k ; k.d.Clean() ; k.hasGarbage := false ; k.s.Wipe() ;assert￢k.hasGarbage ;pack k }

Another example method m(p)requires p.st=Committed{var y, z in y := p.x ; z := sqrt(49) ;assert y = p.xend}

Yet another example method m(p)requires p.st=Committed{ p.x := p.x + 1} Is this field update allowed?

Soundness • Pre- and postconditions are the same for callers and callees, so verification system is sound!

Methodology conclusions • Invariants different from pre/post-conditions • Resulting program invariants hold at every program point • Uses pack/unpack commands, but good defaults can be constructed for these • No linear-type system • Components are not unique references—objects (pointers) can freely be copied • Fields can freely be read • No additional features of abstraction needed to support the specification of modifications • Sound ‥ For more details, see Barnett, DeLine, Fahndrich, Leino, Schulte, 2003 (reference [65] in my lecture-notes bibliography).

Research challenges • experience, understanding of limits • extensions to support more good programs

Summary of lectures • Automatic high-precision programming tools can be built for object-oriented languages • need more research to explore other design trade-offs • Semantics of programming languages (including object-oriented languages) can be defined from the semantics of simple commands • Object-oriented programming requires methodology • needs further research See lecture notes for selected bibliographic references. http://research.microsoft.com/～leino

Technologies for finding errors in object-oriented software