Regression Verification: Proving the equivalence of similar programs

1. Regression Verification: Proving the equivalence of similar programs Benny Godlin Ofer Strichman Technion, Haifa, Israel (This presentation is a subset of the invited cav’09 talk: ie.technion.ac.il/~ofers/presentations/rv1.ppt)

2. Functional Verification • The main pillar of the grand challenge [H’03]. • Suppose we ignore completeness. • Still, there are two major problems: • Specification • Complexity

3. A more modest challenge: Regression Verification • Develop a method for formally verifying the equivalenceoftwosimilar programs. • Pros: • Default specification = earlier version. • Computationally easier than functional verification. • Ideally, the complexity should depend on the semanticdifference between the programs, and not on their size. • Cons: • Defines a weaker notion of correctness.

4. Our notion of equivalence Partial equivalence • Executions of P1 and P2 on equal inputs • …which terminate, • result in equal outputs. • Undecidable

5. Partial equivalence • Consider the call graphs: • … where A,Bhave: • same prototype • no loops • Prove partial equivalence of A, B • How shall we handle the recursion ? A B Side 1 Side 2

6. //in[A] A( . . . ) { . . . //in[call A] call A(. . .); //out[call A] . . . } //out[A] Proving partial equivalence //in[B] B( . . . ) { . . . // in[call B] call B(. . .); //out[call B] . . . } //out[B] A B

7. Q: How can a verification condition for the premise look like? A: Replace the recursive calls with calls to functions that over-approximateA, B,and are partially equivalent by construction Natural candidates: Uninterpreted Functions Rule 1: Proving partial equivalence

8. Proving partial equivalence • Let AUF , BUFbe A,B, after replacing the recursive call with a call to (the same) uninterpreted function. • We can now rewrite the rule: The premise is decidable

9. a, b) y) x, z; g; Using (PART-EQ-1): example unsigned gcd1UF (unsigned a, unsigned b) { unsigned g; if (b == 0) g = a; else { a = a % b; g = gcd1(b, a); } return g; } unsigned gcd2UF (unsigned x, unsigned y) { unsigned z; z = x; if (y > 0) z = gcd2(y, z % y); } return z; } ? = U U Tgcd1Tgcd2 a,b x,y g z

10. Rule 1: example Equal inputs Equal outputs

11. Partial equivalence: Generalization • Assume: • no loops; • 1-1 mapping mapbetween the recursive functions of both sides • Mapped functions have the same prototype • Define: • For a function f, UF(f) is an uninterpreted function such that • f and UF(f) have the same prototype • (f,g) 2map , UF(f) = UF(g).

12. Partial equivalence: Generalization • Definition: is called in A]

13. f f ’ UF UF = g’ g UF UF = Partial equivalence: Example (1 / 3) {(g,g’),(f,f’)} 2 map g’ g f f ’ Side 2 Side 1 Need to prove:

14. UF UF Partial equivalence: Example (2 / 3) • An improvement: • Find a map that intersects all cycles, e.g., (g,g’) • Only when calling functions in this map replace with uninterpreted functions g’ g f f ’ Side 2 Side 1

15. Partial equivalence: Example (3 / 3) Connected SCCs… • Prove bottom-up • Abstract partially-equivalent functions • Inline g g’ f f ’ h h’ UF UF Side 2 Side 1

16. RVT: Decomposition algorithm Legend: Equivalent pair Equivalence undecided yet Could not prove equivalent Syntactically equivalent pair check Unpaired function B: A: check f1() f1’() f2’() f2() U U f5’() f5() f7’() f3() f4() f3’() f4’() f6() U U U U

17. RVT: Decomposition algorithm (with SCCs) Legend: Equivalent pair Equivalence undecided yet Could not prove equivalent Syntactically equivalent pair Equivalent if MSCC B: A: check f1() f1’() f2’() f5’() f2() f5() U U U U U U f6’() f3() f4() f3’() f4’() f6() U U U U

18. The Regression Verification Tool (RVT) • Given two C programs: • loops recursive functions. • Map functions, globals, etc. • After that: • Decompose to the granularity of pairs of functions • Use a C verification engine (CBMC) to discharge

19. The Regression Verification Tool (RVT) • CBMC: a C bounded model checker by Daniel Kroening • Our use: • No loops or recursion to unroll... • Use “assume(…)” construct to enforce equal inputs. • Use assert() to demand equal outputs. • Uninterpreted functions are implemented as C functions: • Return consistent nondeterminisitic values.

20. The Regression Verification Tool (RVT) • The premise of (PART-EQ) requires comparing arguments. • What if these arguments are pointers ? • What our system does: • Dynamic structures: creates an unrolled nondeterministic structure • Arrays: attempts to find references to cells in the array.

21. . . . P1: exp1 exp2 . . . P2: exp’1 exp’2 = = = = RVT: User-defined equivalence specification • The user can define pairs of ‘checkpoints’: side 1: <label1, cond1, exp1>side 2:<label2, cond2, exp2> • In each side: • update an array with the value of expeach time it reacheslabelandconditionholds. • Assert that when executed on the same input…, • … these arrays are equivalent.

22. RVT Version A Version B • result • counterexample RVT • rename identical globals • enforce equality of inputs. • assert equality of outputs • add checkpoints • Supports: • Decomposition • Abstraction • some static analysis • … C program feedback CBMC

24. RVT: Experiments We tested the Regression Verification Tool (RVT) with: • Automaticallygenerated sizable programs with complex recursive structures and loops. • up-to thousands of lines of code • Limited-size industrial programs: • Parts of TCAS - Traffic Alert and Collision Avoidance System. • Core of MicroC/OS- real-time kernel for embedded systems. • Matlab examples: parts of engine-fuel-injection simulation.

25. Testing RVT on programs: Conclusions • For equivalent programs, partial-equivalence checks were very fast: • proving equivalence in minutes. • For non-equivalent programs: • RVT attempts to prove partial-equivalence but fails • then RVT tries to provek-equivalence

26. Summary • Regression verification is an important problem • A solution to this problem has a better chance to succeed in the industry than functional verification • A grand challenge by its own right… • Lots of future research...

27. More Challenges • Q1: How can we generalize counterexamples ? • Q2: What is the ideal gap between two versions of the same program, that makes Regression Verification most effective ? • Q3: How can functional verification and equivalence verification benefit from each other ?

28. The end… • Thank you