Verification of Evolving Software

1. Verification of Evolving Software Natasha Sharygina Joint work with Sagar Chaki and Nishant Sinha Carnegie Mellon University

2. Motivation • Component-based Software • Software modules shipped by separate developers • Undergo several updates/bug-fixes during their lifecycle • Component assembly verification • Necessary on update of any component • High verification costs of global properties

3. P Contribution Assembly A ? Component C Component C’

4. Contribution • Automated substitutability check • Preserving all old behaviors • Allowing new behaviors • Focus on components that have been modified • Allow multiple upgrades simultaneously

5. Substitutability Check • Given old and new components Ci, C’i and assembly A • For each i check if C’i is substitutable for Ci in A • Two phases: • Containment check • Ensures that behaviors of the old component are contained in its substitutable counterpart • Compatibility check • Safety with respect to other components in assembly: all global specifications are satisfied • Approach: • Obtain a finite behavioral model of all components by abstraction: Labeled Kripke Structures(LKS) • Use model checkers to solve both phases automatically

6. Abstraction into LKS p • Labeled Kripke Structures • <Q,,T,P,L> • State-event traces • <p,,q,,…..> • Composition semantics • Synchronize on shared actions • Two types of abstraction: • Predicate abstraction (over-approximations: existential transitions) • Modified predicate abstraction (under-approximations: universal transitions)   q  !q !p

7. Abstraction into LKS • Program Statements ,States • Control flow ,Transitions • Transitions depict observable behavior • Automated

8. L2 lock = 0 Predicate Abstraction into LKS L1 void OSSemPend(…) { L1: lock = 1; if (x < y) { L2: lock = 0; … } if (x >= y) { … L3: lock = 0; … } else { … } } lock=1 if (x < y) x < y   lock = 0 if (x >= y) x < y   L3

9. unlock unlock lock ERROR lock Verification Specification Model L1 lock if (P) L2 unlock if (!P) L3 unlock

10. unlock unlock lock ERROR lock Verification Specification Model L1 lock if (*) L2 unlock if (*) L3 unlock

11. Abstraction Model Yes System OK Improved Abstraction Guidance No Abstraction Refinement Yes Spurious Counterexample ComFoRT: Component Formal Verification Framework Verification Components Assembly Counter Example Guided Abstraction Refinement No Counterexample Counterexample Valid?

12. Containment Step 1. Construct LKSs M’ and M such that (C1) C µ M and (C2) M’ µ C’ Step 2. Verify if M µ M’ • If yes then from(C1)and (C2) C µC’ and we terminate • Else CE is produced Step 3. Validate CE 1. Check if CE C. If yes proceed to next step. Otherwise refine M and repeat from Step 2. 2. Check if CE  C’. If yes than CE C \ C’, Feedback  CE and stop. Else refine M’ (by adding the valid CE) and repeat from Step 2.

13. Containment (contd.) • In contrast to Refinement-based approaches • Containment ensures that all behaviors of M are present in M’ • However, containment allows new behaviors in M’ which could be possibly erroneous • Containment check is not sufficient since new components • may not be safe with respect to other components • must satisfy the global behavioral specifications

14. a a Yes/No b b ±Counterexample/ Yes Unknown Regular Language Learning Regular languages: L* • Forms the basis of the compatibility check • L* proposed by D. Angluin • Polynomial in the number of states and length of counterexample IsMember( trace  ) L* learner Minimally adequate Teacher IsCandidate( DFA D ) Model checker Minimum DFA

15. Compatibility check • Recursive Assume-guarantee to verify assembly properties R1: M1 || A ² P R2: M2 || … || Mn ² A M1 || … || Mn² P • Generate a (smaller) environment assumption A • A:most general environment so that P holds • Constructed iteratively using L* and R1, R2 • Recursion for the R2 rule

16. Compatibility check -CE for Ai R1: M1 || Ai² P L* Assumption Generation Ai true true R2: M2|| … || Mn²Ai true CE Analysis True CE CE False CE +CE for Ai

17. Compatibility check (contd.) • Generate a most general assumption for each M’ • M1 = M’ • M2 = Mn M (all other component LKSs) • Membership queries: • M’ || CEµ P • Candidate queries: • M’ || A µ P • M2µ A

18. Compatibility check (contd.) • CE analysis: M’ || CE µ P • Yes) False CE • No)True CE • Compatibility check infers • Either M’ is substitutable • Or counterexample CE • CE may be spurious wrt. C’ due to the predicate abstraction • CE is present in component LKS M’ • Must refine M’ • Repeat substitutability check

19. Compatibility check (contd.) • Dynamic compatibility check • Restarts learning using information from a previously learned candidate • Account for incremental changes between successive assumptions The dynamic nature is critical for 1) checking evolving software; 2) during abstraction refinement where a single component is updated at a time.

20. Feedback to Developers Feedback - collection of CEs  Cidentified during containment check Feedback Delivery: Given a trace   Feedback, and Rep() – representation of  in terms of program control locations, predicate valuations and actions, the levels of feedback are 1) Rep(Prefix()) – to denote the divergence point of  2) Rep(Suffix()) – to show what exact behavior (exact code) of C is missing in C’ 3) Rep’(Pref()) – to denote location in Ci where changes are to be made

21. Experiments • Prototype implementation in ComFoRT framework • ABB IPC Module • Deployed by a world leader in robotics engineering systems • 15000+ LOC • modified WriteMessageToQueue component • checked for substitutability inside the assembly of four components (read, write, queue, critical section) • Over 30 billion states after predicate abstraction • Discovered synchronization bug • Process can incorrectly block while writing to a queue

22. Related Work • Learning Assumptions: Cobleigh. et. al. • Do not consider state labeling of abstract models • Do not incorporate a CEGAR framework for AG • Compatibility of Component Upgrades: Ernst et. al. • Do not consider temporal sequence of actions in generating invariants • Interface Automata: Henzinger et. al. • Do not have CEGAR, AG

23. ComFoRT framework: www.cs.cmu.edu/~natalie/

24. Questions?