Verification in the Model-Based Design Flow

1. Verification in the Model-Based Design Flow Bruce H. Krogh CMACS Review March 4, 2010 TexPoint fonts use in EMF. Read the TexPoint manual before you delete this box.: AAAAAAAAAAAAAAAA

2. Overview • VVIACS • Model reduction • Heterogeneous verification • Verification of numerical code • Current directions

3. Cover Vincent W. Crum Air Vehicles Directorate Air Force Research Laboratory | AFRL

4. FCS – Design flow (LM – VVIACS Project) Model-based Environments Formal Specification Techniques Advanced V&V-aware Designs Control Analysis Software Implementation Formal V&V Automated Test Process-Based Certification

5. Current Process flows & blocks modifiedfrom current process Other Subsystems Requirements Control Law Requirements Software Requirements Hardware Requirements Subsystem Unit and Component Testing Other Subsystem Design Stability and Control System Level Certification Control Law Design & Analysis Hardware & Software Integration Testing Software Unit and Component Testing System Level Testing VMS: System Requirements Software Design & Analysis Hardware Qual & Acceptance Testing Hardware Design & Analysis Test Tool Development Simulation

6. FCS – Verification via Testing

7. FCS – Cost of Testing Future Military Program Testing Hours Are Forecast to Triple

8. FCS – Possible applications of HSV

9. Significant Cost/Schedule Increase Projected Due to Complexity VVIACS - Impact Analysis Results • Single-Vehicle ECS Increases Development Costs ~ 50%, V&V Costs ~ 100%, and Critical Path Length ~ 50% • Multiple-Vehicle ECS Increases Development Costs ~ 100%, V&V Costs ~ 150%, and Critical Path Length ~ 125% • Software: Single-Vehicle 100% Increase and Multiple-Vehicle 200% Increase in V&V Costs • Test: Single-Vehicle 150% Increase and Multiple-Vehicle 250% Increase in V&V Costs

10. Far-Term (7-9 Yrs) Process • Formal Req/Spec • V&V Aware Subsystem Unit and Component Testing Other Subsystems Requirements Other Subsystem Design Stability and Control System Level Certification Algorithm Design, Analysis and Functional Test Algorithm Requirements Software Analysis, Design, Integration Integration & System Level Testing Vehicle: System Requirements Software Requirements Rapid Proto Type Hardware and Test Environment Hardware Qual & Acceptance Testing Hardware Requirements Hardware Design & Analysis Test Tool Development Simulation

11. Some Goals • Guarantee correct behavior of the complete system especially with respect to • timing constraints in the implementation • performance specifications associated with mission-level objectives • Develop a comprehensive approach integrating verification, validation and test procedures throughout the complete development cycle, from requirements capture to deployment. • Achieve confidence levels that exceed those achievable by current practice and current technologies for systems that incorporate emerging adaptive and intelligent control laws.

12. Model Reduction for Scalability of Hybrid System Verification • Use simplifiedmodels and/or set representations to perform the reach set computations Model Model Order Reduction  -decomposition Piecewise Linearization Set representation Full-dimensionalpolytopes Low-dimensional polytopesand their neighborhoods

13. Heterogeneous Verification • Motivation: • verifying properties of complete systems is beyond the reach of any one tool or modeling formalism • Objective: • reason about verification information collected from multiple sources to achieve system-level verification System Unstructured, semi-structured and structured information from various analyses on components /models. Verification report . . . . Existing docs Hybrid Analysis Discrete Analysis Simulation

14. Heterogeneous Verification Managing knowledge for model-based development embedded system ontology (base domain description) entities, relationships, rules ontology specialization domain experts heterogeneous information sources specialized ontology docs hybrid analysis static ontology + epistemic ontology model development & verification activities developers knowledge assimilation discrete analysis simulation information knowledge base (Protégé) targeted knowledge acquisition database + epistemic rule base requirements inference engine queries knowledge composition, deduction update verification manager inferences + knowledge gaps

15. Verification of numerical programs • Problem definition • Polyhedral domains • Control flow automata (CFA) • CFA reachability • Widening based on coefficient limiting • CFA reduction • Kahan summation example • Conclusions

16. Design and implementation of numerical programs design model model-based development source code implementation code generation target processor compiler platform implementation

17. Verification of numerical programs design model Need to verify how numerical code will execute on the target processor code generation target processor compiler

18. Verification of numerical programs design model disassembler control flow graph numerical program verifier target processor error model CFA generator code generation CFA PHAVer target processor compiler reachability results

19. Verification of numerical programs design model today’s presentation disassembler control flow graph target processor error model CFA generator code generation CFA PHAVer target processor compiler reachable sets error bounds

20. Scope of this work • instructions of the form • real constants and variables • linear arithmetic • floating point error bounds

21. Polyhedral domains • linear predicates • convex polyhedron: conjunction of linear predicates • polyhedron: disjunction of convex polyhedra • Parma Polyhedra Library (PPL): performs exact computations with non-convex polyhedra • PHAVer: performs reachability for LHA • exact and robust arithmetic with unlimited precision (PPL) • bit-constrained over-approximations for termination heuristics • on-the-fly over-approximation of piecewise affine dynamics • support for compositional and assume-guarantee reasoning.

22. 6 x 6 y 6 y 1 109 x 121 y 100 0 1 x Exact Arithmetic in PHAVer • Finite resources require over approximation • Semi-bounded exact arithmetic • exact computations that result in finite precision • * Managing the complexity by over-approximation generate time-elapse polyhedron * compute conservative over-approximation derivative initial set limit the number of bits of coefficients limit the number of constraints

23. Control flow automata (CFA) • same node/transition structure as the control flow graph • instructions replaced by action predicates on the transitions representing the operation error bounds

24. CFA example

25. CFA reachability • CFA state: (q,x) – q discrete state, x valuation of variables x • Reachable states: smallest fixed point of where • All sets are polyhedra • In general, the reachability iteration will not terminate

26. Widening for iterative computations • Accelerates convergence to a fixed point. P. Cousot and R. Cousot. Abstract interpretation: A unified lattice model for static analysis of programs by construction or approximation of fixpoints. In Proceedings of the Fourth Annual ACM Symposium on Principles of Programming Languages, pp. 238-252, New York, 1977. ACM Press.

27. Standard widening • defined for convex polyhedra • retains • constraints of P1 also satisfied by P2 • and • constraints of P2 with equivalent constraints in P1

28. Standard widening: example exact (nonterimintating) • std. widening • terminates at iteration 5 • large overapproximation

29. Widening based on coefficient limiting (NEW) • Preliminaries: • CP : set of linear predicates defining polyhedron P • assume integer coefficients with common divisor 1 • max_coeff(CP) : maximum coefficient in CP • coeff_limit(P,k) : polyhedron P such that • 1) P P • 2) max_coeff(CP)  k • NOTE: Such a P is computed by PHAVer

30. Widening based on coefficient limiting (NEW) • Proposition 1. is a widening operator. • follows from

31. Example 1: Application to Program 1 std. widening coefficient-limiting widening

32. Example 2: Non-convex polyhedra (w/o convex hull)

33. CFA Reduction • Objective: Given a set of variables W, reduce the number of transitions and variables in the CFA without affecting the reachable set for the variables in W.

34. Merging transitions • Transition condition for merging • applied only to the first transition in a pair of transitions

35. Eliminating irrelevant variables • transition merging increases the number of globally irrelevant variables • retains variables that influence error bounds on variables of interest

36. Precision vs. Efficiency using • Value of k introduces a tradeoff between - accuracy of polyhedral approximations and - complexity of the computations • Smaller k increases the over approximation but doesn’t necessarily make termination faster

37. Example: Kahan summation algorithm • adding N numbers: error = Ne • Kahan algorithm introduces intermediate variables to mitigate the effects of repeated summations: error = 2e +O(Ne2) • From Wikipedia: • function kahanSum(input, n) • var sum = input[1] • var c = 0.0 // A running compensation for lost low-order bits. • for i = 2 to n • y = input[i] - c // So far, so good: c is zero. • t = sum + y // Alas, sum is big, y small, so low-order digits of y are lost. • c = (t - sum) - y //(t - sum) recovers the high-order part of y; • // subtracting y recovers -(low part of y) • sum = t // Algebraically, c should always be zero. • // Beware eagerly optimising compilers! • next i // Next time around, the lost low part will • // be added to y in a fresh attempt. • return sum

38. Our implementation • e = 1.192092896e-7 (i386) • x0 = 1 • xi = 0.1249999403953552 • N = 8 reduced: 3 locations, 3 transitions, vars y,t eliminated

39. Current Directions • Integration through architecture • Assume-guarantee approach to controller-plant decomposition • Innovative uses of reachability