Model-Based Design & Analysis

1. Model-Based Design & Analysis Dr. Steven P. Miller Advanced Computing Systems Rockwell Collins 400 Collins Road NE, MS 108-206 Cedar Rapids, Iowa 52498 spmiller@rockwellcollins.com

2. What Problem are We Solving? • Safety-Critical Software Is Too Expensive • Safety-Critical Software Is Often Wrong • DO-178B Certification Is Too Expensive Cut Development Costs/Cycle Time in Half Find 10x More Errors than Current Methods Already Applying This to DO-178B Developments

3. Model-Based Development Routinely Dismissed Then and Now Five Years Ago Today • Widespread Acceptance • 787, FCS 5000, ARJ, MUE, FMS … • Formal Methods Viewed as Impractical & Too Expensive • “This is Buck Rogers!” • actual customer quote • Created Models by Hand Using Research Notations • Automatically Translate Models from Leading Commercial Tools • Verifying Representative Examples • Finding Real Errors in Real Systems - in Seconds - in Weeks • Tools were Research Prototypes • Tools being Matured for Enterprise Use and Support

4. Outline of Presentation Introduction Our Approach Developing the Technology Making Technology Transfer Happen Recent Successes What’s Next?

5. Who Are We? Communications Navigation Automated Flight Control Displays / Surveillance Aviation Services In-Flight Entertainment Integrated Aviation Electronics Information Management Systems A World Leader In Aviation Electronics And Airborne/ Mobile Communications Systems For Commercial And Military Applications

6. Rockwell Collins Headquartered in Cedar Rapids, Iowa 16,000 Employees Worldwide

7. RCI Advanced Technology Center • The Advanced Technology Center (ATC) identifies, acquires, develops and transitions value-driven technologies to support the continued growth of Rockwell Collins. • The Automated Analysis group applies mathematical tools and reasoning to the problem of producing high assurance systems. Government Systems Commercial Systems Advanced Technology Center

8. Automated Analysis Section 1992 AAMP5 Microcode Verification (PVS) NASA LaRC Funded NSA Funded AAMP-FV Microcode Verification (PVS) 1994 AFRL Funded AAMP5 Partitioning (PVS) Tech Transfer 1996 JEM Java Virtual Machine (PVS) FGS Mode Confusion Study (PVS) 1998 FCP 2002 Microcode (ACL2) 2000 AvSSP AAMP7 Separation Kernel (ACL2) NASA FGS Safety Analysis (RSML-e) FGS Mode Confusion (RSML-e) NSA AFRL 2002 vFaat (ACL2, PVS) 2004 FCS 5000 FGS Verification (NuSMV) SHADE (ACL2) GreenHills Integrity RTOS (ACL2) Displays Verification (NuSMV) 2006

9. Methods and Tools for Flight Critical Systems Project • Five Year Project Started in 2001 • Part of NASA’s Aviation Safety Program (Contract NCC-01001) • Funded by the NASA Langley Research Center and Rockwell Collins • Practical Application of Formal Methods To Modern Avionics Systems

10. Outline of Presentation Introduction Our Approach Developing the Technology Making Technology Transfer Happen Recent Successes What’s Next?

11. Convergence of Two Trends Model-Based Development Automated Analysis A Revolutionary Change in How We Design and Build Systems

12. Model-Based Development Examples

13. Does Model-Based Development Scale? Airbus A380 Systems Developed Using MBD • Flight Control • Auto Pilot • Fight Warning • Cockpit Display • Fuel Management • Landing Gear • Braking • Steering • Anti-Icing • Electrical Load Management Length 239 ft 6 in Wingspan 261 ft 10 in Maximum Takeoff Weight 1,235,000 lbs Passengers Up to 840 Range 9,383 miles

14. How Do We Reduce Costsand Improve Quality? Reduces Cost of Testing Clear Specifications Improves Communication Enables More Testing Eliminates Manual Coding Easy Validation Makes Model Primary Artifact Finds Errors Early Requirements Elicitation Reuse 15% 10% Autotest Modeling 5% 10% Autocode Simulation Automated Analysis 10% - 20% Cheaper Than Manual Analysis Finds the Really Hard Errors

15. Outline of Presentation Introduction Our Approach Developing the Technology Making Technology Transfer Happen Recent Successes What’s Next?

16. Flight Guidance System Mode Logic Requirements Elicitation Reuse Modeling Autotest Simulation Autocode Automated Analysis

17. Captured Requirements as Shalls

18. Modeling Requirements Elicitation Reuse Modeling Autotest Simulation Autocode Automated Analysis

19. Modeling Notations Tabular (RSML-e, SCR) Textual (Lustre, PVS, SAL, …) node Thrust_Required( FG_Mode : FG_Mode_Type ; Airborne : bool ; In_Flare : bool ; Emergency_Descent : bool; Windshear_Warning : bool ; In_Eng_Accel_Zone : bool ; On_Ground : bool) returns (IsTrue : bool) ; let IsTrue = (FG_Thrust_Mode(FG_Mode) and Airborne) or (Airborne and Emergency_Descent) or Windshear_Warning or ((FG_Mode = ThrottleRetard) and In_Flare) or (In_Eng_Accel_Zone and On_Ground) ; tel ; Graphical (Simulink, SCADE)

20. Simulation Requirements Elicitation Reuse Modeling Autotest Simulation Autocode Automated Analysis

21. Simulation

22. Automated Analysis Reuse Requirements Elicitation Modeling Autotest Simulation Autocode Automated Analysis Model Checkers Theorem Provers

23. What Are Model Checkers? • Breakthrough Technology of the 1990’s • Widely Used in Hardware Verification (Intel, Motorola, IBM, …) • Several Different Types of Model Checkers • Explicit, Symbolic, Bounded, Infinite Bounded, … • Exhaustive Search of the Global State Space • Consider All Combinations of Inputs and States • Equivalent to Exhaustive Testing of the Model • Produces a Counter Example if a Property is Not True • Easy to Use • “Push Button” Formal Methods • Very Little Human Effort Unless You’re at the Tool’s Limits • Limitations • State Space Explosion (10100 – 10300 States)

24. Advantage of Model Checking Testing Checks Only the Values We Select Even Small Systems Have Trillions (of Trillions) of Possible Tests!

25. Advantage of Model Checking Model Checker Tries Every Possible Input and State!

26. Model Checking Process Does the systemhave property X? SMV Automatic Translation Counter Example Properties SMV Properties SMV Spec. Model Automatic Translation Automated Check Yes! Engineer

27. Translated Shalls into SMV Properties

28. Validate Requirements through Model Checking • Proved Over 280 Properties in Less Than an Hour • Found Several Errors • Some Were Errors in the Model • Most Were Incorrect Shalls • Revised the Shalls to Improve the Requirements

29. What are Theorem Provers? • Available Since Late 1980’s • Widely Used on Security and Safety-Critical Systems • Use Rules of Inference to Prove New Properties • Also Consider All Combinations of Inputs and States • Also Equivalent to Testing with an Infinite Set of Test Cases • Generate An Unprovable Proof Obligation if a Property is False • Not Limited by State Space • Applicable to Almost Any Formal Specification • Limitations • Require Experience - About Six Months to Become Proficient • Constructing Proofs is Labor Intensive

30. Theorem Proving Using PVS PVS Spec. Automatic Translation Why not? Does the systemhave property X? Guru PVS Automatic Translation Properties PVS Properties Model Automated Proof Engineer

31. Searching for Potential Sources of Mode Confusion Used Theorem Proving to Search For • Entry and Exit of Off Normal Modes • Ignored Operator Commands • Certain Forms of Lack of Feedback • Hidden Modes • Unintended Side Effects • Lack of Feedback from Multiple Operators Discrepancy between the perceived and actual state of an automated system.

32. Validate Requirements Using Theorem Proving • Proved Several Hundred Properties Using PVS • More Time Consuming that Model-Checking • Use When Model-Checking Won’t Work • Models that are Numerically Intensive • Automated Safety (Fault Tree) Analysis

33. Outline of Presentation Introduction Our Approach Developing the Technology Making Technology Transfer Happen Recent Successes What’s Next?

34. Original Tool Chain RSML-e to NuSMV Translator NuSMV Model Checker RSML-e PVS Theorem Prover RSML-e to PVS Translator Rockwell Collins/U of Minnesota SRI International

35. Conversion to SCADE NuSMV SCADE Lustre PVS Safe State Machines Design Verifier Rockwell Collins Esterel Technologies SRI International

36. Extension to MATLAB Simulink Simulink Gateway Simulink StateFlow NuSMV SCADE Lustre PVS Safe State Machines Design Verifier Rockwell Collins Esterel Technologies SRI International MathWorks

37. Lustre Translator Framework … • Small Source-To-Source Transformations • Deal with One Language Aspect at a Time • Product Family of Small Translators Supports Reuse Lustre Code Target Code Translator

38. Current Tool Chain Simulink Gateway Simulink Reactis StateFlow ICS Symbolic Model Checker SAL Bounded Model Checker Infinite Model Checker NuSMV SCADE Lustre PVS Safe State Machines Design Verifier Rockwell Collins Esterel Technologies SRI International MathWorks Reactive Systems

39. Translator Optimizations for NuSMV

40. Original Tool Chain RSML-e to NuSMV Translator NuSMV Model Checker RSML-e PVS Theorem Prover RSML-e to PVS Translator Rockwell Collins/U of Minnesota SRI International

41. Current Tool Chain Simulink Gateway Simulink Reactis StateFlow ICS Symbolic Model Checker SAL Bounded Model Checker Infinite Model Checker NuSMV SCADE Lustre PVS Safe State Machines Design Verifier Rockwell Collins Esterel Technologies SRI International MathWorks Reactive Systems

42. Outline of Presentation Introduction Our Approach Developing the Technology Making Technology Transfer Happen Recent Successes What’s Next?

43. Example 1FCS 5000 Mode Logic Mode Controller A 6.8 x 1021 Reachable States Mode Controller B Requirement Mode A1 => Mode B1 Counterexample Found in Less than Two Minutes! Found 24 Errors to Date

44. Example 2 – ADGS-2100 Adaptive Display & Guidance System 883 Subsystems 9,772 Simulink Blocks 2.9 x 1052 Reachable States Requirement Drive the Maximum Number of Display Units Given the Available Graphics Processors Counterexample Found in 5 Seconds! Checking 373 Properties Found Over 60 Errors

45. Outline of Presentation Introduction Our Approach Developing the Technology Making Technology Transfer Happen Recent Successes What’s Next?

46. Extending the Verification Domain Theorem Provers Infinite Bounded Model Checkers Infinite State Models using k - Induction Arbitrary Models Labor Intensive Implicit State Model Checkers 200 < 10 Reachable States • Numerically Intensive Systems • Infinite Bounded Model Checkers • Decision Procedures for Integers and Real Numbers • Non-linear Arithmetic • Automatic Extraction of Conservative Abstractions • Applications • Spacing & Trajectory • Required Navigation Performance (RNP) • Collision Avoidance • Advanced Flight Control

47. Requirements Based Test Case Generation Create Requirements Based Tests Test Case Generator Create Model Code Generator Create Additional Structural Tests Test Case Generator Conformance Testing • Autogenerate Test Cases From Model • Commercial Tools Available • (T-VEC, REACTIS) • Show Code Conforms to the Model • Goal is Structural Coverage (MC/DC) Requirements Properties Requirements Based Testing • State Requirements as Properties • Use Bounded Model Checker to Generate Test Cases • Goal is to Cover the Requirement Model Code

48. System Architectural Modeling & Analysis

49. Model-Based Safety Analysis Green Pump Blue Pump Isolation Valve Isolation Valve Power A System A Selector Valve Pedal 1 Shut Normal A Accumulator N Plant System L O Valve Feed back T R M E Accumulator A System B Pedal 2 R Pump L N Power B AntiSkid A Meter Valve Mechanical Command T Pedal E Braking + Meter Fault Tolerant Meter AntiSkid Valve Valve Braking System Command Control Unit Plant Model ( BSCU ) • Add Fault Model for Physical System • Model the Digital Controller Architecture and the Physical System and Digital Controller Architecture • Integrates System and Safety Engineering About a Common Model • Automation Enables “What-If” Consideration of System Designs

50. Model-Based Safety Analysis • Common Model for Both System and Safety Engineering • Safety Analysis Based on a Formal System Model • Facilitates Consistency & Completeness in Safety Analysis • Reduced Manual Effort in Error-prone Areas • Automated Support for Safety Analysis • Explore Various Failure Scenarios • Focus on Review on Assumptions in the Models • Is the System Model Correct? • Is the Fault Model Complete? • Assume the (Automated) Analysis is Trustworthy • Wide Applicability (Aircraft, UAVs, Shuttle, Space, …)