Formal Methods in Safety-Critical Systems

1. Formal Methods in Safety-Critical Systems Dr. Steven P. Miller Advanced Computing Systems Rockwell Collins 400 Collins Road NE, MS 108-206 Cedar Rapids, Iowa 52498 spmiller@rockwellcollins.com

2. What Problem are We Solving? • Safety-Critical Software Is Too Expensive • Safety-Critical Software Is Often Wrong • DO-178B Certification Is Too Expensive Cut Development Costs/Cycle Time in Half Find 10x More Errors than Current Methods Already Applying This to DO-178B Developments

3. Are We Making Progress? • Model-Based Development Spreading Rapidly • Prove Properties of Simulink & SCADE Models • Finding Errors Early in the Lifecycle Several projects at Rockwell Collins In Seconds on Models with Over 10**100 States On Real Products!

4. Outline of Presentation Introduction Overview of Our Approach An Example – FGS Mode Logic Some Recent Accomplishments The Underlying Technology What’s Next? Summary

5. Who Are We? Communications Navigation Automated Flight Control Displays / Surveillance Aviation Services In-Flight Entertainment Integrated Aviation Electronics Information Management Systems A World Leader In Aviation Electronics And Airborne/ Mobile Communications Systems For Commercial And Military Applications

6. Rockwell Collins Headquartered in Cedar Rapids, Iowa 14,500 Employees Worldwide

7. RCI Advanced Technology Center • The Advanced Technology Center (ATC) identifies, acquires, develops and transitions value-driven technologies to support the continued growth of Rockwell Collins. • The Automated Analysis group applies mathematical tools and reasoning to the problem of producing high assurance systems. Government Systems Commercial Systems Advanced Technology Center

8. Automated Analysis Group • Participants in the MCC Formal Methods Transition Study 1991 • Formal Specification of the μReal Time Executive in RAISE 1992 • Formal Specification of the GE1 Graphics Processor 1996 • Formal Verification of Microprocessors 1993 - 2005 • AAMP5 Microcode Using PVS 1994 • AAMP-FV Microcode Using PVS 1995 • JEM Java Virtual Machine Microprocessor Using PVS 1998 • FCP2002 Microcode Using ACL2 1999 • FCP 2002-2000 Microcode Equivalence Using ACL2 2001 • AAMP7 Security Separation Kernel Using ACL2 2003 • Formal Validation of Embedded System Requirements 1995 - 2005 • FGS Mode Logic using SPC’s CoRE Method 1995 • FGS Mode Logic using NRL’s SCR* Tools 1996 • FGS Mode Logic Using PVS 1997 • FGS Mode Logic Using Matrix-X and T-VEC 1998 • FGS Mode Logic Using RMSL-e, PVS, and NuSMV 2002 • FGS/FMS/AT Logic Using SCADE and Simulink 2004

9. Methods and Tools for Flight Critical Systems Project • Five Year Project Started in 2001 • Part of NASA’s Aviation Safety Program (Contract NCC-01001) • Funded by the NASA Langley Research Center and Rockwell Collins • Practical Application of Formal Methods To Modern Avionics Systems

10. Outline of Presentation Introduction Overview of Our Approach An Example – FGS Mode Logic Some Recent Accomplishments The Underlying Technology What’s Next? Summary

11. Convergence of Two Trends Model-Based Development Automated Analysis A Revolutionary Change in How We Design and Build Systems

12. Model-Based Development Examples

13. Does Model-Based Development Scale? Airbus A380 Systems Developed Using MBD • Flight Control • Auto Pilot • Flight Warning • Cockpit Display • Fuel Management • Landing Gear • Braking • Steering • Anti-Icing • Electrical Load Management Length 239 ft 6 in Wingspan 261 ft 10 in Maximum Takeoff Weight 1,235,000 lbs Passengers Up to 840 Range 9,383 miles

14. How Do We Reduce Costsand Improve Quality? Reduces Cost of Testing Clear Specifications Improves Communication Enables More Testing Eliminates Manual Coding Easy Validation Makes Model Primary Artifact Finds Errors Early Requirements Elicitation Reuse 15% 10% Autotest Modeling 5% 10% Autocode Simulation Automated Analysis 10% - 20% Cheaper Than Manual Analysis Finds the Really Hard Errors

15. Outline of Presentation Introduction Overview of Our Approach An Example – FGS Mode Logic Some Recent Accomplishments The Underlying Technology What’s Next? Summary

16. Flight Guidance System Mode Logic Requirements Elicitation Reuse Modeling Autotest Simulation Autocode Automated Analysis

17. Captured Requirements as Shalls

18. Modeling Requirements Elicitation Reuse Modeling Autotest Simulation Autocode Automated Analysis

19. Modeling Notations Tabular (RSML-e, SCR) Textual (Lustre, PVS, SAL, …) node Thrust_Required( FG_Mode : FG_Mode_Type ; Airborne : bool ; In_Flare : bool ; Emergency_Descent : bool; Windshear_Warning : bool ; In_Eng_Accel_Zone : bool ; On_Ground : bool) returns (IsTrue : bool) ; let IsTrue = (FG_Thrust_Mode(FG_Mode) and Airborne) or (Airborne and Emergency_Descent) or Windshear_Warning or ((FG_Mode = ThrottleRetard) and In_Flare) or (In_Eng_Accel_Zone and On_Ground) ; tel ; Graphical (Simulink, SCADE)

20. Simulation Requirements Elicitation Reuse Modeling Autotest Simulation Autocode Automated Analysis

21. Simulation

22. Automated Analysis Reuse Requirements Elicitation Modeling Autotest Simulation Autocode Automated Analysis Model Checkers Theorem Provers

23. What Are Model Checkers? • Breakthrough Technology of the 1990’s • Widely Used in Hardware Verification (Intel, Motorola, IBM, …) • Several Different Types of Model Checkers • Explicit, Symbolic, Bounded, Infinite Bounded, … • Exhaustive Search of the Global State Space • Consider All Combinations of Inputs and States • Equivalent to Exhaustive Testing of the Model • Produces a Counter Example if a Property is Not True • Easy to Use • “Push Button” Formal Methods • Very Little Human Effort Unless You’re at the Tool’s Limits • Limitations • State Space Explosion (1020 – 10300 States)

24. Advantage of Model Checking Testing Checks Only the Values We Select Even Small Systems Have Trillions (of Trillions) of Possible Tests! System

25. Advantage of Model Checking Model Checker Tries Every Possible Input and State! Model

26. Model Checking Process Does the systemhave property X? SMV Automatic Translation Counter Example Properties SMV Properties SMV Spec. Model Automatic Translation Automated Check Yes! Engineer

27. Translated Shalls into SMV Properties

28. Validate Requirements through Model Checking • Proved Over 280 Properties in Less Than an Hour • Found Several Errors • Some Were Errors in the Model • Most Were Incorrect Shalls • Revised the Shalls to Improve the Requirements

29. Translator Optimizations

30. What are Theorem Provers? • Available Since Late 1980’s • Widely Used on Security and Safety-Critical Systems • Use Rules of Inference to Prove New Properties • Also Consider All Combinations of Inputs and States • Also Equivalent to Testing with an Infinite Set of Test Cases • Generate An Unprovable Proof Obligation if a Property is False • Not Limited by State Space • Applicable to Almost Any Formal Specification • Limitations • Require Experience - About Six Months to Become Proficient • Constructing Proofs is Labor Intensive

31. Theorem Proving Using PVS PVS Spec. Automatic Translation Why not? Does the systemhave property X? Guru PVS Automatic Translation Properties PVS Properties Model Automated Proof Engineer

32. Validate Requirements Using Theorem Proving • Proved Several Hundred Properties Using PVS • More Time Consuming that Model-Checking • Use When Models are Stable and Model-Checking Won’t Work

33. Outline of Presentation Introduction Overview of Our Approach An Example – FGS Mode Logic Some Recent Accomplishments The Underlying Technology What’s Next? Summary

34. Example 1 – Mode Logic Mode Controller A 6.8 x 1021 Reachable States Mode Controller B Requirement Mode A1 => Mode B1 Counterexample Found in Less than Two Minutes! Found 27 Errors to Date

35. Example 2 – Displays Logic 883 Subsystems 9,772 Simulink Blocks 2.9 x 1052 Reachable States Requirement Drive the Maximum Number of Display Units Given the Available Graphics Processors Counterexample Found in 5 Seconds! Checked 178 Properties – Found Several Errors

36. Outline of Presentation Introduction Overview of Our Approach An Example – FGS Mode Logic Some Recent Accomplishments The Underlying Technology What’s Next? Summary

37. Original Tool Chain RSML-e to NuSMV Translator NuSMV Model Checker RSML-e PVS Theorem Prover RSML-e to PVS Translator Rockwell Collins/U of Minnesota SRI International

38. Conversion to SCADE NuSMV SCADE Lustre PVS Safe State Machines Design Verfier Rockwell Collins Esterel Technologies SRI International

39. Extension to MATLAB Simulink Simulink Gateway Simulink StateFlow NuSMV SCADE Lustre PVS Safe State Machines Design Verfier Rockwell Collins Esterel Technologies SRI International MathWorks

40. Adding SRI Tools to the Chain Simulink Gateway NuSMV Simulink SCADE PVS Lustre ACL2 StateFlow Safe State Machines ICS Symbolic Model Checker SAL Bounded Model Checker Infinite Model Checker Design Verfier Rockwell Collins Esterel Technologies SRI International MathWorks

41. Current Tool Chain Simulink Gateway Simulink StateFlow ICS Symbolic Model Checker SAL Bounded Model Checker Infinite Model Checker NuSMV SCADE PVS Reactis Lustre ACL2 Safe State Machines Design Verfier Rockwell Collins Esterel Technologies SRI International MathWorks Reactive Systems

42. Outline of Presentation Introduction Overview of Our Approach An Example – FGS Mode Logic Some Recent Accomplishments The Underlying Technology What’s Next? Summary

43. Extending the Verification Domain Theorem Provers SAT-Based Model Checkers Complex Boolean & Enumerated Types + Integers & Reals Infinite State Spaces BDD-Based Model Checkers Boolean & Enumerated Types Very Large State Spaces Arbitrary Systems (Real Numbers, Large Integers, Infinite State…)

44. Verification of Adaptive Systems

45. Requirements Based Test Case Generation Create Requirements Based Tests Test Case Generator Create Model Code Generator Create Additional Structural Tests Test Case Generator Conformance Testing • Autogenerate Test Cases From Model • Commercial Tools Available • (T-VEC, REACTIS) • Show Code Conforms to the Model • Goal is Structural Coverage (MC/DC) Requirements Properties Requirements Based Testing • State Requirements as Properties • Automatically Generate Tests • Goal is to Cover the Requirement Model Code

46. Model-Based Safety Analysis Green Pump Blue Pump Isolation Valve Isolation Valve Power A System A Selector Valve Pedal 1 Shut Normal A Accumulator N Plant System L O Valve Feed back T R M E Accumulator A System B Pedal 2 R Pump L N Power B AntiSkid A Meter Valve Mechanical Command T Pedal E Braking + Meter Fault Tolerant Meter AntiSkid Valve Valve Braking System Command Control Unit Plant Model ( BSCU ) • Add Fault Model for Physical System • Model the Physical System and the Digital Controller Architecture and Digital Controller Architecture • Integrates System and Safety Engineering About a Common Model • Automation Enables “What-If” Consideration of System Designs

47. Outline of Presentation Introduction Overview of Our Approach An Example – FGS Mode Logic Some Recent Accomplishments The Underlying Technology What’s Next? Summary

48. Summary • Formal Verification is Becoming Practical • Availability of Accurate Models Early in the Lifecycle • Growing Power of Automated Analysis Tools • Benefits • Find Errors Early • Avoid Rework Late in the Lifecycle • Cheaper and Easier than Traditional Methods • Orders of Magnitude Better at Finding Errors

49. Almost 15 Years of Experience Thriving Automated Analysis Group Doing Extensive Work for NASA and the NSA Broad Tool Expertise PVS, ACL2, NuSMV, Prover, SAL, Simulink, SCADE, SCR, … Focus on “Application to Real Systems” Summary Rockwell Collins is a World Leader in the Industrial Use of Formal Methods