Overview

1. Automation for System Safety AnalysisJane T. Malin, Principal InvestigatorProject: Automated Tool and Method for System Safety Analysis Software Assurance SymposiumSeptember, 2008Complex systems typically fail because of the unintended consequences of their design, the things they do that were not intended to be done. - M. Griffin, System Engineering and the “Two Cultures” of Engineering, March 28, 2007

2. Overview • Problem/NASA Relevance • Technical Approach and Overview • 2008 Target Capability • Crew Exploration Vehicle (CEV) Launch Abort System Case • Data – Constellation (Cx) failure modes and effects analysis/critical items lists (FMEA/CILs) • Technical Challenges • Information Extraction • Semi-Automated Model Construction • Analysis and Test Case Generation • 2009 Planned Capability • Potential Applications

3. Problem and NASA Relevance • NASA needs early evaluation of software (SW) requirements and design, to reduce software-system integration risks • Assess system failures and anomalous conditions that may challenge software in system integration testing • Identify robustness issues early (and often) • Identify requirements gaps early (and often) • Project test case: NASA Constellation (Cx) Launch Abort System (LAS) for Pad Abort PLANT and Environment ‘Activate’ Faults and Influence Failures SOFTWARE ‘Activate’ Faults and Influence Failures FAULTS/Reliability FAULTS/Reliability Operations and Stresses

4. Technical Approach Systematic semi-automated extraction and analysis for early evaluation and rapid update • Capture model of the controlled system architecture • Abstract physical architecture models with subsystems, functions, interfaces, connections • Extract directly from requirements and design text and data • Capture risks and hazards in model • Constraints, hazards, risks from requirements and design • Risk and failure libraries • Analyze and simulate to identify risks and constraints • Analyze and simulate hazard/risk propagation in the system • Use operational and off-nominal scenarios and configurations • Identify possible test scenarios for virtual system integration testing

5. Technology Overview Hazard Identification Tool (HIT) Aerospace Ontology CONFIG Hybrid Simulation Extract and Model Information Extractor Identify Test Cases Requirements Text • Analyze, Simulate and Test Early • Identify interaction-propagation paths • Investigate influence of timing • Perform Virtual Tests Virtual System Integration Laboratory (VSIL)

6. Modeler: Architecture Model and Visualization of a Set of Requirements • [C.1] Telecommunication Subsystem (TeleSub) • [C.1.1] The CDHC sends the TeleSub a compressed picture. [FG.1] [TeleSub C.1.4] • [C.1.2] The CDHC sends the TeleSub telemetry. [FG.2] [FR.1] [FR.5] [TeleSub C.1.5]  • [C.1.3] The CDHC sends In View of Ground alerts to the TeleSub. [DP.5.6] [TeleSub C.1.6] • [C.1.4] The CDHC receives plan files from the TeleSub. [FR.3] [TeleSub C.1.3] • [C.1.5] The CDHC receives ground commands from the TeleSub. [FR.3] [TeleSub C.1.2]  • [C.1.6] The CDHC receives the TeleSub operating state from the TeleSub. [DP.5.5] [TeleSub C.1.1] … • [C.2] Camera Subsystem • [C.2.1] The CDHC sends the Camera a "take picture" command. [FG.2] [FR.1] [FR.3] • [C.2.2] The CDHC sends the Camera x, y and z gimballing coordinates. [FG.2] [FR.1] [FR.3]   • [C.2.3] The CDHC sends a turn on command to the Camera. [DP.5.3] [H Constraint 1.1.4] • [C.2.4] The CDHC sends a turn off command to the Camera. [DP.5.3] • [C.2.5] The CDHC receives a compressed picture file from the Camera. [FG.1] [FG.2] [FR.1] • … • [C.4] Attitude Determination Subsystem (ADS) • [C.4.1] The CDHC receives an In View of Ground alert from the ADS. [DP.5.6] [ADS] • [C.4.2] The CDHC receives the ADS operating state from the ADS. [DP.5.5] [ADS] • Note: CDHC is Command and Data Handling Computer Physical/Functional Architecture Visualization

7. CONFIG Simulation: Assess Timed Scenarios CONFIG simulation tool used for software virtual validation testing for NASA 1997 90-day manned Lunar Life Support Test • Software: Intelligent control for gas storage and transfer • Models: Gas volumes and processing systems controlled by software; mixed fidelity, discrete and continuous • Testing: Simulated failures and imbalances that would not be tested in hardware-software integration • Too slow to develop, too expensive, too destructive • Results: Identified software requirements deficiency due to unintended consequences of integrating gas processing systems

8. Virtual System Integration Lab (VSIL) Models and Test Definitions • Triakis has used VSIL in >25 avionics verification projects • Project Output to VSIL: Models and test definitions DE: detailed executable, the simulation of the embedded controller hardware RAM/ROM: memories ES: executable specifications I/O: input/output V&V: verification and validation CPU: processor

9. 2008 Target Capability • Integration: Information extraction, architecture modeling and test generation • Model parts extracted from requirements and FMEA/CIL texts • XML output, including reference traces • Components, physical hierarchy, connections, interface components, flows/resources, time or phase context • Functions, vulnerabilities, limits, failures, causes • Ontology for model extraction and semi-automated modeling • Identify types of components, functions, problems, resources • Paths: A provides power to B; C receives command data from B • Functions and failures: B processes command data; B failure mode is No Output command to D; cause of no output is B does not receive power. • Semi-automated model development from extracted model parts • Component model library: Resource producer; Data processor… • Generic functions, failures and influences: Resource problem, Stressor, Data rate problem, Data Integrity problem… • Model visualization for overview and completeness checking • Simulation and path analysis to identify hazardous configurations, scenarios and test cases • Where failure or degradation of required functions results from unintended system interactions • Project Participants • CEV Flight Software Engineering, Abort Decision Logic, Abort Sequence • Orion Software Safety and Mission Assurance

10. CEV Launch Abort System (LAS) Case Crew Exploration Vehicle (CEV) Pad Abort Sequence - notional

11. CEV Launch Abort System Case • CEV Crew Module (CM) software controls the Pad Abort Sequence – LAS events, trajectory • No direct command feedback • Components for the case • Paths and interactions for commands firing separation pyros during pad abort sequence • CM computer → Remote Interface Unit → LAS Pyros • Possible Addition Case: Inertial measurement unit (IMU) → GN&C → Abort motor • Summer Systems Engineering Intern manually built TEAMS models of pyros and Remote Interface Unit (RIU) from FMEA/CILs • Need to perform simulation rather than pure path analysis on the LAS case • Timing is important in aborts • Hazard Identification Tool (HIT) path analysis models do not capture timing • CONFIG simulations can use timing

12. Data – ConOps, Requirements, Safety Analyses • Developing tools and methods using documents and data from CEV sources, both NASA and Orion contractor • Met with NASA expert on Orion software that controls Launch Aborts • Identified key CEV documents and confirmed analysis approach • Orion Contractor’s Concept of Operations • Best guess at the Abort Sequence • Interface Requirements Documents • Interface Control Documents, when they become available • Project Orion Flight System Safety Hazard Analysis • FMEA/CILs (preliminary now) • Determined that much key information (sensors, feedback) is TBD, and being defined by the Cx Integrated Abort Team

13. Technical Challenges • Limitations of early life-cycle requirements, design, hazard analysis and FMEA/CIL as sources for • Automatic extraction of model information from requirements and design text • Semi-automatic construction of models from extracted information, for simulation and visualization • Combining graph analysis and simulation to identify possible hazard paths and off-nominal test scenarios for complex system interaction models • Maturation Challenge: Develop mature software prototypes that can be used to develop products for broader use

14. Information Extraction • Objective: Extract information from CEV sources for semi-automated model construction • Information Extraction Evaluation • Success Criterion: % of available model information extracted, compared to % of model information available • Types of Extractions (from text to XML) • Interface Requirements → Components, connections, flows/resources, reference trace • Some interface components, vulnerabilities, functions, limits, context (time or phase) • FMEA/CILs → Components, system to component hierarchy, interfaces, subcomponents, functions, failures, causes, effects • Architecture Descriptions → Components, interfaces, hierarchy, functions; some design parameters, acronyms • Challenges • Multiple document formats require definition of data structures for each document, some difficult sentence parsing • Indirect access to Cradle requirements via PDF documents • Progress: • Parser improved by incorporating NESC-funded parser from Univ. Central Florida • Experience coding multiple document data structures, leading to format specification approach • Successful extraction from Cradle-based PDF docs.

15. Example Information Extraction Benchmarking • LAS Description: The LAS consists of a nose cone, a canard section which enables the LAS to reorient the CM for parachute deployment following an abort, three propulsive motors (attitude control, jettison, and abort), a bi-conic adapter which provides the structural interface to the CM, and a boost protective cover (BPC) sized for ascent heating to protect CM thermal protection system (TPS) coatings. • Ideal Model Extraction Benchmark: • Top Level: LAS • Function: reorient the CM • Agent: LAS • Action: reorient • Operand: CM • Function: control attitude • Agent: LAS? • Action: control • Operand: ? • Variable: attitude • … • Function: deploy parachute • Agent: LAS? • Action: deploy • Operand: parachute LAS Components Component: Nose cone Component: Canard Section Function: ? Component Enables Function: LAS: reorient the CM Function Enables Function: LAS?: deploy parachute (following an abort) Component Group: Three propulsive motors Component: attitude control motor Enables Function: LAS?: control attitude … Component: A bi-conic adapter Connection Connector: bi-conic adapter From: LAS To: CM Type: Structural Component: Boost protective cover Acronym: Boost protective cover = BPC Design parameter: size Determined by: ascent heating Function: protect CM thermal protection system coatings Agent: Boost protective cover Action: protect Operand: CM thermal protection system coatings Other: CM Component: Thermal protection system Acronym: thermal protection system = TPS

16. Model Construction • Objective: Use extracted model information for semi-automatic model construction • Challenges • LAS case with timing issues requires CONFIG simulation, not just HIT architecture model • Missing information in pre-PDR documents • Operating modes, vulnerabilities, side effects etc. • Progress • CONFIG provides visualization and supports libraries of generic components • Concept for use of Aerospace Ontology hierarchies with CONFIG library for generic components, operating modes, functions, side effects, problems

17. Analysis and Test Case Generation • Objectives: • Identify and evaluate failure and hazard propagation in the system model • Unintended system interactions and unanalyzed propagation of failure effects • Generate corresponding off-nominal test cases (configurations, scenarios) • Evaluations • Hazard Analysis Success Criterion: Are new hazards and failures identified, compared to standard method? • Test Case Success Criterion: Does model-based hazards and failures analysis make test generation easier than current methods? • Challenges • Path analysis algorithm for HIT models needs to be adapted, because LAS case will use CONFIG simulation rather than HIT • Extracting information for operational and failure scenarios has not yet been addressed • FMEA/CIL effects information can provide parts of failure scenarios • Progress • Concept for combining HIT and CONFIG models, using CONFIG Inner Models for subsystem details

18. 2009 Planned Capability • Capabilities should be valuable from pre-PDR through operations • Continue tool enhancements focusing on • Off-nominal test scenario discovery and evaluation • Component model library and generic defaults • Use on a new CEV case – more complex interactions and more complete system information • Deliver • Tool prototype files – Information extraction tool, Aerospace ontology, HIT graph modeling and analysis, CONFIG simulation modeling, model libraries • Documentation – methods, tools, user manuals

19. Future Applications • Improve efficiency and repeatability of system and software risk analysis • Reduce time spent reanalyzing when specifications and designs change • Visualize integrated requirements • Combined success and failure spaces • Combined system and operation/event spaces • Validate requirements and perform integration tests early with low-fidelity and multi-fidelity simulation • Validate FMEAs and fault trees • Evaluate completeness and consistency of requirements and risk • Support requirements traceability evaluations • Enhance analysis with reliability and event probability information