National Aeronautics and Space Administration

2. From Determinism to “Probabilism” National Aeronautics and Space Administration Changing our mindsets, or why PTC isn’t an easy sell - yet www.nasa.gov

3. My Muddled Mind Robust Design ? Deterministic Design Probabilistic Design DOE Predictive Models & Algorithms SDF PRA Safety Factors Block Diagrams, Systems of Systems Models Life Cycle Prediction Redundancy PDF FMECA, FTA, RAM Design DFMR Diagnostics & Prognostics SWP PD

4. System Safety Tools Feed Risk Informed Decision Making Bryan O’Connor, Chief, Safety and Mission Assurance, NASA SEPT 8, 2007, Huntsville, AL

5. S&MA in Ares Design – Summary • Using the Ares quantitative safety and reliability requirement to enforce the “design for reliability and safety” paradigm shift • Improving system safety by using a functional analysis system approach to model and understand integrated system failures similar to the Shuttle foam problem. • Improving design reliability by using probabilistic engineering physics-based modeling • Evaluating and understating design uncertainty and design margins using probabilistic engineering techniques Excerpted from Chris Cianciola, S&MA, MSFC, CQSDI March 2008

6. Uncertainty Quantification and Risk Assessment risk-informed decision making Uncertainties sources manufacturing, storage, aging, use environments, and scenarios Uncertainties types randomness, lack of knowledge Model to predict uncertainties in systems Uncertainties outside acceptable limits? Uncertaintiesproduce inadequate margins? Most important contributors to uncertainty (sensitivity analyses)? Physics model form uncertainty Sensitivity Analysis Only several contribute most of the uncertainty in system response Requirements Uncertainty

7. Probabilistic Models Limitations One of many decision variables Subjective information - scientific and engineering judgment is necessary Known unknowns and unknown unknowns exist Resources are limited Results uncertainty and credibility – challenges decision makers Social, economic, and political factors exist

8. Credibility • Is the methods framework credible? • Address the credibility concern with verification and validation

9. Special study data • Start with robustness and margin in design • A 30% design margin is not plausible for space flight – it’s just too expensive • Our margins are so thin that we really need to investigate margin • In our case we want to whittle the margin down as much as possible but still maintain the robustness we need • Getting those design margin trades into the system analysis is critical NASA special study data 2006

10. Special study data • We still use deterministic design and build margin into designs even though we have the computational capabilities to us PT methods • We do not have probabilistic design methods in place • Probabilistic methods will have to be an investment by NASA into the long term technology base and tools • First you need a proper deterministic goal and then you can successfully apply a probabilistic model • Numerical Propulsion System Simulation is something that must be implemented to get this country to Mars • The entire thought process needs to change, this is not a deterministic world – everything is probabilistic • In the deterministic method there is no role for uncertainty or margin • Right now technology is so fast we are dumping technology on people who are not educated to use these technologies appropriately NASA special study data 2006

11. Special study data • Redundancy is a part of our culture right now because it is much easier than looking for a solution using margin • When you consider uncertainty in the design phase you are able to examine the trade space more efficiently and thoroughly. • The nature of the probabilistic skills dealing in design is very different from the traditional PRA. NASA special study data 2006

12. Why PTC Now? • Design engineers don’t understand how to account for uncertainty in the design process and how to explore the trades throughout • In the past we’ve had so much money and time for testing, but we don’t have this anymore • Apollo did 14,000 tests on the LEM, do you think we could do that now? • We must trade ability to test versus desired reliability demonstration NASA special study data 2006

13. Other Notable Thoughts • Probabilistic requirements ultimately relate to risk in achieving a level of performance or safety. • The most important reason for using a structured approach is to ensure that the potential failure modes of a design are identified early in the design process to better understand the risks. • An understanding of risks due to incomplete or inaccurate modeling of a design and the operational environment, or due to uncertainty related to the data used in a model is necessary so that a requirement can be verified with confidence. Surendra N. Singhal, Engineering Directorate, Marshall Space Flight Center, Sept. 18, 2007

14. A Way Forward • Need a coherent institutionalized effort • Need a regular training of the entire Center staff (many unaware of PT and its benefits) • Need a core group where people and projects can go for help

15. PT Selling Points • Utilizes Physics based behavioral model • Considers inherent uncertainties, modeling uncertainties, lack of data, human error, measurement error • Compensates for unknowns using statistical methods • Utilizes past performance data to develop behavioral model • Quantifies safety measures • Qualifies prediction accuracy Robert J. Kuper, Executive for Reliability and Quality, Systems Engineering Conference, 23 October 2002