Terry Hardy AST-300/Systems Engineering and Training Division May 19, 2004

1. Assessment of Alternate Methodologies for Establishing Equivalent Satisfaction of the Ec Criterion for Launch Licensing Terry Hardy AST-300/Systems Engineering and Training Division May 19, 2004

2. Project Description • P. Birkeland and J. Greason have proposed alternate methods to satisfy the Ec criterion of 30x10-6. • One alternate methodology uses derived reliability requirements, reliability allocation strategies, and Failure Modes, Effects and Criticality Analysis (FMECA) to demonstrate an equivalent level of safety to Ec criterion. • AST will examine this alternate methodology and compare it to existing approaches.

3. Plan • AST will conduct the following in-house tasks: • Review documentation by Birkeland/Greason on the proposed process. • Examine the FMECA process, and investigate its advantages and disadvantages • Investigate the use of FMECA within FAA, for aircraft, ELV and RLV. • Compare the proposed FMECA process to AST’s existing process. • Document findings in a white paper • Present findings to COMSTAC RLV Working Group • Milestones include: • Status to RLV Working Group May 2004 • Draft Report July 2004 • Final Report September 2004 • Presentation at COMSTAC in October 2004

4. Expected Casualty Analysis Expected Casualty Analysis: • Is a well-established collective risk measure. • Has been successfully used for decades in the aerospace community for both launch vehicle risk analyses and explosive safety analyses. • Quantifies both probability and severity in assessing risk to public safety. But… • Ec analysis method is complex (inputs include characteristics for explosive and inert debris, weights/sizes, influence of wind and aerodynamic properties on debris, shelter effects on casualty area, etc.)

5. Alternate Approach The alternate approach proposed by P. Birkeland can be summarized as follows: • Derive an allowable probability of catastrophic failure for RLVs based on commercial aircraft failure rate requirements and historical ground casualty rate. • Allocate RLV failure probability to hardware and operator and allocate by phase of flight. • Use an FMECA to demonstrate that the allocated failure rate has been obtained.

6. Initial Findings The proposed approach is appealing because: • Setting reliability goals is consistent with approaches used by NASA, FAA for commercial aircraft. • Setting reliability requirements for passenger-carrying RLVs is consistent with approach taken by FAA for commercial aircraft. • Allocating reliability between hardware and non-hardware systems explicitly recognizes that system safety is more than just hardware failure.

7. Initial Findings The proposed approach is appealing because: • The methodology directly links the system safety process to expected reliability objectives. • FMECA is an excellent tool for methodically identifying safety issues and mitigation measures during design, especially when used in parallel with other hazard analysis/risk assessment tools.

8. Initial Findings AST has some concerns with this approach: • It may be difficult to justify a top-level failure probability or reliability allocations based on a comparison to aircraft. • Consequence of a launch vehicle crash is potentially higher than aircraft due to presence of fuel and oxidizer. • Most RLVs will operate at high velocity with higher kinetic energy and higher risk of aerodynamic breakup. • Aircraft industry is mature, with years of history and performance, in comparison to RLV industry. • The proposed approach does not explicitly incorporate population density or casualty area.

9. Initial Findings AST has concerns with any approach based only on an FMECA: • It is likely that the FMECA will miss some failure modes, especially if software and human interactions are not considered. • FMECA does not consider combinations of failures, and most accidents are the result of a confluence of factors, and often due to a combination of seemingly low-consequence failures. • FMECA does not include a quantitative assessment of severity. • FMECA does not usually account for uncertainties in the input data. • FMECA can provide optimistic system reliability estimates.

10. Further Work • AST will continue its investigation of this proposal, comparing it to the existing Ec methodology and other analytical approaches. • AST will prepare a white paper documenting its findings. • AST will present the full report at the next COMSTAC meeting.