Accident Investigation and Aircraft Hazard Areas in the Post- Columbia World

1. Accident Investigation and Aircraft Hazard Areas in the Post-Columbia World Paul D. Wilde, Ph.D., P.E. FAA/AST-4 Columbia Accident Investigator

2. Introduction • I was an investigator for the Columbia Accident Investigation Board (CAIB). • At the CAIB, I investigated the technical cause and the public safety issues. • The implications listed are derived from my CAIB and other experience. • Some things have changed since the CAIB, but some thing have not. • Aircraft Hazard Area (AHA) implementation has evolved substantially (Murray AIAA 2010-1349) • Aircraft and space safety and investigation paradigms remain vastly different.

3. Overview of CAIB Findings and Implications for Space Safety Implication Finding • Space launches are risky • Past success does not provide future success • Standards and formal structure can help • Independent technical authorities are valuable • Be prepared for accidents • Understand anomalies • Don’t short cut formal processes • Safety vigilance is challenging

4. Space Vehicles Are Dangerous Evidence Implication • “Building rockets is hard. Part of the problem is that space travel is in its infancy.” CAIB Vol. 1 page 19 • “Building and launching rockets is a very dangerous business and will continue to be so for the foreseeable future” CAIB Vol. 1 page 19 • Rockets fail catastrophically 10-100 thousand times more often than commercial transport aircraft (per flight). • Accidents should be expected; prepare plans for emergency response • Prepare investigation and RTF plans, including interface to media and other orgs • No presumption of safety: accidents usually stop all flights until cause is found

5. Independent Technical Authorities Are Valuable Implication Evidence • A compliance verification organization independent of operational program cited as key to success for Navy subs and nuclear reactors, and in Air Force launch verification. • “Organizations that deal with high risk operations must always have a healthy fear of failure - operations must be proved safe rather than the other way around.” CAIB Vol. 1 page 190 • Independent compliance verification enhances the safety of complex technical systems • Checks and balances promote communication (in-flow of new info, addressing minority opinions) • Safety takes real effort

6. Public Safety • Columbia break-up during re-entry clearly could have caused public casualties • Lack of public casualties due to Columbia break-up was the expected outcome given the sparse population • P>1 serious injury was <50% (~10-30%) • Same accident over a major city expected to produce a few public casualties • Hypersonic ops late at night lowers risk • Roofs protect effectively from most debris • Relatively high probability of failure makes “safe” for public difficult to verify

7. Risk to Aircraft Flying Near Columbia Break-up • At the time of Columbia break-up, FAA was unaware of any hazard to aircraft. • TFR issued ~ 45 minutes afterward based on radar detection of debris, media rept., etc. • Post CAIB analysis by FAA showed aircraft PI ~ 0.001 to 0.01 • Post CAIB simulation illustrates the issue • Actual aircraft flight locations/trajectories • Blue dots are recovered debris locations • Statistical distribution of debris during fall • The view is from the southeast • Green lines show County boundaries

8. 8

9. Safety of Aircraft Flying Near Space Launch or Re-entry • To provide safety and efficiency in US NAS, both pre-defined and real-time AHA are used. • AHA for planned debris (jettisoned stages) • Break-up generally spreads debris over a large area; aircraft PI often exceeds 1E-6 • During exo-atmospheric flight, several minutes between break-up and debris reaching aircraft altitudes. • Vulnerability of aircraft to such debris strikes is highly uncertain and under investigation.

10. BACK-UP

11. Sub-models for AHA Development The last two (vulnerability and impact probability), plus the risk criteria for aircraft, have aspects that are necessarily unique to aircraft hazard area analysis; all other sub-models are common with the debris risk analysis

12. Aircraft Grid & Trajectory Approaches to PI Estimate • Grid approach • Assumes aircraft continuously present in each grid cell • Produces conservative results • Specified trajectory • Accounts for aircraft azimuth and limited dwell time in each cell • More realistic PI is 2x to 7x lower

13. Aircraft Vulnerability Modeling Airbus A300: Struck by a missile at 8,000 ft but landed safely 22 Nov 2003 See Wilde & Draper AIAA paper 2010-1542

14. Current Efforts Toward Higher Fidelity Aircraft Vulnerability Models (AVMs) • FAA sponsored higher fidelity analysis using previously developed tools (e.g. military) and input data • FAA impact testing to improve skin penetration eq., evaluate • Influence of obliquity, fragment density, distance from support, etc. • Available results show • Current penetration equation is conservative • 321-10 AVMs are excessively conservative, esp. for “catastrophe”

15. Public Safety Findings • NASA should • Implement public risk acceptability policy • Mitigate public risk from STS flight • Study debris to improve risk estimates • Collective public risk from space flight is small compared to civil aircraft operations. • Principle reason is huge number of aircraft operations relative to launches. • One in a million risk to individuals is a recognized benchmark for both and others • Complete report at www.caib.us Vol.II D-16

16. Understand Anomalies Implication Evidence • O-ring blow-by and foam impacts were previously detected as anomalies • The cause, effect, and limits of these anomalies were not understood • “Engineers understood what was happening, but they never understood why.” CAIB Vol. 1 page 196 • Anomalies are often early warnings • Successes do not prove problem solved or not dangerous • Examine all data on anomalies separately and as a set • Provide technical rigor in all requirements, rationales, validations

17. Formal Structure Can Help Implication Implication • Formal standards help define what is an anomaly • More uncertainty, justifies more attention and more caution • Formal documents and peer reviews promote better decisions and help inform future generations • Formal documentation traces what was done to verify requirements were satisfied • Formal structure can ensure that the burden of proof is on those saying it’s safe • Formal structure identifies the responsible party

18. Informal Processes Are Not Effective Implication Evidence • Several informal attempts to obtain on-orbit imagery failed • Lack of ground rules hampered engineering teams that evaluated the issues CAIB Vol. 1 page 200 • Management teams violated their own rules • “When …analyses are condensed to fit on a…overhead slide, information is inevitably lost.” CAIB Vol. 1 page 191 • Clearly defined roles and rules improve effectiveness • Design structure to promote communication • Minority opinions should be addressed • Communication needs to flow both up and down the chain of command