A Review of CAT II/III LAAS Integrity Requirements and their Antecedents

1. A Review of CAT II/III LAAS Integrity Requirements and their Antecedents Sam Pullen Stanford University (with lots of help from Tim Murphy of Boeing) Stanford GPS Laboratory Group Meeting 4 August 2006

2. English Word of the Day… • Antecedent: (Webster online dictionary) 1 : a substantive word, phrase, or clause whose denotation is referred to by a pronoun (as John in "Mary saw John and called to him"); broadly : a word or phrase replaced by a substitute  grammar only 2 : the conditional element in a proposition (as if A in "if A, then B")  grammar only 3 : the first term of a mathematical ratio  rarely used 4 a : a preceding event, condition, or cause b plural : the significant events, conditions, and traits of one's earlier life  very general 5 a : PREDECESSOR; especially : a model or stimulus for later developments b plural : ANCESTORS, PARENTS CAT II/III Integrity Requirements and Antecedents

3. Presentation Outline • Review of LAAS Precision Approach Requirements • Antecedents of these requirements: • ICAO Annex 10 Requirements for ILS • FAA AC 25.1309 and AC 120-28D wording • FAA Hazard Risk Index table • Total Aircraft Safety sub-allocation • What should the “real” be, and how should it be derived? • Some initial thoughts… CAT II/III Integrity Requirements and Antecedents

4. Precision Approach Requirements in Updated LAAS MASPS(RTCA DO-245A, December 2004)

5. GBAS Service Level (GSL) Definitions Table 1-1 (Section 1.5.1) of DO-245A CAT II/III Integrity Requirements and Antecedents

6. GSL Requirements Table Table 2-1 (Section 2.3.1) of DO-245A CAT II/III Integrity Requirements and Antecedents

7. Antecedents of Precision Approach Requirements1: FAA Hazard Risk Index Useful reference: Ch. 3 of FAA System Safety Handbook (12/30/00) http://www.faa.gov/library/manuals/aviation/risk_management/ss_handbook/media/Chap3_1200.PDF

8. FAA Risk Severity Classifications* • Minor: failure condition which would not significantly reduce • airplane safety, and which involve crew actions that are well within • their capabilities • Major: failure condition which would significantly: • (a) Reduce safety margins or functional capabilities of airplane • (b) Increase crew workload or conditions impairing crew efficiency • (c) Some discomfort to occupants • Severe Major (“Hazardous” in ATA, JAA): failure condition resulting • in more severe consequences than Major: • (a) Larger reduction in safety margins or functional airplane capabilities • (b) Higher workload or physical distress such that the crew could • not be relied upon to perform its tasks accurately or completely • (c) Adverse effects on occupants • Catastrophic: failure conditions which would prevent continued safe • flight and landing (with probability --> 1) Cat I Cat III * Taken from AC No. 25.1309-1A, AMJ 25.1309, SAE ARP4761 (JHUAPL summary) CAT II/III Integrity Requirements and Antecedents

9. FAA Hazard Risk Index (HRI) Table • Several versions exist, all with essentially the same meaning • Source of this version: 1999 Johns Hopkins Applied Physics Laboratory “GPS Risk Assessment Study” final report http://www.faa.gov/asd/international/GUIDANCE_MATL/Jhopkins.pdf Cat. I ILS case Cat. III ILS case CAT II/III Integrity Requirements and Antecedents

10. Antecedents of Precision Approach Requirements2: FAA Advisory Circulars Defining Certification and Airworthiness Criteria • For AC 25.1309-1A, “System Design and Analysis,” 6/21/88: • http://www.airweb.faa.gov/Regulatory_and_Guidance_Library%5CrgAdvisoryCircular.nsf/0/50BFE03B65AF9EA3862569D100733174?OpenDocument • For AC 120-28D, “Criteria for Approval of Category III Weather Minima for Takeoff, Landing, and Rollout,” 7/13/99: • http://www.airweb.faa.gov/Regulatory_and_Guidance_Library%5CrgAdvisoryCircular.nsf/0/BBADA17DA0D0BBD1862569BA006F64D0?OpenDocument

11. Key Elements of AC 25.1309-1A • AC 25.1309-1A is the primary basis for safety certification within the FAA • AC 25.1309-1A specifies a “fail-safe” policy (quote): • In any system or subsystem, the failure of any single element, component, or connection during any one flight (e.g., brake release through ground deceleration to stop) should be assumed, regardless of its probability. Such single failures should not prevent continued safe flight and landing, or significantly reduce the capability of the airplane or the ability of the crew to cope with the resulting failure conditions. • Subsequent failures during the same flight, whether detected or latent, and combinations thereof, should also be assumed, unless their joint probability with the first failure is shown to be extremely improbable. • AC 25.1309-1A defines the likelihood and severity terms found in the Hazard Risk Index • Provides guidance as to what factors can be taken credit for in probability assessments and how this should be done • Refers to RTCA DO-178 for software safety assurance guidance • More recent SAE standards (ARP 4754 and 4761) provide much more detailed guidance on FAA safety-assurance methods CAT II/III Integrity Requirements and Antecedents

12. Summary of CAT III Airworthiness Requirements (Table from Tim Murphy of Boeing) Tim Murphy’s presentation is inside RTCA SC-159 WG-4 Archive File: http://sc159.tc.faa.gov/wg4/060706/Jun072006.htm CAT II/III Integrity Requirements and Antecedents

13. CAT III Touchdown Zone (or “Box”) Figure from Figure 3 of Tim Murphy’s requirements report to FAA: Boeing Doc. # D6-83447-4, 10/19/05 Numbers taken from App. 3, Section 6 of FAA AC 120-28D Additional “bank angle hazard” requirement limits probability of any part of wing or engine touching ground to 10-7 or less CAT II/III Integrity Requirements and Antecedents

14. Translation of Touchdown Zone into Landing System Requirements • Provided in ICAO Annex 10 for ILS (April 1985) • not available online • Annex 10 was amended for MLS and is being amended for GBAS  Amendment 79 is latest (?) • Annex 10 specifies 95% accuracy limits and monitor limits in terms of ILS measurements (DDM) • Translation to LAAS required knowledge or assumption of several non-obvious intermediate parameters • In my understanding, ILS requirements in Annex 10 were designed around already-fielded ILS systems that were already deemed to be safe • CAT III guidance requirements were not much more strict  main difference was tighter, higher-reliability monitoring needed CAT II/III Integrity Requirements and Antecedents

15. Antecedents of Precision Approach Requirements3: Example Risk Allocations Source: R.J. Kelly, J.M. Davis, “Required Navigation Performance (RNP) for Precision Approach and Landing with GNSS Application,” Navigation, Vol. 41, No. 1, Spring 1994, pp. 1 – 30. http://www.ion.org/search/view_abstract.cfm?jp=j&idno=106

16. Breakdown of Worldwide Accident Causes: 1959 - 1990 (from ICAO Oct. 1990 Study) • Total hull loss probability per flight (“mission”) as of 1990 = 1.87 × 10-6 • Current probability per commercial departure in U.S. = 2.2 × 10-7 (3-year rolling average last updated in March 2006) • http://faa.gov/about/plans_reports/Performance/performancetargets/details/2041183F53565DDF.html CAT II/III Integrity Requirements and Antecedents

17. U.S. Accident Breakdown by Cause (2000-01) 2000 2001 From NSTB Annual Review of Aircraft Accident Data, 2000 and 2001; ARC 04/01; 06/01http://www.ntsb.gov/publictn/A_Stat.htm CAT II/III Integrity Requirements and Antecedents

18. Semi-unofficial “Serious Accident” Risk Allocation (proposed in 1983 SAE paper†) Numbers based on approximations of observed accident history. Total Serious Accident Risk 10-6 per flight hour 10% 90% 1 × 10-7 p. f. hr. 9 × 10-7 p. f. hr. Aircraft System Failures (engines, control, avionics, etc.) All Other Causes (human error, weather, etc.) Assume 100 sepa-rate aircraft systems Not subject to certification; thus not broken down in detail here. Each individual system is allocated 1 × 10-9 p. f. hr. (or per flight). †D.L. Gilles, “The Effect of Regulation 25.1309 on Aircraft Design and Maintenance,” SAE Paper No. 831406, 1983. CAT II/III Integrity Requirements and Antecedents

19. How should the “real” CAT II/III requirements (and other aviation safety requirements) be determined (work in progress )?

20. Weaknesses in Current Safety Approach • No clear means to adapt safety requirements to continued improvement in overall aircraft safety • 10-9 requirement per individual aircraft system appears to be out-of-date given that current overall serious accident risk is approaching 10-7 per flight • 10-6 probability for landing in CAT III touchdown zone seems dated • No clear means to appropriately balance rare-event probabilities • 10-9 qualifies as “extremely improbable”, but 5 × 10-9 only qualifies as “improbable” and must be treated as “latent” with probability 1 according to strict reading of AC 25.1309-1A • No means to “trade off” safety benefit vs. safety risk for new systems that, when working properly, reduce the risk of accidents caused by pilot/weather/ATC/etc. • Most new systems, including SBAS and GBAS, likely retire more pilot/weather/ATC risk than they introduce due to the possibility of their own failure CAT II/III Integrity Requirements and Antecedents

21. FAA Safety Engineering Tries to Adapt • FAA shows no interest in fundamentally changing current certification standards • Instead, FAA reacts to accidents on a case-by-case basis and tries to change individual rules interpretations subtly and quietly • New interpretations also apply to new systems, such as SBAS and GBAS • Example 1: aircraft rolling out long and off runway (recent SWA 737 accident at Midway) • FAA now promulgating requirements “clarification” mandating a specific 15% runway margin; see: http://aviationnow.com/avnow/news/channel_busav_story.jsp?id=news/FAA06196.xml CAT II/III Integrity Requirements and Antecedents

22. FAA Safety Engineering Tries to Adapt (2) • Example 2: TWA 800 (July 1996) 747 explosion most likely caused by ignition of center fuel tank • NTSB accident report (August 2000): http://www.ntsb.gov/publictn/2000/AAR0003.pdf • Many small fuel-tank risk- reduction steps implemented under SFAR 88 beginning in 2001 • Major ignition-suppression retrofit proposed in Notice of Proposed Rule Making (NPRM; Nov. 2005) • http://dmses.dot.gov/docimages/pdf94/373450_web.pdf • Lengthy technical and cost-benefit debate on this NPRM continues to this day; see: • http://dmses.dot.gov/docimages/pdf94/373645_web.pdf • http://dmses.dot.gov/docimages/pdf95/389033_web.pdf CAT II/III Integrity Requirements and Antecedents

23. FAA Safety Engineering Tries to Adapt (3)(Continuation of Example 2: TWA 800 Accident) • Previous certification of fuel tank safety relied on need for multiple triggering events to occur  joint probability was below 10-9 per flight • However, initiating event could lie undiscovered for many flights prior to being detected by periodic maintenance • New FAA “specific risk” concept requires that “knowable” latent defects be treated as present with probability 1 • Thus, 10-9 mitigation argument no longer holds in this case • Also, undetected latent failure could leave aircraft only one failure away from “catastrophic” incident • FAA and manufacturers have been debating this application of “specific risk” since 2002; see: • https://www.faa.gov/regulations_policies/rulemaking/committees/arac/minutes/media/TAE_OCT_05.pdf • http://edocket.access.gpo.gov/2006/pdf/E6-4024.pdf CAT II/III Integrity Requirements and Antecedents

24. Summary • A complex set of requirements and guidance documents links today’s CAT II/III landing requirements to overall FAA safety objectives • As CAT II/III requirements are refined to be more “GBAS-specific,” re-thinking of the intent of the antecedents of these requirements is important • FAA safety requirements evolution is limited in scope and is limited to “new” systems like SBAS and GBAS and response to external events, e.g., accidents • Further changes to better reflect improved overall aircraft safety and safety contribution of newer systems would be desirable CAT II/III Integrity Requirements and Antecedents

25. Backup Slides Follow… CAT II/III Integrity Requirements and Antecedents

26. Integrity Requirement Definitions • Integrity relates to the trust that can be placed in the information provided by the navigation system • Misleading Information (MI) occurs when the true navigation error exceeds the appropriate alert limit (an unsafe condition) without annunciation • Time-to-alert is the time from when an unsafe condition occurs to when the alarm message reaches the pilot (guidance system) • A Loss of Integrity (LOI) event occurs when an unsafe condition occurs without annunciation for a time longer than the time-to-alert limit, given that the system predicts it is available CAT II/III Integrity Requirements and Antecedents

27. Notes to GSL Requirements Table Section 2.3.1 of DO-245A 1. The values given for GNSS accuracy and alert limits are those required for the intended operation at the lowest height above threshold (HAT) where the GNSS guidance is relied upon. 2. The definition of the integrity requirement includes an alert limit and a time to alert, against which the requirement can be assessed. 3. The accuracy requirements include the nominal performance of a fault-free airborne subsystem. 4. The integrity requirements are specified in terms of a probability to be evaluated over a specified period. The duration of this period is intended to correspond to the most critical portion of an approach & landing for the operations the GSL is intended to support. Integrity risk includes the probability of latent failures, and the exposure time to these types of failures may exceed the specified period, therefore the requirement must apply during “any” period. Note that if the integrity requirements for GSL D-F are met, the integrity requirements for GSL A-C are also automatically met. 5. For these GSLs (D, E, and F), the combined lateral and vertical risk shall not exceed 1 × 10-9, where the risk for vertical applies over any 15 sec, and the risk for lateral applies over any 30 sec. The lateral period is longer because these GSLs are intended to support operations that require LAAS guidance during roll-out. 6. The time-to-alert (TTA) is the maximum time between the onset of a failure condition that affects the integrity of any information that could be applied by the airborne subsystem and the time that the alert indication is available at the output of the airborne subsystem, where the airborne subsystem is assumed to have zero latency. Compliance with the TTA requirement must include consideration of the probability of missed VDB messages by a fault-free airborne subsystem. CAT II/III Integrity Requirements and Antecedents

28. Actual “Hull Loss” Probability Breakdown (from October 1990 ICAO Study Data) • Total final approach and landing risk (as of 1990) = 7.8 × 10-7 per flight (~ 42% of total risk!) • Target level of safety (via “tunnel concept”) for final approach and landing = 0.2 × 10-7 per flight (~ 13% of total risk) • Hazard due to loss of navigation system integrity is only a small part of the total “final approach and landing” risk CAT II/III Integrity Requirements and Antecedents