Commercial Aviation Safety Team (CAST) Process Overview

Commercial Aviation Safety Team (CAST) Process Overview

In the U.S., our focus is set by theWhite House Commission on Aviation Safety 1.1 Government and industry should establish a national goal to reduce the aviation fatal accident rate by a factor of five within ten years and conduct safety research to support that goal. 1.2 The FAA should develop standards for continuous safety improvement, and should target its regulatory resources based on performance against those standards 5.3-2

The National Civil Aviation Review Commission (NCARC) on Aviation Safety Provided Additional Direction • FAA and the aviation industry must develop a strategic plan to improve safety, with specific priorities based on objective, quantitative analysis of safety information and data. • Government should expand on their programs to improve aviation safety in other parts of the world. 5.3-3

In Response • Ongoing Industry and FAA Safer Skies Initiatives were Combined into CAST • Data-Driven, Consensus-Based, Integrated Strategic Safety Plan Developed • In place and fully supported by Government and Industry with Worldwide Recognition - “CAST”

We Need to Continuously ImproveAviation Safety 50 45 40 35 30 25 20 15 10 5 0 Hull loss accidentrate 1965 1975 1985 1995 2005 2015 Year

Airplanes in service 28,550 12,595 Boeing 11,300 1997 2019 We Need to Continuously ImproveAviation Safety 50 45 40 35 30 25 20 15 10 5 0 Millions of departures Hull loss accident rate 1965 1975 1985 1995 2005 2015 Year

Our goal We Need to Continuously ImproveAviation Safety 50 45 40 35 30 25 20 15 10 5 0 Airplanes in service 28,550 12,595 15,800 Business as usual Boeing 11,300 Hull loss accidentsper year 2000 2019 1997 Millions of departures Hull loss accident rate 1965 1975 1985 1995 2005 2015 Year

Regional PerspectiveAccident Rates Vary by Region of the World Western-built transport hull loss accidents, by airline domicile, 1992 through 2001 C.I.S.1 Europe 0.9 JAA - 0.7 Non JAA – 3.0 China 1.1 United States and Canada 0.4 Middle East 3.4 Asia 2.6 (Excluding China) Africa 12.4 Latin America and Caribbean 3.1 Oceania 0.0 Accidents per million departures World 1.3 1Insufficient fleet experience to generate reliable rate. 3/19/02 REG-053a

Safety Responsibilities Are Shared Air Safety Manufacturers Government Operators Safe Airplane + Safe Operation + Safe Infrastructure = Safe Air Travel

Safety Analysis Process 1. NTSB Accident Incident Reports 2. 21.3 Reports NASDAC data Airclaims data Historical Data FOQAdata Pareto Plots JSAT Industry JSAT 5.3-23

Safety Analysis Process 1. NTSB Accident Incident Reports 2. 21.3 Reports NASDAC data Airclaims data Historical Data FOQAdata Pareto Plots JSAT Industry JSAT 3. Accident Threat 4. Combined Threat Causal Analysis Intervention Strategy Cause Cause Cause Cause JSAT JSAT 5.3-23

Safety Analysis Process 1. NTSB Accident Incident Reports 2. 21.3 Reports NASDAC data Airclaims data Historical Data FOQAdata Pareto Plots JSAT Industry JSAT 3. Accident Threat 4. Combined Threat Causal Analysis Intervention Strategy Cause Cause Cause Cause 5. JSAT JSAT 7. Industry Safer Skies 6. Government Implementation Strategy JSIT AvSP Measuring Progress to Goal Coordinated Plan 5.3-23

Excluding all security events

Safer Skies COMMERCIAL AVIATION Controlled Flight GENERAL AVIATION Into Terrain CABIN SAFETY Aeronautical Loss of Control Decisionmaking Passenger Interference Loss of Control Uncontained Engine Failures Weather Passenger Seat Belt Use Runway Incursion Controlled Flight Carry-on Baggage Into Terrain Approach and Landing Child Restraint Survivability Weather Turbulence Runway Incursions HUMAN FACTORS IMPROVED DATA IN OPERATIONS & & ANALYSIS MAINTENANCE

Commercial Aviation Safety Team (CAST) Industry Government AIA Airbus ALPA APA ATA NACA Boeing P&W* RAA FSF IATA AAPA ATAC APFA • DOD • FAA • Aircraft Certification • Flight Standards • System Safety • Air Traffic Operations • Research • NASA • ICAO • JAA • TCC • NATCA Commercial Aviation Safety Team (CAST) *Representing GE and RR

CAST Goals • Reduce the U.S. commercial aviation fatal accident rate by 80% by 2007 • Work together with airlines, JAA, ICAO, IATA, FSF, IFALPA, other international organizations and appropriate regulatory/ government authorities to reduce worldwide commercial aviation fatal accident rate

Commercial Aviation Safety Team (CAST) CAST • Data analysis Joint Safety Analysis Teams (JSAT) • Safety enhancement development Joint Safety Implementation Teams (JSIT) • Master safety plan • Enhancement effectiveness • Future areas of study Joint Implementation Measurement Data Analysis Team (JIMDAT)

CASTA Three-Stage Process Data Analysis Implement Safety Enhancements - U.S. Set Safety Priorities Agree on problems and interventions Influence Safety Enhancements - Worldwide Achieve consensus on priorities Integrate into existing work and distribute

Joint Safety Analysis Team (JSAT) Process

ALPA/APA FAA (AIR,AFS, ASY, AAI) Airbus JAA ATA Transport Canada NASA Engine companies – (PW, GE, RR-Allison) Boeing RAA Air Traffic Services NATCA Typical JSAT Membership

JSAT Process Charter Development Establish Team Select Data Set Review Data Assign Standard Problem Statements Evaluate Problem Importance Develop Event Sequence Record Characteristics/ Indicators Identify Problems (what/why) Global Review of Characteristics/ Indicators Technical Review & Report Results Identify Intervention Strategies Evaluate Intervention Effectiveness Prioritize Interventions 5.5-24

Developed Event Sequence • Facts and data • pilot - controller voice events • missed calls • events that occurred or should have • Time coded each event

Develop Problem Statements • Problem statements • What went wrong • Deficiency definition • Potential reason • Something which happened or didn’t happen

Sample Standard Problem Statements • 10 FLIGHTCREW – Failure of flight crew to follow established procedures (SOP) • 39 AIRCRAFT EQUIPMENT – DESIGN NOT ERROR TOLERANT System design does not provide adequate redundancy to counteract errors or alerting of the effects of errors. • 44 FLIGHTCREW – Flight crew failure to recognize and correct unstable approach. • 100 REGULATORS – INSUFFICIENT AIR CARRIER OVERSIGHT . Insufficient regulatory oversight of air carrier operations including management and training practices.

Scoring - Problem Importance • POWER – P1 • Importance of the problem or contributing factor in the cause of the accident • FUTURE GLOBAL APPLICABILITY • Frequency of the problem or contributing factor (to this accident type) in future operations on a worldwide basis

RATING SCALES – Problem Statements POWER – P1 This scale is to be used to judge the importance of a specific specific problem or contributing factor in the cause of the accident. 0 1 3 2 6 5 4 Highly important Completely important Slightly important Quite important No importance Little Importance Moderately important FUTURE GLOBAL APPLICABILITY This scale is to be used to estimate the frequency of the problem or contributing factor in future operations on a worldwide basis (for example: how often the situation occurs in accident scenarios; its impact is on present and future operations (equippage, traffic, regulatory differences); and whether it is applicable across airlines/airplanes/regions. 0 1 3 2 6 5 4 Completely applicable Hardly any applicable Slightly applicable Moderately applicable Highly applicable Quite applicable Not at all applicable

Identify Intervention Strategies • Intervention strategies • Suggested solutions • Things to do to prevent or mitigate the problem • Etc.

Scoring - Intervention Effectiveness • POWER – P2 • Ability of the intervention to mitigate the problem or contributing factor (in a “Perfect World”) • CONFIDENCE • Confidence that the intervention will have the desired effect

RATING SCALES – Interventions POWER – P2 This scale is to be used to judge the effectiveness of a specific intervention in reducing the likelihood that a specific accident would have occurred had the intervention been in place and operating as intended. (“perfect world”) 0 1 3 2 6 5 4 Highly effective Completely effective Not at all effective Slightly effective Quite effective Hardly any effect Moderately effective CONFIDENCE This scale is to be used to define the level of confidence that you have that this specific intervention will have the desired effect. 0 1 3 2 6 5 4 Completely confident Slightly confident Moderately confident Hardly any confidence Highly confident Quite confident Not at all confident

JSAT Reports • Standard Problem Statements • Interventions Prioritized • Recommendations

JSIT Feasibility Scales • Technical • Financial • Operational • Schedule • Regulatory • Sociological

JSIT Safety Enhancements • Develop Safety Enhancements from Interventions • Collect detailed resource information • Prepare Detailed Implementation Plans (DIP’s)

Joint Implementation Measurement Data Analysis Team (JIMDAT) Process

Developed a Prioritization Methodology (JIMDAT) • Identified the most effective solutions derived from all accident categories • Considered effectiveness vs. resources • Tested solutions against all fatal and hull loss accidents, U.S. domestic Part 121 Operations, 1987-2001 • Created draft Master strategic safety plan • Identified areas for future study/mitigation

JIMDAT Process • Provides consistent estimates of the accident prevention potential of safety enhancements • Accounts for the benefit of a single intervention or a combined group of interventions • Addresses overlap with other interventions/technologies • Preserves analysis criteria and results

Using JIMDAT for CAST Plan Development • Risk reduction is primary measure • Identified effectiveness and associated resource requirements for each enhancement • Used for initial differentiation (2007 and 2020)

Integrated Strategic Safety Plan • Contains 46 safety enhancements optimized to include those actions with the best effectiveness vs. resource relationships • 22 complete/24 committed and underway • Initially combines short-term “liveware”-based enhancements with transition to design change enhancements long term • Projected 73% risk reduction by 2007 • Foundation for U.S.-driven continuous improvements in worldwide aviation safety

Sharpening Our Focus • All safety enhancements are valuable • Some safety enhancements are more effective than others across the various accident categories • Implementation reality demands better prioritization

Output Evaluation Spreadsheet Example

Resource Cost vs. Risk Reduction 100% 10000 APPROVED PLAN Risk Reduction 9000 Total Cost in $ (Millions) 8000 75% 7000 6000 Risk Eliminated by Safety Enhancements 50% 5000 Resource Cost ($ Millions) 2007 2020 4000 3000 25% 2000 1000 0% 0 All JSIT Proposed Enhancements (2020 Implementation Level) Completed Completed + Plan (2020 Implementation Level) Completed + Plan (2007 Implementation Level)

Safety Plan Benefits • Prediction of a 73% risk reduction that also results in approximately $620 million annual savings to the industry • Current accident cost per flight is approximately $76 cycle • Implementation of the 46 selected safety enhancements reduces this cost by $56 per flight cycle Safety is morally required and is also good for business

Cost Savings Part 121 Aviation Industry Cost Due to Fatal/Hull Loss Accidents 100 Historical cost of accidents per flight cycle 80 Savings ~ $56/Flight Cycle Or ~ $620 Million Dollars/Year Dollars/Flt. Cyc 60 73% Risk reduction 40 20 Cost of accident fatalities following implementation of the CAST plan @ 2007 levels 0 2002 2007

U.S. Hull Loss & Fatal Accidents Portion of Total Fatality Risk Mitigated by the CAST Plan (2007 Implementation Values) 100% Risk Eliminated 90% Risk Remaining 80% 70% CAST 1987-2000 Fatal/Hull Loss Database – Security events excluded 60% Portion of Risk 50% 40% 30% 20% 10% 0% Total CFIT Evac Midair LOC Flt Crew Inc LOC GND Sys-Comp Eng-UCEF Turbulence Fire/Explosion Runway Collision

All Regions Combined Worldwide Hull Loss & Fatal World Wide Accidents Portion of Fatality Risk Mitigated If CAST Plan is Adopted Worldwide (2007 Implementation Values) 100% Risk Eliminated 90% Risk Remaining 80% 70% 60% Portion of Risk 50% 40% 30% 20% 10% 0% RI RE ICE ARC Total CFIT ADRM FUEL LOC-I USOS Midair FIRE-NI SCF-PP SCF-NP WSTRW OTHER-BIRD Accident Classes Defined by CAST/ICAO Common Taxonomy

CASTA Three-Stage Process Data Analysis Implement Safety Enhancements - U.S. Set Safety Priorities Agree on problems and interventions Influence Safety Enhancements - Worldwide Achieve consensus on priorities Integrate into existing work and distribute

Joint Implementation Measurement Team (JIMT) Process

Reporting Standards Progressing on plan Plan developed but not progressing - may not make plan or no plan exists Late or critically off plan On plan but effected by late or critically off plan Output in plan completed Status of non-aligned carriers (color same as above) 5-30-00 CAST-3-05

Commercial Aviation Safety Team (CAST) Process Overview