Beyond the TAF: User Benefits Driven Terminal Aviation Weather Services

1. Beyond the TAF: User Benefits Driven Terminal Aviation Weather Services Workshop on Nowcasting, Value Added Services for Air Navigation and User-oriented Aerodrome Forecasts WMO Aeronautical Meteorology Programme James Evans Weather Sensing Group MIT Lincoln Laboratory

2. Outline • Motivation for work • Conceptual framework for US aviation system operations • Decision Processes and Their Implications • Convective Weather • Ceiling/visibility at San Francisco • Measurement of changes in system performance • Summary

3. Motivation • Increased emphasis user benefits and metrics for system performance and investment decision making by sponsors and users FAA OPSNET delays Convective weather plus low ceilings/visibility are principal cause of delays in US system Terminal weather product developers need to consider fromtheoutset what will be necessary to obtain and demonstrate benefits

4. What is Generating the Delays: NAS Network Constraints Old paradigm: Insufficient adverse weather capacity at airports New paradigm: Storms impact the network by reducing the capacity of jet routes, en route sectors and terminals. Bad convective delay days invariably involve en route and terminal demand > effective capacity… ability to reroute is a key factor in delay magnitude 2 4 1 5 Airport 3

5. Weather Impact Mitigation Paradigm Operational Decision Loop CurrentWeather Determine weather impact Determine ATC impact Develop mitigation plans Decide on mitigation plan Execute mitigation plan • Surveillance • Systems • Weather Radar • Satellite • Obs User Displays ATC Decision Support Tools Forecast Products Benefits are achieved only when the operational decision loop is executed in a timely manner

6. Coordination for Decisions Terminal Davison and Hansman, ATC Quarterly 2002

7. Outline • Motivation for work • Conceptual framework for US aviation system operations in convective weather • Decision Processes and Their Implications • Convective Weather • Ceiling/visibility at San Francisco • Measurement of changes in system performance • Summary

8. # of aircraft Corridor Integrated Weather System (CIWS) Air Traffic 09/12/02 1000 UTC – 09/13/02 1000 UTC US ATC Chokepoints CIWS 2004 Domain From FAA 2001 ACE Plan User Identified Benefits • Higher airport departure rates • More efficient airport arrivals • Fewer miles-in-trail constraints • Fewer jet route closures • Identify opportunities to fly over storms • Reopening jet routes sooner • Improved transitions from en route to terminal airspace • Productivity enhancements

9. Terminal Weather Delays From Weather Not at the Airport METARs as an indication of weather: There was never a thunderstorm observation at ATL on this date 200 km from ATL TRACON boundary 12 August 2003 Delays for aircraft holding > 200 km from ATL were not captured by statistics

10. Lesson Learned in CIWS • Nature of the inter- and intra-facility interactions to arrive at and implement mitigation plans was not appreciated at the outset • Key personnel (supervisors of controllers) who traditionally are not thought of as traffic flow managers were not provided displays at many facilities • When controller supervisors were provided with displays, benefits at key facilities nearly doubled • Supervisors could take over some decisions and, were easier to convince on other decisions • Continuing effort to provide CIWS displays to • Key terminals • En route facilities that interface to the ATC facilities with CIWS

11. Outline • Motivation for work • Conceptual framework for US aviation system operations in convective weather • Decision Processes and Their Implications • Convective Weather • Ceiling/visibility at San Francisco • Measurement of changes in system performance • Summary

12. SFO Marine Stratus Forecast System Display

13. Reliability of High Probability San Francisco Forecasts: Summer 2003 13Z Forecasts 15Z Forecasts

14. Lessons Learned at San Francisco • Users (traffic flow managers) were reluctant to use forecasts to proactively end ground delay programs (GDP) • Current practice is to use higher than minimum capacity rates for time periods after predicted burn off time • Result continues to be unused capacity at end of capacity restricting events • Problem realized in retrospect • Previous analyses had focused on airline benefits (less delay) and possible costs (airborne holding if capacity constraint ended later than forecast time • FAA has a major concern about more planes holding than could be safely accommodated • Calculation of FAA risk requires very detailed statistics of duration of weather events that occur after forecast time

15. Outline • Motivation for work • Conceptual framework for US aviation system operations in convective weather • Decision Processes and Their Implications • Convective Weather • Ceiling/visibility at San Francisco • Measurement of changes in system performance • Summary

16. Key Elements of Benefits Analysis • Understand delay generation mechanism • Capability of candidate measurement tools • Observations of use of system by operational users • Delay statistics: FAA data bases (ASPM, OPSNET) • Aircraft data: ATC automation or TFM systems , plane reports • Weather data: METARs, Weather radar data, lightning • Develop and validate approach to making measurements on system to see if expected effect : reduction of the delays that would have occurred without the system under test is present and, estimate the magnitude of the delay reduction

17. 2003 CIWS Benefits Approach Facilities Visited ZBW ZAU ZOB ZNY C90 N90 ARTCC ZID ZDC TRACON SCC FedEx Airline ATCSCC Detailed Case Study Modeling 6 Multi-day “Blitz” observations of product usage Observations during “Blitz” Events Cases identified in Daily feedback # of improved ‘Annualized’ CIWS Delay Reduction Benefits ATC decisions Average Benefit Individual Delay Savings Events Frequency of convective weather at various ATC facilities (for each ATC decision)

18. Unquantifiable with 2003 Assessment Approach Annual benefit 28,383 hours delay reduction ($108.7 M) 3. 1289 Annual benefit 11,702 hours delay reduction ($43.8 M) 6. 953 756 749 439 311 276 196 194 183 157 110 69 52 14 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 CIWS Domain Delay Benefits in 2003 3314 3500 3000 2500 2000 1500 1000 500 0 Quantifiable Benefit Estimate of Annual Benefit Occurrences • Situational awareness • Inter facility coordination • Routes open longer • Improved Arrival Transition • Area (ATA) management • 5. Reduce workload 6 Proactive reroutes 7. Directing pathfinders 8. More SWAP departures 9. Shorter/fewer ground steps 10. Directing traffic through gaps 11. Improved safety 12. Close route proactively 13. Optimize runway usage 14. Avoid ground stop 15. Reduced MIT restrictions 16. Improved use of Ground Delay Program (GDP)

19. Summary • Adverse weather continues to dominate overall aviation safety and delays. Policy changes/economics make it critical to assess whether products have improved the situation • Direct observations of decision making at ATC facilities before and after a forecast is deployed can be exceptionally beneficial in • Refining the product (including its interface to decision support systems) • Developing training • Benefits assessment • Examination of flight tracks and radar data very helpful for convective weather benefits • Direct comparison of delays or accidents is a very challenging task that will require developing an in depth understanding of the aviation system