Progress Toward The Extended Flight Level Dataset

1. Progress Toward The Extended Flight Level Dataset Jonathan L. Vigh National Center for Atmospheric Research 2:35 PM Thursday May 16, 2013 Fort Collins, CO Joint CSU/NOAA/NCAR Hurricane Workshop Funded by the DTC Visitor Program (FY2012): Development of an HWRF Diagnostics Module to Evaluate Intensity and Structure Using Synthetic Flight Paths Through Tropical Cyclones Collaborators: Eric Uhlhorn, Neal Dorst, Frank Marks, Hugh WIlloughby, LigiaBernardet, Vijay Tallapragada, ChanhKieu

2. Overview • The goal of this project is to make an apples-to-apples comparison between the simulated vortex structure in HWRF and direct in situ and remote sensing observations from aircraft. Figure 6 from Uhlhorn and Nolan (2011)

3. The resulting traces of surface and flight level wind speed in time, obtained by sampling the model storm along the flight paths shown to the left. Figure 8 from Uhlhorn and Nolan (2011)

4. Primary objectives: • Structure of the wind field at flight level • Structure of the wind field at the surface • Additional evaluations are also possible: • Extrapolated sea level pressure • Flight level temperature • Other kinematic and thermodynamic parameters (e.g. vorticity, theta-E, etc.)

5. Project Goals • Construct a library of all storms (2010 – 2012) that were observed by aircraft. • Both NOAA and Air Force Reserve • Develop technique to construct synthetic profiles through simulated storms along the observed flight paths • transform earth-relative observational data to frame moving with the storm center • navigate resulting flight pattern in storm-relative coordinates to the moving center of the simulated storm • sample through simulated storm along flight pattern • smooth resulting wind profile to the effective resolution of the 3-km HWRF • Evaluate model’s wind profiles with respect to the observed profiles • -> Diagnose model errors with goal of improving model

6. Background • Willoughby-Rahn flight level dataset (1977-2001) • Since ~1995-2001 the following have become prevalent • Microwave satellite data (~1998 – present) • GPS dropwindsondes (~1998 onward) • CIRA GOES IR satellite archive (1995-present) • SFMR (2005-present) • QuikSCAT (1999-2009) • Would be great to have the aircraft data readily available for the great storms of 2004/2005/2007.

7. Famous last words: “All I want are some radial profiles of tangential wind and temperature . . .” • Data issues • HRD raw flight level data come in variety of formats • Several USAFR ASCII formats (1-minute, 30-second, 10-sec, 1-sec) • Older data at 1-minute time resolution (prior to 1999 or so) on HRD web site – have to ask to get higher time resolution if available • standard tape format (binary) – used until about 2006/2007 • NOAA ASCII listings (1-sec and 10-sec) • Newer NOAA data in netCDF format with its own share of problems (no vetting of variables, variable names change from year to year and file to file – situation has improved since standardization efforts by AOC; major changes in 2011) • Raw flight level data are in earth-relative coordinates (Lat/Lon) • NOT translated to moving storm center • Winds not decomposed into tangential and radial components • No separation of “useful” radial legs from all the other stuff

8. Features of the Willoughby-Rahn dataset • Raw flight level data used to calculate dynamic center of storm – a track is produced and fit to these center using Ooyama’s beta splines • Willoughby, H.E., and M. B. Chelmow, 1982, "Objective determination of hurricane tracks from aircraft observations", Mon. Wea. Rev., 110, p.1298-1305. • Neal Dorst (HRD) generates these tracks. • Winds are translated to the moving storm center, decomposed into radial and tangential components

9. Willoughby and Chelmow 1982

10. Characteristics of W-R data set • The flight level data were parsed by hand into the “good” radial legs - other portions of flight discarded • Data are put into 300 overlapping radial bins using a linear distance weighting (Bartlett window). Weighting decreases linearly from 1.0 at the nominal bin radius to 0.0 at plus or minus the half bin width (DR). • Typical half bin width of 1.0 km with bins 0.5 km apart, so each data point is represented in 4 bins. Typical profiles go out to 150 km. • Legacy format is “ASCII ProFile” with accompanying metadata listed in a variety of other little ASCII files which serve as indices for navigating the data by flight and leg.

11. Epiphany • While these issues are not intractable, they present a high barrier to anyone who’d like to use the flight level data • To use a substantial amount of flight level data would require mastering the various not-so-nice raw data formats – not trivial • Getting data for many storms (for compositing, data assimilation studies, or research on wind profiles) requires an overwhelming data request to HRD – something they haven’t had the man-power for in the past (this aspect has improved since 2008) • Wind center finding too technical for the casual data user • Future users could be spared this major chore – hopefully spur much more usage of the flight level dataset • Solution – an (overly?) ambitious graduate student with a pressing need and a hankering for large coding projects + one gigantic Cloud Physics class project

12. Skisondes!

13. Birth of a “side” project • Extend the dataset to 1999-current storms • Challenge – design an automated algorithm to parse the radial profiles so that is no longer has to be done by hand • Initially preserve the methodology and functionality of the Willoughby-Rahn dataset (including the legacy output format – uggh!) • Eventually reprocess all storms (1977-current) with consistent methodology and improved output format • This will be version 1.1

14. Modern code and improved output format • Coding accomplished with NCAR Command Language (NCL) • Free, supported, open source • Improved, standardized time coordinate • Data processing and visualization tasks unified • Codes to read, manipulate, and plot dataset can be provided to dataset users • Extended dataset uses netCDF output format • Readable by Matlab, IDL, NCL, etc. • All metadata included in same file (no need for separate ASCII index files) • Flexible data structure – no rigid file formats

15. More incremental data processing • Several levels of data processing: • Level 0 – “native” raw data files (ASCII, non-QC’dnetCDF, standard tape format) for each flight • Level 1 – raw flight level data converted into a common netCDF format for the entire era (individual files by flight, one big file for each storm)– a format useful for data assimilation! • Level 2 – ALL processed flight level data translated to the moving storm center (netCDF) • Level 3 – Processed flight level data parsed into “good” radial legs (netCDF)

24. Progress Since 2008 • Dusted off the 6000-line code and brought up to modern NCL standards • Readers written for all AFRES data formats since 2005: • ARWO Software Version 15.400.21.4 (circa 2005) • ARWO Software Version 15.400.21.7 (circa 2006) • ARWO Software Version 15.400.22.2 (circa 2007 - 2008) • ARWO Software Version 15.400.23.0 (circa 2008 - used for WPAC) • ARWO Software Version 15.400.24.0 (circa 2009) • ARWO Software Version 15.400.25.0 (circa 2010-2011) • ARWO Software Version 16.0.1.3 (circa 2012) • Robust, flexible reader written for AOC data formats: • 2010 • 2011 • 2012 • Data processed into Level 1 common NetCDF file for each storm during 2010-2012 • New earth-relative plotter developed for visual QC • Code set now exceeds 10,000 lines

35. Example of parsing algorithm for one flightfrom Hurricane Ike (2008

36. -

39. Enhanced, extended dataset (v2.0) • Improved center-finding method (??) • Willoughby/Chelmow method is useful, but performance suffers from cases of strong eye convection, eye mesovortices • Improved radial binning method • Narrower frequency response • More consistent data structure • Don’t allow variable bin widths • Do allow radial legs longer than 150 km • Possibility of including SFMR • Could include aerosonde and other mobile platforms • Add dropsondes

40. Comments/Feedback