The Multiple Uses of a Comprehensive Climatology Database

1. The Multiple Uses of a Comprehensive Climatology Database John P. Gagan, Steve A. Listemaa and Eric E. Carpenter NWS WFO Jackson, MS And Jennifer L. Colson NWS WFO Tampa Bay, FL

2. The Purpose of this Presentation • List useful climate information readily available to all National Weather Service Forecast Offices (NWS WFOs) • Identify methods of data collection • Compare methods of storing and accessing the data • Examine examples of how these databases can improve forecasts and services

3. Why take the time and effort to create these databases? • The NWS is mandated to protect life and property • In order to protect life and property, the NWS forecaster is responsible for forecasting extreme (anomalous) events as far in advance as possible • These databases provide a comprehensive knowledge of a CWA’s weather history and offer an explicit investigation into what can occur

4. Climatological Data • A portion of the vast NCDC archive includes hourly and dailysurface and upper air data for thousands of stations across the United States for a variety of time spans • This large data set serves as a footprint for each NWS WFO county warning area (CWA) and is an essential resource for all forecasters

5. Climatological Data (con’t) • Available Surface Data: • Hourly -- From station inception through the present • Wind Speed, Direction and Gusts • Visibility • Temperature • Dewpoint • Ceiling Height and Coverage • Pressure • Rainfall

6. Climatological Data (con’t) • Available Surface Data (con’t): • Daily (Varying Time Spans) • High/Low Temperature • 24-hour Rainfall • 24-hour Snowfall • Snow Depth • Upper Air • Twice Daily, with intermediates

7. Data Acquisition • Climatological data sets available through the National Climatic Data Center • www.ncdc.noaa.gov • Most data available on CDROMs or web-based queries • Many of these data sets available in digital ASCII format • Lists of station histories provided via ftp (also in Adobe Acrobat, Word or Text files)

8. Data Acquisition (con’t) • Available data for the Jackson, MS CWA • Hundreds of daily coop reports, with histories dating back as far as 1890s • Eleven hourly surface observation sites with histories as far back as 1901 (most back to the 1970s) • All available North American RAOB data from 1946 onward (best coverage from 1957 through the present)

9. Data Manipulation • Files acquired through CDROM and download were formatted to comma delimited (if not already in this format) • Upper air mandatory-level data from the KJAN site extracted using GEMPAK • Comma delimited data saved as text files

10. Data Manipulation (con’t) • Data delimited by Year, Month, Day and Hour (when applicable) • Once all the acquired files have been saved as text files they are imported to a MySQL database • Databases are created for each data type

11. KJAN Daily Database Query – The “Old” Database • Initially, Microsoft Access was used as it allowed for easy “access” to all data • Queries were created to search for any year, month, date, parameter, or combination therein • After testing the database, we found that the Access GUI was user-unfriendly • A simple, web-based version received a much better response from users

12. KJAN Daily Database Query – The “Old” Database Example How many times has KJAN observed low temperatures at or below 32 F in October?

13. KJAN Daily Database Query Example • Between 1930 and 2002 (2232 observation records) a temperature of 32 F or lower was observed only 30 times in October • Obviously a 98th percentile event!

14. The “New” KJAN Daily Database • COOP Search • Search for Tmax, Tmin, and Precip for a station

15. The “New” KJAN Daily Database • Web Table output • Simple • Basic Info Fast

16. The “New” KJAN Daily Database • Text output • Save as a .txt • Easily imported into a spreadsheet • Quick analysis

17. The “New” KJAN Daily Database • METAR Search • Example: just temperature

18. The “New” KJAN Daily Database

19. The “New” KJAN Daily Database Graphing Feature

20. The “New” KJAN Daily Database

21. Special Queries • Show all observations with Temps < 15F in the year 1982 • This is just a partial listing

22. KJAN Daily Database Query –Example How many times has KJAN observed low temperatures at or below 32 F in October?

23. KJAN Daily Database Query Example • Between 1930 and 2002 (2232 observation records) a temperature of 32 F or lower was observed only 30 times in October • Obviously a 98th percentile event!

24. KJAN Upper Air Database • All available North American raob data (TTAA/TTBB format) available from NCDC converted to GEMPAK • Each sounding viewable through NSHARP • Mandatory level data extracted and put into text files • Text data then imported to Microsoft Access

25. KJAN Upper Air Database • Sounding coverage available in the database

26. KJAN Upper Air Database 0000 UTC 22 February 1971 KJAN Sounding – Delta Outbreak

27. KJAN Upper Air Database – Text Data • Mandatory Text data includes: • Temperature • Dewpoint • Heights • Wind Direction/Speed

28. KJAN Upper Air Database – Text Data (con’t)

29. KJAN Upper Air Database – Text Data example • How many times has the 850hpa temperature been observed at -10C or lower in January? 16 times out of ~1300 observations

30. Uses of the Databases • All databases can be linked to one another (Hourly to Daily to Upper Air) • Break down any data set for any period of time, for any parameter • Statistical analysis of parameters pertinent to general forecasting, aviation, fire weather, etc. • Verify historical occurrences of special weather criteria such as wind advisories, wind chill, heat indices, Red Flag, etc.

31. Uses of the Databases (con’t) • Sounding data can be used to create composites of conditions that have led to heavy rainfall, severe weather and winter weather in your particular area • Inclusion of teleconnection data from the CPC allows a user to define average temps during each ENSO phase, PDO phase, etc. • The user is bound only by their imagination • A few examples…

32. Uses of the Databases (con’t) • Break down climatological hourly temperature and dewpoint curves to be used in the WFO’s Graphical Forecast Editor (GFE) • Further analysis can break down effects of cloud cover and wind direction on the hourly temperature and dewpoint curves • These curves can significantly increase accuracy of point forecast matrices (PFMs)

33. Hourly Temperature Curves

34. Uses of the Databases (con’t) • Hourly Data • Climo temperature/dewpoint curves • Historical analysis of wind speed and direction (Homeland Security issues) • Typical Ceilings and Visibilities associated with particular weather phenomena • Analysis of wind chill/heat index values

35. Uses of the Databases (con’t) • Daily data • Define “average” temperature and “normal” temperatures for a given time period • Create box and whiskers diagrams to visually separate the 50th percentile from truly anomalous temperatures

36. Uses of the Databases (con’t) Craven 2003

37. Uses of the Databases (con’t) Late October KJAN stats

38. Uses of the Databases (con’t) • Excellent source for investigating significant weather events • Linking Daily data with text Upper Air data gives a 3-D view of anomalously hot/cold air masses • Can take model and ensemble consensus in the extended forecast period to explicitly quantify what an air mass is capable of • Quite powerful when used in tandem with NCEP/NCAR reanalysis data

39. Uses of the Databases (con’t) • NCEP/NCAR Reanalysis data available at the Climate Diagnostics Center • http://www.cdc.noaa.gov/cdc/reanalysis/ • Visualization of surface and upper air parameters for any date back to 1948 • GRADS image creation allows user to define viewing area • Highly recommend this site

40. Uses of the Databases (con’t) • Let’s briefly revisit our upper air database example • How many times has the 850hpa temperature been observed at -10C or lower in January? 16 times out of ~1300 observations

41. Uses of the Databases (con’t) • Where do the associated daily high and low temperatures fit in with respect to normal?

42. Uses of the Databases (con’t) • Using the Daily database, a distribution of all high and low temperature recorded in January can be exported to Microsoft Excel • Finding the max, min, median, 25th percentile and 75th percentile values for both the high and low temperature in January, a box and whiskers diagram can be created

43. Uses of the Databases (con’t) January 40 33.3 27 24 16 2 HIGH LOW

44. Uses of the Databases (con’t) • Although not surprising by any means, a forecaster can quantify that 850hpa temperatures of  -10 C lead to surface high and low temperatures in the lowest quartile of the temperature distribution for January • Statistics just like these are currently being developed at KJAN and are inserted into a binder next to the synoptic forecaster’s workstation

45. Benefits of the Database • Increased knowledge of your forecast area of concern • Statistics can easily be generated for any station, parameter, and time period • Generate smart tools for use in IFPS (GFE) • Increased situational awareness of anomalous air masses/events in the extended forecast period • Enhances in-depth studies of any weather phenomena that affects your region

46. Future Endeavors • Complete the “Climate Database Retrieval System” for each WFO within Southern Region • Each office will have quick access to: • Hourly observations and COOP observations for your area • ALL raobs from North America • This data will be queried through a simple web-based platform

47. Future Endeavors • Acquisition of hourly coop rainfall data • In depth studies of heavy rainfall events by season • Define what “heavy rainfall” means for an area • Acquisition of snow cover data • Contour all upper level height data using GEMPAK for quick reference

48. MANY THANKS! • Eric, Jennifer, Steve and I would like to thank Alan Gerard (MIC) and Jeff Craven (SOO) for allowing us to tackle this large project • My wife, Jennifer, for showing me the finer points of Microsoft Access • Also, the CDC, CPC and NCDC for their monumental efforts in collecting and creating quality data sets and web generated products

49. Questions, comments? Contact myself or Eric at: John.Gagan@noaa.gov Eric.Carpenter@noaa.gov