NOAA’s National Weather Service Tools To Access the NDFD

1. NOAA’s National Weather ServiceTools To Access the NDFD Arthur TaylorRS Information Systems / Meteorological Development Laboratory National Digital Forecast Database Technical Workshop August 13, 2003

2. NDFD: How do I use it? • Since NDFD uses the WMO’s GRIB2 format, which is relatively new, NWS is providing a decoder library. The current versions of the library are located at: • http://www.nws.noaa.gov/mdl/iwt/grib2/decoder.htm • http://www.nws.noaa.gov/mdl/iwt/grib2/encoder.htm • In addition, NWS is providing a driver “degrib” (aka “NDFD_GRIB2Decoder”) for the library. Degrib’s primary purposes are: • Allow users to convert GRIB2 to other formats for use with various Geographic Information Systems (GIS), without need for a programmer. • Provide an example of how to call the decoder libraries • Enable some form of access to the data without requiring other packages • Generate Images • Probe a give lat / lon location for all the relevant data

3. Degrib: Installation • Degrib can be downloaded from: • http://www.nws.noaa.gov/mdl/NDFD_GRIB2Decoder/register.htm • On the download page, MS-Windows users should get “ndfd-demo.exe”, which contains: • An installation wizard • Compiled code • Source code for the libraries, and the “degrib” and “tkdegrib” programs (superImageGen and htmlmaker source code are not provided) • Documentation • Unix users should get “degrib-unix.tar.gz”, which contains: • Source code for the libraries and the “degrib” and “tkdegrib” programs • Documentation. • To compile the code you need: • FORTRAN (g77 should work) : for the decoder library • C (gcc should work) : for the “drivers” and projection library • Optional : Tcl/Tk (free from http://scriptics.com) : for the Graphical User Interface

4. Degrib: Data download • Step 1: Download some data. • The GUI version of degrib, “tkdegrib”, provides a convenient interface to get NDFD either via ftp or http. • Highlight the desired sector • Click on either “Download by ftp” or “Download by http” • Alternatively, you can use various UNIX tools to get the data directly from: • http://weather.noaa.gov/pub/SL.us008001/ST.expr/DF.gr2/DC.ndfd/ • ftp://tgftp.nws.noaa.gov/SL.us008001/ST.expr/DF.gr2/DC.ndfd/

5. Degrib: Convert • Currently degrib can convert to the following formats: • ESRI Point .shp files • ESRI Polygons .shp files • GrADS lat / lon grids • GrADS Projected grids • .flt files for use with ESRI Spatial Analyst • .flt files for use with other programs that can read NxM 4 byte floats. • In addition, degrib creates a .txt file which contains the meta data that was in the GRIB2 message.

6. Degrib: Convert to .shp file • To convert to .shp format: • Double click on the GRIB2 file (top pane) • Select message to convert (middle pane) • Choose an “output file name” • Click on “Generate .shp files” • “Polygon .shp” creates better images in ESRI and is easier to manipulate, but it is larger than the “Point .shp” • “Include Missing Values” allows you to study the entire grid, or limit the result to where data exists

7. CONUS MaxT in ArcView

8. Polygon vs Point, DC area

9. Degrib: Convert to .flt file • To convert to .flt file (for use with GrADS or ESRI Spatial Analyst): • Double click on the GRIB2 file (top pane) • Select message to convert (middle pane) • Choose an “output file name” • Click on “Generate .flt files” • “GrADS .ctl file”: creates a control file which GrADS can use to access the .flt file • “Interpolate”: bi-linear interpolates to a lat / lon coverage grid. • “M.S.B. first”: creates the .flt files in “Big Endian” format.

10. CONUS MaxT in GrADS

11. NDFD, GRIB2 & Weather • To encode Weather (Wx), NDFD disseminates “ugly strings”. • Advantage: A flexible format that can encode a description of weather such as “Chance of thunderstorms and chance of heavy rain showers” • Difficulty: An “ugly string” is not a number, so it is challenging to store in GRIB2. • Solution: Use GRIB2 section 2 to provide an ASCII look up table, and store the numeric values in the regular GRIB2 • Note: each weather grid, for each forecast projection, has a different ASCII look up table, so without section 2 it has no meaning. • Some Questions: • What does “Chc:T:<NoInten>:<NoVis>:^Chc:RW:+:<NoVis>:” mean, and how, for example, is an Emergency Manager supposed to know that? • What can be done for the .flt file?

12. Weather & .shp files • Degrib looks up the value, and parses it when it saves Wx to a .shp file • Wx-Index: Ugly string index used in the GRIB2 message. • Visibility: The lowest visibility in the 5 “ugly string words” (in statute miles). • NDFD wxCode: An encoding of the ugly string using the first wx-type, intensity, and coverage, and the second wx-type. • Weather 1: English translation of the “ugly word” • Wx-Inten 1: A number code for the wx-type and wx-intensity • Cover 1: A number code for the coverage • Hazard 1: A number code for hazards. • There can be 5 hazards per word, there are 9 types of hazards. So each hazard is given a digit, the digits are sorted. Result: “00045” = 45 for “Damaging Wind (4) and “Small Hail (5)”

13. Weather Codes: “Coverage” Table “Wx-Inten” Table “Hazards” Table

14. Example: Weather in ArcView • In “/degrib/ArcView/”, there is a weather.avl which was applied to the “Wx-Inten 1” to create this image. • Problem: Chance of thunderstorms (T), and severe T have the same color. • Intent: get people started. • If you have a better .avl, and want to share, let me know.

15. Weather & .flt “-SimpleWx”

16. Example: .flt “-SimpleWx”

17. NDFD CONUS Map Projection • For CONUS (continental U.S.), NDFD uses a Lambert Conformal Conic Map Projection: • Tangent latitutude 25, orientation longitude -95, mesh latitude 25, mesh size 5079.406 m. • In ESRI ArcMap that would be: • 1st Parallel = 2nd Parallel = 25 degrees north (tangent latitude) • Lon of center of projection = -95 degrees (orientation longitude) • Lat of center of projection = 25 degrees (tangent latitude) • False easting = 0 m, False northing = 0 m, X Shift = 0, Y Shift = 0 • In addition, NDFD uses a spherical earth with radius 6371.2 km • Note: 6367.47 km is currently in the GRIB2 messages, but NDFD used the same methods that NCEP used, which depended on the 6371.2 km sphere. • To assist ESRI ArcView 3.x, degrib creates a .ave (ArcView Avenue script) to set the projection and radius correctly.

18. Un-projected vs Projected

19. When Earth Radii go Bad. GOOD: Point used 6371.2 km Polygon used 6371.2 km BAD: Point used 6371.2 km Polygon used 6367.47 km

20. Degrib: Using superImageGen • To generate images using superImageGen • Select the “Download” tab • Highlight the sector in question • Download the data • Click on “Generate Images” • This results in the following: • Tkdegrib generates a set of “.mosaic” files • “superImageGen” generates the corresponding .png images • htmlmaker generates a set of web pages to browse the .png files • IE is called to view the web pages.

21. CONUS MaxT using superImageGen

22. Degrib: Probe point • This is for people who know their lat/lon and just want text output. To probe a point from a command prompt (not available yet in “tkdegrib”) : • C:\ndfd\degrib1.2\bin\degrib maxt.bin –P –pnt 38.99,-77.03 • C:\ndfd\degrib1.2\bin\degrib maxt.bin –P –pntFile point.txt • The first example probes just 1 point, while the second reads a file which is of the form: Station ID, lat, lon per line. The second form allows for numerous stations. • By default, probe chooses the “nearest” grid cell, but using the “-Interp” option, causes it to perform bi-linear interpolation. • Everything previously discussed can be done using degrib, except data download, so the degrib step can be automated. See (./degrib1.2/docs/degrib.txt for help)

23. Probe point (MaxT, MinT, PoP) element, unit, refTime, validTime, (38.993600,-77.022400) PoP12, [%], 200307212300, 200307220000, 39.000 PoP12, [%], 200307212300, 200307221200, 39.000 PoP12, [%], 200307212300, 200307230000, 50.000 PoP12, [%], 200307212300, 200307231200, 50.000 PoP12, [%], 200307212300, 200307240000, 50.000 PoP12, [%], 200307212300, 200307241200, 50.000 PoP12, [%], 200307212300, 200307250000, 20.000 PoP12, [%], 200307212300, 200307251200, 20.000 PoP12, [%], 200307212300, 200307260000, 14.000 PoP12, [%], 200307212300, 200307261200, 5.000 PoP12, [%], 200307212300, 200307270000, 5.000 PoP12, [%], 200307212300, 200307271200, 11.000 PoP12, [%], 200307212300, 200307280000, 11.000 PoP12, [%], 200307212300, 200307281200, 21.000 PoP12, [%], 200307212300, 200307290000, 21.000 • element, unit, refTime, validTime, (38.993600,-77.022400) • MaxT, [F], 200307212300, 200307230000, 86.882 • MaxT, [F], 200307212300, 200307240000, 82.375 • MaxT, [F], 200307212300, 200307250000, 85.233 • MaxT, [F], 200307212300, 200307260000, 87.753 • MaxT, [F], 200307212300, 200307270000, 89.395 • MaxT, [F], 200307212300, 200307280000, 87.775 • MaxT, [F], 200307212300, 200307290000, 86.800 • element, unit, refTime, validTime, (38.993600,-77.022400) • MinT, [F], 200307212300, 200307221200, 72.625 • MinT, [F], 200307212300, 200307231200, 73.165 • MinT, [F], 200307212300, 200307241200, 69.025 • MinT, [F], 200307212300, 200307251200, 65.965 • MinT, [F], 200307212300, 200307261200, 67.945 • MinT, [F], 200307212300, 200307271200, 69.745 • MinT, [F], 200307212300, 200307281200, 69.025

24. Degrib: Future? • Maintain the program by keeping up with NDFD as more variables / sectors are introduced. • In order to inform users of updates, degrib has an “announcement mailing list”, which currently has 119 people on it. • Improvements: Based on user feedback, current thoughts are: • Create merged polygons. Instead of a polygon per cell, join similar values together to form large polygons. • Create some form of graphical probe point capability • Create some form of un-compressed database so that a CGI callable probe program can be used to access the data quickly • Add some way for UNIX users (without GrADS) to plot the images • Add some way for users to zoom in on a map • Interface to more file formats