1 / 72

SHEF* Primer *Standard Hydrometeorological Exchange Format

SHEF* Primer *Standard Hydrometeorological Exchange Format. Mike Callahan WFO Louisville September 29, 2009. Objectives. Know the three different types of SHEF reports and when they are used Understand the various elements in a SHEF report

buffy
Download Presentation

SHEF* Primer *Standard Hydrometeorological Exchange Format

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. SHEF* Primer*Standard Hydrometeorological Exchange Format Mike Callahan WFO Louisville September 29, 2009

  2. Objectives • Know the three different types of SHEF reports and when they are used • Understand the various elements in a SHEF report • Understand the different descriptors in a SHEF data element • Know how to encode/decode SHEF reports

  3. The SHEF Manual • The SHEF Manual Version 2.2 (NWSM 10-944) is the ultimate source of all information about SHEF • Every office should have a copy of this manual • It is available online in PDF format at www.nws.noaa.gov/directives • This primer refers to it constantly, get it and be prepared to use it

  4. The Three Type of Reports • The .E format is used to encode an EVENLY spaced sequence of data from a SINGLE location • The .B format is used to encode a collective of SIMILAR data from a GROUP of locations • The .A format is used for everything else

  5. A Simple ExampleWe want to encode this data: • The river gage reading at Deputy IN was 13.45 at 7:00 AM EST on 4/1/2009 • Since we have a single piece of data for one location we will use the .A format. • We need: • The location ID: DEPI3 • The date and time of the data: 20090401 E DH0700 • The type and value of the data: HG 13.45

  6. .A Reports • .A reports are the simplest SHEF report • These reports can encode any type of data but may not be the most efficient choice • Use for a set of non-repeating or random sequence of data for a single location • Split into two sections, positional fields and a data string • Using the data from the previous slide, lets generate our first SHEF report: .A DEPI3 20090401 E DH07/HG 13.45 data string positional fields

  7. .A Positional Fields .A DEPI3 20090401 E Time Zone Date Location ID Format Specifier

  8. Format Specifier • Specifies the report type • Only choices are .A, .B, or .E • Optionally followed by an R for a revision or a number for a continuation • Data stored in the AWIPS database as missing will be overwritten regardless of the revision code • The revision code must be present to overwrite data non-missing already in the AWIPS database • When revising, make sure the date and time of the new data exactly matches the old data

  9. Location ID • An approved ID listed in the NWSLI database • It is possible to encode data from non-ID locations using stranger reports • Stranger reports use the latitude and longitude as an ID • Xlatlong: North latitude, West longitude • Ex. X4511001 = 45.1 N latitude, 100.1 W longitude • Ex. X3090988 = 30.9 N latitude, 98.8 W longitude • Be sure to include the leading zero for longitudes less than 100 degrees • Ylatlong: North latitude, East longitude • Ex. Y5030202 = 50.3 N latitude, 20.2 E longitude

  10. Date • See SHEF Manual, Section 4.1.4 • mmdd: Month and Day • If year is missing, the decoder will look 6 months in the future and 6 months in the past to assign the year nearest to the current date • Ex: 0401, 1225 • yymmdd: Year, month, and day • If a two digit year is used, the decoder will look 10 years in the future and 90 years in the past to assign the century • Ex: 090915, 990210 • yyyymmdd: 4 digit year, month, and day • Ex: 20091010, 19950415

  11. Time Zone • See SHEF manual, Table 8 • Common codes are: • Z: UTC (GMT) default • E: Eastern Local Time (follows shift between standard and daylight saving time) • C: Central Local Time • M: Mountain Local Time • P: Pacific Local Time • L: Alaskan Local Time • H: Hawaiian Local Time • Followed by a D to force daylight time or S to force standard time

  12. Quick Quiz: .A Positional FieldsGrab a pen and write down your answers • 1. List the elements of the positional fields • 2. What format specifier is used to correct a previous .A report • 3. Encode the date for July 4, 2009 • 4. Encode Mountain Daylight Time

  13. Answers to .A Positional Fields • 1. Format specifier, Location ID, Date, Time Zone • 2. .AR • 3. 20090704 or 090704 • 4. MD

  14. Data String DH07/HG 13.45 • All elements in the data string must be separated by slashes (/) expect for the parameter code and data pair Value Parameter Code Time

  15. Time • See SHEF Manual, Table 9 • Default is 12:00:00 for Z time zone, 00:00:00 for all others, missing time defaults are rarely used and could lead to some confusion • DHhhmmss = Data hour, minutes, seconds • mm and ssdefault to 00 • DDddhhmmss = Data day, hours, minutes, seconds • Use this if the day is not the same as the day in the positional code • DMmmddhhmmss = Data month, day, hours, minutes, second • DYyyyymmddhhmmss = Data year, month, day, hours, minutes, second

  16. Time Relative • DRt±xx • Adds or subtracts xx units of time t to/from the last explicit time • t values • S: seconds, N: minutes, H: hours, D: days, M: months, Y: years, E: end of month • Ex: • DRN+60 or DRH+1 = add an hour • DRD-1 = subtract one day

  17. Date Relative End of Month • DRE±x • Allows you to encode end of the month values without knowing how many days are in the month • x: the number of months • DRE+1 = the last day of the next month • DRE+2 = the last day of the next month • Ex: encode spring (Mar., Apr., May) monthly precipitation for ALTP1 • .A ALTP1 0331 E DH07/PPM 4.46/DRE+1 .A1 3.56/DRE+2/6.80

  18. Parameter Codes PEDTSEP • All codes except PE have a default Probability Extremum Source Type Duration Physical Element

  19. Physical Element • See SHEF Manual, Table 1 & Appendix G • First Letter Category, Second Letter Type, no default • Categories: • A-C: agricultural, E: evaporation, F: fish, • G: ground, H: height, I: ice, L: lake, • M: moisture & fire/fuel, N: dam & gate, • P: pressure & precipitation, Q: discharge, • R: radiation, S: snow, T: temperature, • U: wind, V: electrical, W: water quality, • X: weather, Y: other

  20. Physical Element (con’t) • Some common codes • HG: river stage height • HP: pool elevation • HT: tailwater elevation • PC: accumulated precipitation • PP: incremental precipitation • SD: snow depth • SF: snowfall • SW: snow water equivalent • QT: total discharge • TA: air temperature

  21. Special Physical Elements • Some physical elements are complex and have their own tables in the SHEF Manual • IR: ice report, Table 14 • SR: snow report, Table 15 • GR: frost report, Table 16 • PT: precipitation type, Table 17 • GS: ground state, Table 18 • HI: river trend, Table 19 • AF: surface frost intensity, Table 20 • AM: surface dew intensity, Table 21 • PM: precipitation probability, Table 22 • XW: present weather, Appendix C • XP: past weather, Appendix D

  22. Duration • See SHEF Manual, Table 3 • Some common codes: • I: instantaneous, default for most PE codes • D: 1 day, default for PP, SF, & others, see Table 7 • E: 5 minutes • G: 10 minutes • C: 15 minutes • H: 1 hour • T: 3 hours • Q: 6 hours • K: 12 hours • P: total since the previous 7AM • V: other, duration defined using DV code

  23. Special Durations • If the duration you want is not listed in Table 3, use the DVtxx code: • t values • S: seconds, N: minutes, H: hours, D: days, • M: months, Y: years • Z: reset duration back to default • Ex: • DVH2/SFV = 2 hour snowfall • DVN45/PPV = 45 minute precipitation

  24. Be Careful with P Durations • C DH07/PPP 0.35 • 24 hours of precipitation, 7:00AM yesterday to 7:00AM today • C DH06/PPP 0.50 • 23 hours of precipitation, 7:00AM yesterday to 6:00AM today • C DH0730/PPP 1.20 • 30 minutes of precipitation, 7:00AM today to 7:30AM today! • C DH0730/PPD 1.20 • 24 hours of precipitation, 7:30AM yesterday to 7:30AM today. • PPP is can be used for non-7AM observer reports if the observer dumps the gage at 7AM • Use PPD for morning reports

  25. Type • See SHEF Manual, Table 4 • R: Reading (observed), default • C: Contingency • F: Forecast • H: Historical • M: Model • P: Non-forecast Processed

  26. Source • Meaning changes with Type code, see SHEF Manual, Table 4 • Some common Type & Source code pairs • RZ: non-specified (observer) reading, default • RG: DCP reading • RP: phone reading (LARC, CR10, etc.) • RR: radio reading (ALERT, IFLOWS, etc.) • FZ: non-specified forecast, default for F type • FF: forecast including QPF

  27. Extremum • See SHEF Manual, Table 5 • Some common codes • Z: no extremum, default • N: minimum of day • H: minimum of 6 hours • P: minimum of 12 hours • X: maximum of day • R: maximum of 6 hours • Y: maximum of 12 hours

  28. Probability • See SHEF Manual, Table 6 • Some common codes • Z: no probability, default • A: .2% • 1: 10% • G: 25% • 3: 30% • 5: 50% • H: 75% • 8: 80% • Y: 99.8% • M: mean expected value

  29. Send Codes • Many parameter code combinations are so common that a new PE has been created • See SHEF Manual, Table 2 • Some common codes • TNTAIRZNZ = minimum air temperature of the day • TXTAIRZXZ = maximum air temperature of the day • HNHGIRZNZ = minimum gage height of the day • HXHGIRZXZ = maximum gage height of the day • PFPPTCFZZ = 3 hour flash flood guidance • SFSFDRZZZ = 24 hour snowfall

  30. Values • The data value • Trace amounts are encoded as T or t, stored as .001 • Missing data are encoded as M, m, MM, mm, +, -, or -9999 • For precipitation data: values missing a decimal will be divided by 100, thus PP 1 is stored as 0.01! This feature is obscure and may lead to confusion. • Data can be followed by a qualifier • See SHEF Manual, Table 10 • Some common codes • E:estimated, Q: questionable, F: flagged • R: rejected, M: manual edit, B: bad • Ex: PP 5.07Q

  31. Retained Comments • It is possible to attach comments to data values that will be stored • Follow the data value with the comment enclosed in single or double quotes • The comment should not exceed 80 character in length • Ex: we want to note that the temperature was from a backup thermometer • TA 88 "backup thermometer"

  32. Quick Quiz: .A Data Strings • 5. List the elements in data strings • 6. Encode 12:00 UTC • 7. Encode 12 hour new snowfall • 8. Encode an estimated river stage at 18:15 UTC of 3 feet

  33. Answers: .A Data Strings • 5. Time, parameter code, value • 6. Z DH1200 or Z DH12 or DH1200 or DH12 or nothing • 7. SFK • 8. DH1815/HG 3.0E

  34. Vector Physical Elements • Some Physical Elements have paired values, see SHEF Manual section 7.4.6 • Lets look at TV: temperature in vegetated soil at depth • The paired values are the depth and the temperature • The two values are encoded like this: dd.tttt, where dd is the depth in inches and tttt is the temperature in degrees and tenths in °F • Ex: • 12.0621 = 62.1 °F at 12 inches depth • 5.450 = 450.0 °F at 5 inches depth (meant 5.045) • -1.0022 = -2.2 °F at 1 inch depth

  35. Back to the Simple Example • The river gage reading at Deputy IN was 13.45 at 7:00 AM EST on 4/1/2005 • Data time: 20090401 E DH0700 • We could leave off the century or year in the date • We will use DH since the data day is the same as the positional field day • We could leave off the minutes • River gage reading is HGIRZZZ 13.45 • We could leave off IRZZZ since they are the default

  36. Putting the Report Together .A DEPI3 20090401 E DH07/HG 13.45 • The data elements are separated by a / • To continue the report on the next line use .An, you don’t need slashes at the end or beginning of a line • .A DEPI3 0401 E DH07/HG 13.45 .A1 PPD 0.15/TA 45/TX 60/TN 30 • To correct a bad report use .AR • .AR DEPI3 0401 E DH07/HG 12.45 • Each station is on a different line • .A ABEK2 0401 E DH07/PPD 0.25 • .A BNNK2 0401 E DH07/PPD 0.34

  37. Other Data String Elements • DCccyymmddhhnn:Creation Date • Defaults to same as observation time • Required for river forecasts • Similar to Date codes • DUa:Units Code • DUE = English units, default • DUS = Standard units • DQa:Qualifier Code • Same as data qualifier but applies qualifier to ALL data

  38. Internal Comments • Allowed anywhere in report • Colon (:) turns comments on • End of line or another colon turns comments off • SHEF decoder ignores them and they are not stored • Ex: • .A ABYV1 1123 DH11/PPD 0.23 : Albany VT • .A BELV1 : Belle VT : 1123 DH11/PPD 0.12

  39. Quick Quiz: .A Reports • 9. Encode data AAWC1 for 5/20 at 08:30 PDT: 24 hour pcpn 1.23, 24 hour max temp 70 °F, 24 hour min temp 50 °F • 10. At BLWW1, a DCP, the river crested on 7/4 at 34.5 feet at 1623Z, correcting an earlier incorrect crest of 34.2 feet • 11. On 6/1 at 1500 EDT, you get a questionable report of 5 inches of rain in the last 12 hours at latitude 40.3° and longitude 105.0°

  40. Quick Quiz: .A Reports (con’t) • 12. BNNM8, an ALERT gage, collects 3.45 inches over the last 5 hours ending 8/9 at 13Z. • 13. LARC river readings at 6Z on 10/12: BSNK2 5.67, CYCK2 12.34, FFTK2 10.78 • 14. Hourly river readings for PTTP1 on 8/9 starting at 18Z: 9.45, 9.47, 9.50, 9.53

  41. Answers: .A Reports • 9..A AAWC1 0520 PD DH0830/PP 1.23/TX 70 .A1 TN 50 • PP defaults to PPDRZZZ, TX defaults to TAIRZXZ, TN defaults to TAIRZNZ • 10. .AR BLWW1 0704 DH1623/HGIRGX 34.5 • Must use .AR to correct a bad report • Don’t need Z time zone since this is the default • Cannot use HX since the send code translates to HGIRZXZ • 11. .A X4031050 0601 ED DH15/PPK 5.0Q • Be sure to include the Q qualifier

  42. Answers: .A Reports (con’t) • 12. .A BNNM8 0809 DH13/DVH5/PPVRR 3.45 • Must use DVH05/PPVRR since there is no duration code for 5 hours • 13. .A BSNK2 1012 DH06/HGIRP 5.67 .A CYCK2 1012 DH06/HGIRP 12.34 .A FFTK2 1012 DH06/HGIRP 10.78 • There is a better way to encode this data • 14. .A PPTP1 0809 DH18/HG 9.45/DRH+1/HG 9.47 .A1 DRH+2/HG 9.50/DRH+3/HG 9.53 • Good use of the time relative code • There is also a better way to encode this data

  43. The .B Format • Problem 13 shows that the .A format can get redundant for collectives with the same data and time • This is why the .B format exists • The repeated information is gathered into a header and is separated from the data

  44. .B Reports .B LMK 1012 DH06/HGIRP BSNK2 5.67 CYCK2 12.34 FFTK2 10.78 .END Header • .B reports consist of three parts • Header, which describes the data • Body, which contains the data • Terminator, .END Body Terminator

  45. .B Header .B LMK 1012 DH06/HGIRP • Similar to the .A format • Positional fields are the same except that the location ID is replaced with the message source ID • The data string is replaced with the parameter control string positional fields parameter control string

  46. .B Parameter Control String • Similar to the .A data string without the data • Possible elements are the same as .A data string • DH: data time, DC: creation time • DU: units, DQ: qualifier, DV: duration • Parameter codes are listed together separated by slashes

  47. .B Body BSNK2 5.67 CYCK2 12.34 FFTK2 10.78 • Each line in the body: • Location ID • One or more spaces • The data for that location separated by slashes

  48. .B Terminator .END • Required after the body • Other SHEF reports may follow

  49. Another .B Example • Encode the following collection: Location Ztime Pcpn Max Temp Min Temp ATDK1 1200 1.23 78 45 BELK1 1200 M 67 43 CBDK1 1200 0.56 DSOK1 1100 1.56 75 40 ECLK1 1200 0.89 78 M • The data is similar but differs: • BELK1 pcpn report is missing • CBDK1 does not have a thermometer • DSOK1 time was at 11Z not 12Z • ECLK1 minimum temp is missing

  50. First: Create the Header .B TOP 0614 DH12/PPD/TX/TN Parameter Codes Time Date Message Source Format Specifier

More Related