1 / 26

Interpreting TAF Verification Statistics: The Impact of TEMPO Forecasts (Corrected May 21, 2007)

Interpreting TAF Verification Statistics: The Impact of TEMPO Forecasts (Corrected May 21, 2007) Chuck Kluepfel National Weather Service Headquarters Silver Spring, Maryland 301-713-0090 x132 Charles.Kluepfel@noaa.gov. KABC 301728z 301818 24010KT P6SM SCT030

wood
Download Presentation

Interpreting TAF Verification Statistics: The Impact of TEMPO Forecasts (Corrected May 21, 2007)

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Interpreting TAF Verification Statistics: The Impact of TEMPO Forecasts (Corrected May 21, 2007) Chuck Kluepfel National Weather Service Headquarters Silver Spring, Maryland 301-713-0090 x132 Charles.Kluepfel@noaa.gov

  2. KABC 301728z 301818 24010KT P6SM SCT030 TEMPO 1820 1SM +TSRA BKN008CB FM2100 VRB05KT P6SM SKC= Question: If it only thunders for 15 minutes out of 2 hours (the length of my TEMPO group), will my False Alarm Ratio be 1 hour 45 minutes = 0.88 ? 2 hours

  3. KABC 301728z 301818 24010KT P6SM SCT030 TEMPO 1820 1SM +TSRA BKN008CB FM2100 VRB05KT P6SM SKC= Answer: Absolutely not! We do not compute False Alarm Ratios (FAR) from the TEMPO forecast, alone. TEMPOs are not intended to be applied to the entire TEMPO period. We only compute FARs for the Prevailing Forecast “Operational Impact” Forecast

  4. Prevailing Forecast KABC 301728z 301818 24010KT P6SM SCT030 TEMPO 1820 1SM +TSRA BKN008CB FM2100 22005KT P6SM SKC= Prevailing Forecast = No TS. Therefore, the prevailing forecast FAR is undefined. Reminder, FAR = False alarm time Forecast time

  5. Operational Impact Forecast KABC 301728z 301818 24010KT P6SM SCT030 TEMPO 1820 1SM +TSRA BKN008CB FM2100 VRB05KT P6SM SKC= What about the Operational Impact Forecast (OIF)? From 18-20z, two TAFs in effect. Which drives the OIF during this time period? Answer: It depends.

  6. KABC 301728z 301818 24010KT P6SM SCT030 • TEMPO 1820 1SM +TSRA BKN008CB • FM2100 VRB05KT P6SM SKC= • Apply each of these steps to every 5 minute interval of the TAF. • Step 1 Variability Test • Variability test --- 2 changes +/- 90 minutes • Reminder - TS is a binary element (yes or no) • Step 2 • Pass test – OIF is defined as the forecast equal to the observation (for binary elements). • Fail test – OIF is defined as the “worst” (most pessimistic) of the two forecasts.

  7. TAF Example 1 KABC 301728z 301818 24010KT P6SM SCT030 TEMPO 1820 1SM +TSRA BKN008CB FM2100 VRB05KT P6SM SKC= Observation: TS began 1907z TS ended 1921z

  8. Obs’n Prevail TEMPO Pass VAR Test? OIF 1805 No TS No TS TS Yes No TS . . . . . . 1905 No TS No TS TS Yes No TS 1910 TS No TS TS Yes TS 1915 TS No TS TS Yes TS 1920 TS No TS TS Yes TS 1925 No TS No TS TS Yes No TS 1930 No TS No TS TS Yes No TS . . . . . . 2000 No TS No TS TS Yes No TS TAF Example 1

  9. Example 1Operational Impact Forecast POD = 3/3 = 1.00 FAR = 0/3 = 0.00

  10. Example 1TEMPO Evaluation # Hours TEMPO 2 hours (24) Justified TEMPO 100% (24/24) Justified TEMPO Hit 12.5% (3/24) TEMPO S/B FM --- (0/0) TEMPO Benign --- (0/0) TEMPO Hurt --- (0/0)

  11. TAF Example 2 KABC 301728z 301818 24010KT P6SM SCT030 TEMPO 1820 1SM +TSRA BKN008CB FM2100 VRB05KT P6SM SKC= Observation: TS began 1806z TS ended 1821z

  12. Obs’n Prevail TEMPO Pass VAR Test? OIF 1805 No TS No TS TS Yes No TS 1810 TS No TS TS Yes TS 1815 TS No TS TS Yes TS 1820 TS No TS TS Yes TS 1825 NoTS No TS TS Yes No TS 1830 No TS No TS TS Yes No TS . . . . . . 1935 No Ts No TS TS Yes No TS 1940 No TS No TS TS No TS 1945 No TS No TS TS No TS 1950 No TS No TS TS No TS 2055 No TS No TS TS No TS 2100 No TS No TS TS No TS TAF Example 2

  13. Example 2Operational Impact Forecast POD = 3 / 3 = 1.00 FAR = 5 / 8 = 0.63

  14. Ex. 2 - What Went Wrong? Both PODs were great (1.0) Ex. 2 – much higher FAR (0.63 vs. zero) • Variability Test very Important • Failing Test means you will likely contribute heavily to FAR statistic. • Last 34 minutes of Example 2 failed this test! • Limit TEMPO valid times to when you are relatively sure that two or more changes will occur within 90 minutes. • Don’t be afraid to amend frequently!!

  15. Example 2TEMPO Evaluation # Hours TEMPO 2 hours Justified TEMPO 79.2% (19/24) Justified TEMPO Hit 15.8% (3/19) TEMPO S/B FM 0 TEMPO Benign 0 TEMPO Hurt 100% (5/5)

  16. Caveat The +/- 90 minute variability test that is performed at the end of each 5-minute period of the TAF does “look back” to the times that preceded the valid period of the TEMPO group, but it does not “look back” to times that preceded the valid period of the TAF.

  17. Extending Operational Impact Rules to Other Elements Variability Test (Step 1) Look for two or more changes +/- 90 minutes. For ceiling and visibility, the variability test looks for category changes. For wind direction, wind speed, and gusts, the variability test uses threshold values to define change. Sig Wx Type – Begin or end.

  18. All changes are defined categorically: > 3000 feet 2000-3000 feet 1000-1900 feet 500-900 feet 200-400 feet < 200 feet Ceiling Categories

  19. All changes are defined categorically: > 5 statute miles 3 to 5 miles 2 to < 3 miles 1 to < 2 miles ½ to < 1 mile < ½ mile Visibility Categories

  20. Wind Direction: Change >= 40 degrees Wind Speed: Change >= 8 knots (+/-) Wind Gusts: Change >= 10 knots (+/-) Wind Changes

  21. Extending Operational Impact Rules to Other Elements Step 2 Var. Test Passes => OIF is forecast w/ smallest error Ceiling / visibility (categorical error) Wind (actual error in degrees, knots) Tie: OIF = Prevailing Forecast Var. Test Fails => OIF is the forecast w/ worst weather Highest wind speed / wind gusts Lowest ceiling / visibility Wind direction – OIF is undefined.

  22. Returning to TAF Example 2 KABC 301728z 301818 24010KT P6SM SCT030 TEMPO 1820 1SM +TSRA BKN008CB FM2100 VRB05KT P6SM SKC= Observation: TS began 1806z TS ended 1821z

  23. Obs’n Prevail TEMPO Pass VAR Test? OIF 1805 No TS No TS TS Yes No TS 1810 TS No TS TS Yes TS 1815 TS No TS TS Yes TS 1820 TS No TS TS Yes TS 1825 NoTS No TS TS Yes No TS 1830 No TS No TS TS Yes No TS . . . . . . 1935 No Ts No TS TS Yes No TS 1940 No TS No TS TS No TS 1945 No TS No TS TS No TS 1950 No TS No TS TS No TS 2055 No TS No TS TS No TS 2100 No TS No TS TS No TS TAF Example 2Thunderstorm Database

  24. TAF Ex 2: Ceiling Database Obs’n Prevail TEMPO Pass VAR Test? OIF 1805 None (6) None (6) 800 (3) Yes None (6) 1810 600 (3) None (6) 800 (3) Yes 800 (3) 1815 600 (3) None (6) 800 (3) Yes 800 (3) 1820 1500 (4) None (6) 800 (3) Yes 800 (3) 1825 None (6) None (6) 800 (3) Yes None (6) … … 1935 None (6) None (6) 800 (3) Yes None (6) 1940 None (6) None (6) 800 (3) No 800 (3) 1945 None (6) None (6) 800 (3) No 800 (3) 1950 None (6) None (6) 800 (3) No 800 (3) 1955 None (6) None (6) 800 (3) No 800 (3) 2000 None (6) None (6) 800 (3) No 800 (3)

  25. F O R E C A S T (1) (2) (3) (4) (5) (6) (1) O (2) B S (3) 2 E R (4)1 V E (5) D (6)5 16 TAF Ex 2: OIF Results

  26. Example 2TEMPO Evaluation (Ceiling) # Hours TEMPO 2 hours (24) Justified TEMPO 79.2% (19/24) Justified TEMPO Hit 10.5% (2/19) Just TEMPO Imp TAF 5.3% (1/19) TEMPO S/B FM 0 TEMPO Benign 0 TEMPO Hurt 100% (5/5)

More Related