Model Flip-Flops and Forecast Opportunities

1. Model Flip-Flops and Forecast Opportunities Bernard N. Meisner Scientific Services Division NWS Southern Region Fort Worth, Texas The views expressed herein are those of the author and do not necessarily reflect the position of the National Weather Service. National Weather Service

2. Model Flip Flops • Primary Motivation: • Introduction of 1200 UTC MEXMOS guidance caused concerns among some forecasters. • Perception was that guidance from 1200 UTC model runs would be substantially different from the0000 UTC guidance.(Flips and Flops) • Secondary Motivation: • Given substantial improvements in MOS guidance, are there still opportunities for forecasters to add value, particularly beyond Days 1 and 2?(Forecast Opportunties)

3. Model Flips and FlopsHow to define them? Run-to-run change in MOS max/min temperatures >10oF. Current model run lies outside envelope of previous ensemble. Run-to-run change in MOS max/min temperatures exceeds monthly Mean Absolute Error.

4. 108 Hr Forecast 102 Hr Forecast 96 Hr Forecast Verification

5. Model Flip Flops • Data Used: • MEXMOS Guidance for DFW and PIT Airports. • (Have also looked at a few other sites in NWS Southern Region.) • Flip (and Flop) Criterion: • Run-to-run change in Max/Min temperature guidance greater than 10oF.

6. Flip-Flop Spreadsheet Run-to-run change exceeds this criterion Model “Flips” All these flips were in the proper direction

7. Flip-Flop Spreadsheet Run to run change exceeds this criterion for two consecutive model runs Model “Flip/Flop” This flop was in the wrong direction.

8. 12 00 00 12 00 12 12 00 00 00 12 12 12 00 00 No discernable difference between 0000 and 1200 runs

9. October 2005 November 2005 Mean Absolute Guidance Errors - KDFW December 2005 January 2006

10. DFW Max/Min Temperature10oF Threshold

11. Model Flips…and Flops • Summary: • No substantial difference between 0000 and 1200 UTC model guidance. • Perception of systematic run-to-run changes is not valid. • Flips (and flops) occur during the cool season. • Typically occur for just one verification time. • Model flips are rare; flip-flops are very rare. • 53 Flips; 11 Flops (out of 400+ model runs/7000+ forecasts) • Flips are most common for Days 4-6. • Flip-flops are most common for Days 5-6.

12. Forecast Opportunities National Weather Service

13. Forecast Opportunities • Data Used: • MEXMOS Guidance for DFW and PIT Airports. • (Have also looked at a few other sites in NWS Southern Region.) • Forecast Opportunity Criterion: • Max/Min temperature guidance error greater than 10oF. • Caveat: • Smaller errors at certain thresholds can be significant! • 28oF vs 33oF; 99oF vs 104oF

14. Forecast Opportunities Spreadsheet Observed temperature cells turn pink when they exceed climate normal by this amount Performance of National Weather Service Forecasts Compared to Operational, Consensus, and Weighted Model Output Statistics by Jeffrey A. Baars and Clifford F. Mass

15. Forecast Opportunities Spreadsheet These cells turn brown when day-to-day change in observed max/min exceeds this amount

16. Forecast Opportunities Spreadsheet These cells turnblue when guidance error exceeds this amount

17. Forecast Opportunities Spreadsheet Forecast Opportunities occur for calendar days with large departures from climatology.

18. Percent of Forecasts: 14%

19. October 2005 8% November 2005 9% Forecast Opportunities - KDFW December 2005 14% January 2006 13% Threshold 10oF

20. October 2005 32% November 2005 34% Forecast Opportunities - KDFW December 2005 43% January 2006 35% Threshold 5oF

21. October 2005 8% November 2005 9% Forecast Opportunities - KDFW December 2005 14% January 2006 13% Threshold 10oF

22. February 2006 14% March 2006 16% Forecast Opportunities - KDFW April 2006 6% May 2006 4% Threshold 10oF

23. June 2006 0% July 2006 0% Forecast Opportunities - KDFW August 2006 1% September 2006 4% Threshold 10oF

24. Forecast Opportunities • Forecast Opportunities: • Typically occur for calendar days and can be consistent from one run to the next. • Are most common for Days 3-7. • Frequently occur for days on which the observed max/min temperature departs substantially from the climatological norm. • MEXMOS guidance typically underestimates the observed departure from normal. • Rarely occur for days when the temperature change from the previous day is large. • GFS seems to handle these events well.

25. Future Work • Flips and Flops: • Can one formulate an objective definition of a model flip (and flop)? • Is it possible to anticipate model flips? • When a flip occurs, can we determine whetherthe flip is in the correct direction? • Forecast Opportunities: • How can we identify Forecast Opportunities?

26. Questions? mailto: bernard.meisner@noaa.gov National Weather Service