Using Ensembles in a Deterministic Forecast Era

1. Using Ensembles in a Deterministic Forecast Era Bernard N. Meisner Scientific Services Division NWS Southern Region Fort Worth, Texas The views expressed herein are those of the author and do not necessarily reflect the position of the National Weather Service. National Weather Service

2. Using Ensembles in a Deterministic Forecast Era • What Information is Available in Ensemble Guidance? • Obviously, the ensemble mean, spread and extremes. • But what about the results of a cluster analysis? • Might the ensemble MOS guidance be calibrated to provide useful information? • What about applying pattern recognition?

3. Model Flip-Flops National Weather Service

4. Model Flips and FlopsHow to define them? Run-to-run change in MOS max/min temperatures >10oF. Current model run lies outside envelope of previous ensemble. Run-to-run change in MOS max/min temperatures exceeds monthly Mean Absolute Error.

5. Using Ensembles in a Deterministic Forecast Era • Identifying Model Flip-flops • Model flip flops usually result from forecast uncertainty, rather than radical changes in the initial data. • Operational forecast will often resemble one of the ensemble membersfrom a previous model run valid at the same time, and almost always fall within the range of forecasts given by the prior ensemble run. • The ensemble mean and spread is a better measure of fundamental run-to-run model forecast change than consecutive operational forecasts.

6. 108 Hr Forecast 102 Hr Forecast 96 Hr Forecast Verification

7. Confidence Graph, Spaghetti ChartsHigh & Low Confidence ExamplesFSU/NWS TLH Web Sitehttp://moe.met.fsu.edu/confidence/ Climatological Spread NCEP/NCAR Reanalysis Spread of Current Ensemble Run Average Ensemble Spread for time of year Sfc Wind Speed Sfc Vorticity

8. Using Ensembles in a Deterministic Forecast Era • Analysis has indicated that forecast errors are greater when the spread of the ensemble is relatively large. • But we don’t yet know how to minimize that error, given any additional information that may be contained within the ensemble guidance.

9. Model Flip Flops • Data Used: • MEXMOS Guidance for DFW and PIT Airports (Have also looked at a few other sites in NWS Southern Region.) • Flip (and Flop) Criterion: • Run-to-run change in Max/Min temperature guidance of 10oF or greater.

10. DFW Max/Min Temperature10oF Threshold

11. Model Flips…and Flops • Summary: • Flips (and flops) occur during the cool season. • Typically occur for just one verification time. • Model flips are rare; flip-flops are very rare. • 53 Flips; 11 Flops (out of 400+ model runs/7000+ forecasts) • Flips are most common for Days 4-6. • Flip-flops are most common for Days 5-6.

12. Forecast Opportunities National Weather Service

13. Forecast Opportunities • Data Used: • MEXMOS Guidance for DFW and PIT Airports (Have also looked at a few other sites in NWS Southern Region.) • Forecast Opportunity Criterion: • Max/Min temperature guidance error 10oF or greater. • Caveat: • Smaller errors at certain thresholds can be significant! • 28oF vs 33oF; 99oF vs 104oF

14. Forecast OpportunitiesThree Days in August National Weather Service

15. August 2006

16. August 2006

17. August 24, 2006 • Period One MOS Guidance: • Operational: 94 OVC • Cntrl: 99 PC • P1–P4 100 PC 98 PC 99 PC 98 OVC • P4-P8 99 PC 99 PC 100 PC 99 PC • P9-P12 97 PC 98 PC 99 PC 99 PC • P13-P14 99 PC 99 PC • While the OVC sky cover was triggered in the operational run, it was only triggered in one ensemble member.

18. August 2006

19. August 12, 2006 • Period One MOS Guidance: • Operational: 94 OVC • Cntrl: 94 OVC • P1–P4 93 OVC 94 OVC 94 OVC 93 OVC • P4-P8 96 OVC 94 OVC 98 OVC 93 OVC • P9-P12 95 OVC 93 OVC 96 OVC 93 OVC • P13-P14 96 OVC 95 OVC • All forecast runs triggered the OVC sky cover.

20. August 2006

21. August 22, 2006 • Period One MOS Guidance: • Operational: 94 OVC • Cntrl: 95 OVC • P1–P4 95 OVC 95 OVC 95 OVC 95 OVC • P4-P8 96 PC 95 OVC 95 OVC 95 PC • P9-P12 96 PC 95 OVC 97 PC 94 OVC • P13-P14 95 OVC 97 PC • All but five of the forecast runs triggered the OVC sky cover. Runs with PC sky cover still substantially underestimated the maximum temperature of 103oF.

22. Forecast Opportunities • Forecast Opportunities: • Typically occur for calendar days rather than model runs. • Are most common for Days 3-7. • Frequently occur for days on which the observed max/min temperature departs substantially from the climatological norm. • MEXMOS guidance typically underestimates the observed departure from normal. • Rarely occur for days when the temperature change from the previous day is large. • GFS seems to handle these events well.

23. Calibrated Ensemble MOS PoP Guidance National Weather Service

24. Calibrated PoP Guidance • WFO Jackson, Miss. Procedure: • Initialize each forecast package with 0000 UTC MEX MOS guidance. • Only change grids when/where necessary.(Forecast Opportunities) • Increase forecast PoP when ensemble mean PoP exceeds predetermined thresholds. • Decrease forecast PoP when emsemble mean PoP is below predetermined thresholds. • Technique applicable for cool season only.

29. Improvement over MOSKJAN PoP 2005 vs 2004 2005 2004

30. Calibrated PoP Guidance • Results: • Big improvement over previous year. • Removed previous dry bias for all forecast periods. • Average forecast PoP was greater than MOS guidance on rainy days for all forecast periods. • Average forecast PoP was less than MOS guidance on dry days for all forecast periods.

31. Using Ensembles in a Deterministic Forecast Era • Even in a probabilistic forecast era, some users will continue to require the most likely scenario. Will that always be the ensemble mean? • As the size of the ensembles increase it will not be possible to manually examine the output from each member. (No more spaghetti charts!) • How might we glean all the useful information from an ensemble – much more than just the means and extremes?

32. Questions? mailto: bernard.meisner@noaa.gov National Weather Service