Predictability, Ensemble Forecasts, and the use of Statistical Guidance in the Forecast Process

1. Predictability, Ensemble Forecasts, and the use of Statistical Guidance in the Forecast Process Steve Keighton National Weather Service Blacksburg, VA

2. Outline • Chaos theory and predictability in the atmosphere • Numerical Weather Prediction (NWP) and use of “ensemble” forecast methods • Use of statistical guidance in the forecast process (Model Output Statistics) – if time Acknowledgments • Josh Korotky – NWS Pittsburgh • Mark Antolik – NWS Meteorological Development Lab

3. “Prediction is very difficult, especially about the future”- Niels Bohr

4. Prelude – What is Chaos and why is it important? • Chaos leads us from the laws of nature to their consequences • …shows us that simple systems can exhibit complex behavior…and vice versa • …demonstrates that unpredictable behavior can develop in a system governed by deterministic laws • As forecasters…chaos shows us the limits of predictability • …highlights the importance of probabilistic thinking • …shows us the value of expressing uncertainty in forecasts • …helps us understand why the future of forecasting will lean heavily on ensemble rather than deterministic approaches

5. Edward Lorenz (1917 – 2008) • Small errors in the initial state estimate of a nonlinear system can limit the prediction of later states of the system • Chaos occurs when error propagation, seen as a signal in time, grows to the same size or scale as the original signal... “… one flap of a sea-gull’s wing may forever change the future course of the weather” (Lorenz, 1963)

6. Elements of Chaos • Dynamical system – future states caused by past states (determinism) • Nonlinearity – system output (response) isn't proportional to input (forcing)… • a small forcing can lead to a disproportionately large response and vise versa • a system's values at one time aren‘t proportional to the values at an earlier time • Non-periodic behavior – future states never repeat past states • Extreme sensitivity to initial conditions – small initial state uncertainties amplify… • a "prediction horizon” is inevitable • Even though the governing laws of a system are known, long-term predictions can be meaningless • Chaos occurs only in deterministic, nonlinear, dynamical systems

7. Attractors – General Statements • An attractor is a dynamical system's set of conditions • In a phase space diagram, an attractor shows a system's long-term behavior. It's a compact, global picture of all of a system's possible steady states. • All attractors are either nonchaotic or chaotic • Nonchaotic attractors generally are points, cycles, or smooth surfaces (tori), and have regular, predictable trajectories…small initial errors or minor perturbations generally don't have significant long-term effects • Chaotic or strange attractors occur only after the onset of chaos. Long term prediction on a chaotic attractor is limited…small initial errors or minor perturbations can have profound long-term effects

8. Strange Attractors • A Strange Attractoris dynamically unstableand non periodic • - A chaotic system is unstable…its behavior changes with time rather than settling to a fixed point • - Chaotic systems are non periodic…trajectories do not settle into repeatable patterns …and never cross • - A chaotic attractor shows extreme sensitivity to initial conditions… trajectories initially close, diverge, and eventually follow very different paths

9. Non-periodic Dynamical System • A dynamical system that never settles into a steady state attractor • Non periodic systems never settle into a repeatable (predictable) sequence of behavior. • Prediction of a future state of a non periodic system is eventually impossible, due to nonlinear dynamics (feedback) • The atmosphere illustrates non periodic behavior • Broad patterns in the development, evolution, and movement of weather systems may be noticeable, but no patterns ever repeat in an exact and predictable sequence • The atmosphere is: • …damped by friction of moving air and water • …driven by the Sun’s energy • …the ultimate feedback system • Weather patterns never settle into a steady state attractor

10. Sensitivity to Initial Conditions • Small uncertainties (minute errors of measurement which enter into calculations) are amplified • Result: system behavior is predictable in the short term…unpredictable in the long term

11. The Lorenz Discovery • From nearly the same starting point (tiny rounding error), the new forecast diverged from the original forecast…eventually reaching a completely different solution! • Why? …Slight differences in the initial conditions had profound effects on the outcome of the whole system • Lorenz found the mechanism of deterministic chaos: simply-formulated systems with only a few variables can display highly complex and unpredictable behavior (.506) vs. (.506127) Initial condition

12. Chaos and Numerical Weather Prediction (NWP) If a process is chaotic… knowing when reliable predictability dies out is useful, because predictions for all later times are useless. • Weather forecasts lose skill because of: • Chaos…small errors in the initial state of a forecast grow exponentially • Model uncertainty • Numerical models only approximate the laws of physics (important small scale processes are parameterized) • Very small errors in the initial state of a forecast model grow rapidly at small scales, then spread upscale • Forecast skill varies both spatially and temporally as a result of both initial state and model errors, which change as the atmospheric flow evolves

13. Models must simulate numerous irresolvable processes

14. NWP Skill as a Function of Scale and Time Feature/Variable Feature/Variable < Day1 < Day1 Days 1-2 Days 1-2 Days 3-5 Days 3-5 Days 6-7 Days 6-7 Hemispheric flow transitions Hemispheric flow transitions Excellent Excellent Excellent Excellent Very Good Very Good Good Good Cyclone life cycle Cyclone life cycle Excellent Excellent Very Good Very Good Fair-Good Fair-Good Low skill-Fair Low skill-Fair Fronts Fronts Excellent Excellent Good Good Fair Fair ---- ---- Mesoscale banded structures Convective clusters Mesoscale banded structures Convective clusters Good Good Fair Fair ---- ---- ---- ---- Temp / wind Temp / wind Excellent Excellent Very Good Very Good Skill with max/min Temp Skill with max/min Precip/ mean clouds QPF/ mean clouds Very Good Very Good Good Good Some skill in 5-10 day QPF Fair • Predictability falls off as a function of scale • Large scale features (planetary waves) may be predictable up to a week in advance • Small systems (fronts) are well forecasted to day 2.. cyclonic systems to day 4

15. How do Ensembles help us cope with Chaos?

16. Why can’t we count exclusively on single model NWP? • Overlooks forecast uncertainty • Initial condition and model uncertainty • Chaotic flows vs. stable flow regimes • Potentially misleading • Oversells forecast capability

17. NAM 84 hr forecastValid 00Z 22Nov GFS 84 hr forecastValid 00Z 22 Nov Single Model NWP Which model do you believe?

18. Single Forecast High Res Control PDF PDF PDF Reality Reality Time Ensembles and PDF • Recognizing the eventuality of chaos…weather forecasts can provide more useful information by describing the time evolution of an ensemble probability density function (PDF) • Initial PDF represents initial uncertainty • Single forecast doesn’t account for initial and model error…often fails to predict the real future state past a certain point • Ensemble of perturbed forecasts accounts for initial and model error… PDF of solutions more likely to contain real future state • Ensemble PDF contains additional information, including forecast uncertainties

19. Ensemble Prediction System (EPS) Goals • Represent initial condition and/or model uncertainty • Determine a rangeof possible forecast outcomes • Estimate the probabilityfor any individual forecast outcome • General: provide a framework for decision assistance

20. General EPS forecasting tools • Spaghetti Plots (shows all solutions) • Mean/Spread (“middleness” and variability) • Probabilities • Most Likely Event

21. “Spaghetti” Plots

22. Mean and Spread • Characteristics of mean • The ensemble mean performs better on average than operational model on which it is based. Why? • Because predictable features remain intact, less predictable features are smoothed out • Characteristics of spread • Allows assessment of uncertainty, since more spread means more uncertainty 4 1 3 2

23. Probability of Exceedance • Helps determine the probability of a specified event. • Gives probability of exceeding meaningful threshold • Calculation represents count of what % of ensemble members exceed the threshold of interest • Example here is for 12-hour precipitation exceeding 0.25 inches.

24. Most Likely or Dominant Event Diagram • Used to show what is most often predicted by the ensemble forecast • A common example • Precipitation type (snow, sleet, freezing rain, rain)

25. Summary • Chaos and model uncertainties impose a very real physical limit on predictability • Predictability falls off (sometimes rapidly) as a function of scale and time • Forecast accuracy varies both spatially and temporally as a result of initial state and model errors, which change as the atmospheric flow evolves • Ensemble NWP optimizes predictability for all scales, and extends the utility of forecasts…especially at extended ranges (days 4-7) • Allows for quantification of uncertainty, and foundation for decision assistance

26. Statistical Guidance in the Forecast Process

27. WHY STATISTICAL GUIDANCE? • Add value to direct NWP model output • Objectively interpret model • - remove systematic biases • - quantify uncertainty • Predict what the model does not • Produce site-specific forecasts • (i.e. a “downscaling” technique) • Assist forecasters • “First Guess” for expected local conditions • “Built-in” model/climatology

28. MODEL OUTPUT STATISTICS (MOS) Relates observed weather elements (PREDICTANDS) to appropriate variables (PREDICTORS) via a statistical approach. Predictors are obtained from: • Numerical Weather Prediction (NWP) Model • Forecasts • 2. Prior Surface Weather Observations • 3. Geoclimatic Information • Current Statistical Method: • MULTIPLE LINEAR REGRESSION • (Forward Selection)

29. MODEL OUTPUT STATISTICS (MOS) Properties • Mathematically simple, yet powerful • Need historical record of observations • at forecast points • (Hopefully a long, stable one!) • Equations are applied to future run of • similar forecast model • Probability forecasts possible from a • single run of NWP model • Other statistical methods can be used • e.g. Polynomial or logistic regression; • Neural networks

30. MODEL OUTPUT STATISTICS (MOS) • ADVANTAGES • - Recognition of model predictability • - Removal of some systematic model bias • - Optimal predictor selection • - Reliable probabilities • - Specific element and site forecasts • DISADVANTAGES • - Short samples • - Changing NWP models • - Availability & quality of observations

36. Now approx. 1820 sites

38. Gridded MOS • “MOS at any point (GMOS) • - Support NWS digital forecast database • 2.5 km - 5 km resolution • - Equations valid away from observing sites • - Emphasis on high-density surface networks • - Use high-resolution geophysical data • - Some problems over steep terrain or data-sparse • regions

39. Gridded MOS

40. Use of MOS at a Forecast Office • Can ingest GMOS directly into local digital forecast database • Can apply bias correction (based on performed in past 30 days) • Can ingest point-based MOS and spread it to entire grid • MOS from single models or from ensemble mean/max/min • We verify our forecast against MOS, so we may use as a starting point but we try to improve on it based on local experience or recent trends

41. Questions?