Can we Predict the Impact of Observations on 3 to 6 day Winter Weather Forecasts?

1. Can we Predict the Impact of Observations on 3 to 6 day Winter Weather Forecasts? Masters Thesis Defense May 10, 2007 Kathryn J. Sellwood University of Miami, R.S.M.A.S., M.P.O. Dept.

2. Why winter weather?It’s not all about Hurricanes!

3. Observations Upstream in Storm Tracks Used to Improve Forecast in Downstream Locations Storm Tracks for 1994 winter season (left) 6 year average eddy kinetic energy (right) from James, 1994

4. Influence of Observations Propagates eastward Poorly Observed Region Future Winter Storm Location

5. Upstream Observations improve the Downstream Forecast More Accurate Forecast Early warning

6. IF we can predict the impact of observations on specific 3-6 day winter weather forecasts THEN Observations can be used to improve accuracy and extend effective time range of forecasts

7. “Signal” from Observationsat t=0 (left) and t=80 hours (right) combined 200 h-Pa u,v and T squared signal at 0 hours (left) and 80 hours (right)

8. Outline • Background: Targeting ETKF Technique Results from Previous Targeting Field Programs • Research Methodology • Results • Conclusions • Future Work

9. Operational Targeting Timeline Targeting Method Future Analysis Verification initialization time (targeting) time time ti ta tv Decision time  36-60 hours  1-7 days 

10. Targeting in WSR target regions identified for 2 day east coast forecast Observation time A Day 1 A B All WSR flight paths Operational flight path Day 2 -Verification time A B

11. Ensemble Transform Kalman Filter(ETKF) • Quantifies impact of observations • Estimates the forecast error covariance matrix from an ensemble • Assimilates observational data using a Kalman filter • Computes resulting reduction in forecast error variance 5400m and 5820m 500 h-Pa height ensemble

12. Forecast error covariance matrix computed from matrix of ensemble perturbations Z Pf = Zf ZfT Kalman Filter equation used to obtain new error covariance matrix if observations are assimilated Pq= Pr – Pr HqT (HqPrHqT+Rq)-1 HqPr Difference is the Signal Covariance Matrix Sq Sq=Pr–Pq Signal Variance = reduction in forecast error variance

13. ETKF Issues • ETKF relies on linear theory • Depends on ensemble quality • Assumes Kalman filter data assimilation / operational scheme is 3D-Var • DA scheme introduces small scale noise that contaminates signal (Hodyss and Majumdar, 2007)

14. Results from previous Field Programs • Targeting is effective in reducing short term forecast errors (Langland et al, 1999, Szunyogh et al., 1999, 2000, 2002) • ETKF effective for short range 1-3 day targeting (Majumdar et al., 2002-a, Szunyogh et al., 2000, 2002) • Flow regime related to effectiveness of ETKF • Data “signal” propagates in the form of upper tropospheric Rossby wave packets (Szunyogh et al., 2000, 2002) • Downstream development maintains wave packets and influences signal propagation (Szunyogh et al., 2002, Majumdar et al., 2002-b) • The presence of coherent wave packets beyond operational lead times is evidence of data influence in the medium range

15. Observations over Pacific improve 3 day forecast over U.S. east coast and 6 day European forecast Improvement at 78 hours (u,v,T) 144 hour improvement (u,v,T)

16. Main Objectives • Quantify ETKF’s ability to predict signal variance in the medium range • Determine scales at which ETKF is effective • Explore influence of flow regime • Determine whether ETKF can distinguish between promising targeting cases and those where observations would have minimal effect

17. 2 Comparison Methods • Method 1: measures the spatial correlation between the ETKF predicted signal variance and the squared GFS signal • Method 2: makes a quantitative evaluation of ETKFs’ skill in predicting signal variance

18. The Data Set • Data is from the 2006 Winter Storm Reconnaissance Program • 19 individual cases • Forecast variables are 200 h-Pa winds and temperature • ETKF signal variance derived from a 50 member ECMWF ensemble • Signals calculated as the difference between 2 forecasts that are identical except 1 omits the WSR observations • Forecast produced using NCEPs Global Forecast System Model (GFS) • Both fields at 1 degree resolution

19. Methodology • Spatial fields of ETKF “predicted” signal variance and GFS squared-signal (“verification”) are smoothed by averaging over lat-lon grid cells. • Field domain includes 180° W to 20° E from 20 to 80°N • Correlation coefficients between these smoothed spatial fields are calculated, at all lead times, for various grid spacings. • Correlation coefficients for actual case-specific predictions are then assessed relative to a “no-skill” baseline, constructed by randomizing the predictions of all 19 weather cases in the sample.

20. The Randomized Baseline • Baseline random correlations are computed for all lead times from 0 to 144 hours • The ETKF predicted signal variance for each of the 19 cases is compared to the squared GFS signals from the 18 different cases • Results in a distribution of 342 random correlations • This baseline captures the (non) skill of case-independent spatial structure (like climatological storm tracks) • ETKF’s skill for the individual cases is compared against this baseline

21. Case Specific vs. Random Correlations Same Case Similar Pattern Randomly Selected Case Less Correlated

22. Case specific (top) Random (bottom) at 108 hour lead time

23. Correlation skill of case-specific vs. randomized predictions (2° grid) Blue = Random Red = ETKF

24. Same as previous but for 5° grid

25. 10° grid

26. 15° grid

27. 20° grid

28. 30° grid

29. Average ETKF Correlations (solid) and Random Correlations (dashed) as a Function of lead time and resolution

30. Significance test • Are the ETKF correlation coefficientssignificantlygreater than the random distribution? • Use Kalmogorov-Smirnov test for the difference of two PDFs.

31. Example of K-S test

32. 13/20 data points = Maximum difference between distributions 4/ 16 data points

33. The Kolmogorov – Smirnov test • Compares cumulative distribution function (CDF) • Produces 2 statistics based on “D” value • H statistic tests the null hypothesis that the 2 distributions are equal H = 0 cannot reject null hypothesis H = 1 can reject with 95% confidence • P statistic gives probability that the 2 distributions are indistinguishable • Test applied for all lead times and resolutions

34. 2° grid

35. 5° grid H = 1 for all lead times > 0

36. 10° grid H = 1 for all lead times > 0

37. 15° grid H = 1 for all lead times > 0

38. 20° grid H =1 for all lead times > 0

39. 30° grid H = 1 for all lead times > 0

40. ETKF significantly beats random for all grid spacing and lead times > 0 • At 0 hour leads ETKF predictions are not significantly better than random climatology • ETKF case-specific predictions exhibit significantly better than random skill for the time ranges (3-6) days that we are interested in • Skill tends to improve (relative to random) at longer lead times • Higher correlations at lower resolution (larger grid)

41. ETKF has been shown to have skill in predicting the general pattern of signal variance over a large domain but… • We want to apply ETKF to specific forecasts • Can the ETKF predict signal variance specifically in predetermined verification regions at 3-6 day lead times? • If so at what resolutions and for what size verification regions

42. Verification Regions • Same methodology as full domain comparison • ETKF predicted signal variance compared to squared GFS signal over 20° x 20°, 40°x 40° and 60°x 60° verification regions • Verification regions selected using wave packet technique of Zimin et al., 2003 • Verification regions placed at the leading edge of wave packet maximum in ETKF predicted signal variance

43. Typical 120 hour V.R.s 60 X 60 40 x 40 20 x 20

44. ETKF vs Random20° Verification Region - 2° grid RED = ETKFBLUE = RANDOM

45. K-S test20° Verification Region 2° grid RED = ETKFBLUE = RANDOM

46. ETKF vs Random20° Verification Region - 5° grid

47. K-S test20° Verification Region 5° grid

48. ETKF vs Random40° Verification Region - 2° grid

49. K-S test40° Verification Region 2° grid

50. ETKF vs Random40° Verification Region 5° grid