Update to COPC: Global Model Performance Dropouts

1. Update to COPC: Global Model Performance Dropouts Dr. Jordan Alpert NOAA Environmental Modeling Center contributions from Dr. Brad Ballish, Dr. Da Na Carlis, Dr. Rolf Langland and CDR Mark Moran

2. Outline • Tools to compare ECMWF and NCEP Dropouts • Model Runs for Dropout Experiments • Results from SH Dropout Experiments • Model Runs for Dropout Experiments • Comparison of How Well the GSI and ECM “Fit” Obs • Wind Speed Biases • How GSI and ECM Fit Observations

3. Poor Forecasts or Skill Score “Dropouts” Lower GFS Performance. Detailed on next slide

4. Skill Score “Dropouts” Lower Overall GFS Performance.

5. Analysis Team for Dropout Issue Jordan Alpert - EMC DaNa Carlis - EMC Brad Ballish - NCO Krishna Kumar - NCO Rolf Langland - NRL

6. Tools to compare ECMWF and NCEP Dropouts • Use ECMWF analysis to generate “Pseudo Obs” for input to the Gridded Statistical Interpolation (GSI) and GFS forecasts. • Generate a Climatology of NH and SH dropouts – what are the systematic differences • Interpolate ECM and GSI analyses to observations to determine comparative strengths and weaknesses. • Statistically analyze observation type fits • stratify by pressure, type, and difference magnitude Goal: Diagnose Quality Control problems to implement real-time QC detection/correction and improvements to analysis system algorithms

7. Model Runs for Dropout Experiments • GFS: Operational (Production) GFS • ECMWF: Operational • ECM: Pseudo Obs from ECMWF analysis using the GSI as a “Grand Interpolator” • ecmanlGES: GSI run using ECM as background guess plus GDAS observations • CNTRL: Updated GSI system with GDAS obs • InterpGES: Updated GSI with previous 3, 6, & 9-hour ECM forecasts as background guess plus GDAS observations

8. Results from SH Dropout Experiments • Using ECM (Pseudo Obs) yielded a 90% success rate for alleviating SH dropouts. • Adding GDAS Obs (ecmanlGES) lowers the success rate to ~75%. • Combining updated GSI system / GDAS obs (CNTRL) with all obs improves GSI but not enough to alleviate dropouts • Combining background guess from ECM plus all obs (InterpGES ) also improves GSI but not enough to alleviate dropouts • Composites show low-level temperature difference near 850mb in SH. Adding the SH observations and the 3, 6, 9-h background guess degrades the ECM forecast

9. Rolf Langland (NRL Monterey) • shows systemic height differences between all models and ECMWF (shown is ECMWF-NCEP). • Cause may be the difference in satellite window coverage (under study): • ECMWF (12-h) vs. others (6-h). • Plots at left show height difference plots of time (October to December 2007) vs. longitude, averaged over 35-65N latitudes. • The range of the bias is ±12 m

10. Comparison of How Well the GSI and ECM “Fit” Observations • Statistics are made on how well the GSI and ECM analyses “fit” the observations for different observation types, as function of pressure, for different regions • This study shows that each analysis “fits” certain observation types with differing amounts of success, but does not conclude which performs better.

11. Comments on Wind Speed Biases • The ECM analysis winds at satellite wind locations are stronger than those of the GSI • The ECMWF data quality control is more aggressive at deleting satellite winds with speeds slower than the background • ECM wind speeds are stronger as weaker satellite winds are deleted or given less weight in the ECMWF analyses • For ACARS and sondes, the biases are similar Work continues to analyze satellite wind speed biases to determine an implementable algorithm for QC, bias correction, and analysis weights

12. How GSI and ECM Fit Observations • The GSI is found to draw more closely for most winds, especially satellite winds and when there are moderate differences in the analyses (outliers) • The ECM analyses draw more closely for radiosonde temperatures, especially away from the middle atmosphere and for moderate differences in the analyses • Fits for surface pressure and moisture are similar – not shown here • ECM winds are stronger at satellite wind locations based on speed bias stats but with similar fits for radiosonde and ACARS speeds Changes in the QC and Ob errors could retune these data fits

13. Summary and Future Work • ECM analysis show dropouts can be alleviated in GFS forecasts • Running the operational GSI with an ECM derived background guess results in better forecast skill than the operational GFS but not as good as ECM runs • Running the operational GSI after removing select observation types offers a systematic approach for assessing the impact of different observation types • There are systematic height differences between all models and ECMWF perhaps from the larger satellite window coverage of ECMWF (12-h) vs others (6-h) • Work continues to analyze what is the optimal fit of the analysis to observation types and to determine an implementable algorithm for improved QC, bias correction, and analysis weights • Work has started to use baroclinic instability rates to help analyze dropout cases and analysis differences

14. Background Slides

15. ECMWF INITIAL CONDITIONS FOR GFS FORECASTS “ECM” Runs” ECM(WF) Analysis 1x1 deg, 15 levels ------------------------- PSEUDO ECM “OBS” ECM ANL ------------ GFS FORECAST PSEUDO ECM “OBS” GFS GUESS -------------- ECM ANLYSIS “PRE-COND” GUESS Run GSI PRE-COND ECM GUESS PSEUDO ECM “OBS” --------------- ECM ANL ( ) INPUT ----------- OUTPUT Run GSI again

16. InterpGES Analysis ------------ GFS FORECAST 5-days “InterpGES Runs” ECM(WF) Analysis ------------------------- PSEUDO ECM “OBS” GFS FORECAST BKGND GES -3, 0, +3 GDAS Obs ------------ InterpGES Analysis ECM ANL ------------ GFS FORECAST 3, 6, 9-hour PSEUDO ECM “OBS” GFS GUESS -------------- ECM ANLYSIS “PRE-COND” GUESS Run GSI Run GSI again PRE-COND ECM GUESS PSEUDO ECM “OBS” --------------- ECM ANL ( ) INPUT ----------- OUTPUT

17. Excerpt of SH Dropout Climatology List (AC skill scores) NB: 90% ECM Success in alleviating SH dropout. Adding GDAS Obs (ecmanlGES) lowers the success rate to ~75%. GSI upgrades (CNTRL) improves GSI but not enough to alleviate dropouts, and for InterpGES runs also improves GSI but not enough to alleviate dropouts meaning that adding the SH observations (and the 3, 6, 9-h ges) degrade the ECM forecasts.

25. SH Dropout Composite Cntrl vs InterpGES SH CNTRL 5-day Composite AC = 0.71 SH InterpGES 5-day Composite AC = 0.73 Zonal average (above) and mean composite difference between GFS Cntrl and InterpGES (Increments). GFS has low level SH warm “bias” if we consider ECMWF as ground truth.

26. Guess Comparison of CNTRL vs. InterpGES Expt. Analysis • InterpGES run introduces a lower height and temperature bias to GSI • ANL differences are not significantly different after GSI interpolation but CNTRL maintains high bias

27. Impact of GSI and OBSThe addition of GSI Observations causes a positive height bias in the SH CNTRL InterpGES Typical Dropout case

28. EU Satwinds are not used in the GSI • The GSI draws more for most Satwinds

29. The GSI draws more for sonde winds especially away from jet level

30. The GSI draws more for outlier winds – less so around jet level

31. ECM fits better 300 hPa and up The draw for all temperatures is similar – But see next slide

32. The GSI draws more for outlier temperatures in the middle atmosphere and the ECM draws more in the jet level to upper atmosphere

33. Satellite Wind Speed Biases OB-ANL Apr 2008 12Z ECM (Red), GSI (Blue) m/sec Note ECM biases are mostly negative compared to GSI, meaning ECM winds are stronger than GSI winds at satellite wind locations

34. Baroclinicity parameter for GFS and ECMWF A suitable measure of the baroclinicity is given by the Eady growth rate maximum defined as: σBI = 0.31 f |(-gp/RT) ∂V/ ∂p| N-1 where f is the Coriolis parameter, V is the total vector wind, N is the Brunt Väisällä frequency and all other parameters have their usual meaning. This is done to show potential action areas or volatility to propagate initial condition errors into forecast differences

35. NH Eady Stability index Difference: GFS - ECMWF Largest differences for this NH Dropout case, 20081021, are in Pacific (disregard most mountain areas as quantities are extrapolated values below Ground). SH Eady Stability index GFS - ECMWF The predominance of Red over Blue shows GFS has more pronounced baroclinicity compared to ECMWF Ops in low levels (800mb)…

36. NH GFS Eady Stability index The total Eady-index shows the Greatest baroclinic potential in the eastern part of the broad Pacific trough, the differences in this case, cause a dropout in the 5-day forecast. NH ECMWF Eady Stability index

37. NH Eady Stability index GFS - ECMWF Large differences are along trough line with dipole, indicated by Blue and Red indicating difference in position (phase), and large potential for this Rossby wave details. SH Eady Stability index GFS - ECMWF And at 500 mb and 200 mb (not shown), GFS has more pronounced baroclinicity compared to ECMWF Ops. The differences are as much as 20% of the total index shown on next slide.