Lightning Assimilation Project: Experiment Review and Case Study Results

1. WRF-Ensemble Kalman Filter Lightning Assimilation Project December 18th, 2008 Phil Regulski, Cliff Mass and Greg Hakim

2. Overview • Experiment review • Case study results • December 2002 (review) • October 2004 • November 2006 • Statistical analysis • Lightning flash rates and model output variables • Conclusions • Implications of real-time assimilation of lightning • Modeling techniques

3. Experiment Review • Control run • No lightning observations were assimilated • Radiosondes, surface stations (ASOS, ship, buoy), ACARS and cloud drift wind (no radiances) observations assimilated • Experimental runs • Lightning “observations” are assimilated along with observations used in the control run • Experiment 1 (LTNG1): Non-thinned lightning • Lightning strike observations are converted into 30 min. lightning flash rates from nearby lightning points • Lightning rate is converted into an “observation” of convective rain using the Pessi/Businger convective rain rate/lightning rate relationship • Convective rain is assimilated into the WRF-EnKF • Experiment 2 (LTNG2): Thinned lightning • Same as previous experiment except that any lightning strikes used in a previous density calculation are no longer allowed to be an assimilation point for the filter, resulting in a thinning out of the lightning “observations” (although strikes can be reused to calculate nearby densities)

4. Case StudiesDecember 16-21, 2002

5. Case StudiesDecember 16-21, 2002 Thinned Exp. • Summary: • Assimilating lightning has an impact on the analysis and 12 (not shown) / 24 hour forecasts • Are the impacts improvements? 24-hr Forecast of SLP: Lightning assimilation (black lines) versus control (red lines) with differences between the two shaded. Lightning drawn as black dots with observations in blue.

6. Case StudiesDecember 16-21, 2002 12-hr Forecast SLP RMS Errors over domain (% Improvement compared to control) Verified versus GFS Analysis • Summary: • 12- and 24-hr forecasts show improvement when lightning is assimilated into the WRF-EnKF 24-hr Forecast SLP RMS Errors over domain (% Improvement compared to control) Verified versus GFS Analysis

7. Case StudiesDecember 16-21, 2002 • Summary: • Dec. 2002 test case has abnormally high lightning flash rates • Are results repeatable with typical winter extra-tropical cyclones?

8. Case Studies October 1-14, 2004

9. Case Studies October 1-14, 2004 Non-thinned Thinned Example of 24-hr Fcst. of SLP: Lightning assimilation (black lines) versus control (blue lines) with differences between the two shaded. Summary:Smaller amounts of lightning having less impact on the 24-hr forecasts. Are the smaller impacts, at least, improving the forecasts?

10. Case Studies October 1-14, 2004 Control Thinned Example of 24-hr Fcst. SLP Error Maps: WRF-EnKF Analysis (black lines) versus control (blue) and experiment (red) with differences between experiment and the “truth” (WRF-EnKF Analysis) shaded. Summary:Positive and negative impacts to the forecasts balance out with no net improvements.

11. Case Studies October 1-14, 2004 Control Thinned Example of 24-hr Fcst. SLP Error Maps: WRF-EnKF Analysis (black lines) versus control (blue) and experiment (red) with differences between experiment and the “truth” (WRF-EnKF Analysis) shaded. Summary: There are positive and negative impacts to the experimental forecasts. No net improvement over the control.

12. Case Studies November 6-20, 2006

13. Case Studies November 6-20, 2006 Control Non-thinned Example of 24-hr Fcst. SLP Error Maps: WRF-EnKF Analysis (black lines) versus control (blue) and experiment (red) with differences between experiment and the “truth” (WRF-EnKF Analysis) shaded. Summary:No major improvements to forecasts with assimilation of lightning.

14. Case Studies November 6-20, 2006 Control Thinned Example of 24-hr Fcst. SLP Error Maps: WRF-EnKF Analysis (black lines) versus control (blue) and experiment (red) with differences between experiment and the “truth” (WRF-EnKF Analysis) shaded. Summary:No major improvements to forecasts with assimilation of lightning.

15. Statistical Analysis • Why are we seeing less forecast improvements, if any, in Case 2 and Case 3? • Variance analysis • Statistical analysis of predictor/predictand relationships

16. Statistical Analysis During intense lighting events the model forecasted convective rain rate exhibits good agreement with lightning observation locations. As a result model variance values are nicely spread over wide range of model output values. Model forecasted 6-hr accumulated convective rain with 1-hr lighting values.

17. Statistical Analysis When lightning is scattered in less dense packets, such as for cold advection cumulus, model values trend to zero causing a much smaller range in variance. Model forecasted 6-hr accumulated convective rain with 1-hr lighting values.

18. Statistical Analysis Reducing the period of model accumulation of precipitation from 6- to 1-hr may position the lighting better but the lightning now has higher probability of sitting on zero valued model gridpoints reducing variance. Model forecasted 1-hr accumulated convective rain with 1-hr lighting values.

19. Statistical Analysis Variance analysis: Subtleties of Lightning Assimilation Technique • If the background forecast variance is too low the filter will assume the forecast is correct and lightning will have little to no impact. • Cases 2 and 3 exhibit this problem consistently due to less intensive precipitation events. • Reducing the convective rain rate totals from 6- to 1-hr more accurately positioned the dynamical features (i.e., fronts and convective cells) with the lightning but 1-hr precipitation totals reduced the values of the model field which meant that many more lightning assimilation points had greatly reduced variances.

20. Statistical AnalysisOctober 2004 and November 2006 • If lightning is not impacting the forecasts enough can we change the filter sensitivity to the observations? • Altered observation errors • Changing the error associated with the observation proxy (convective rain) changes the “weight” the Ensemble Kalman Filter gives to the assimilated convective rain value versus the background 6-hour forecast value • No major changes in results with new observation error values in areas with low lightning density • You can change the observation error values all you want but if the model background field is zero it wont matter. • The problem with assimilating a field, such as convective rain rate, which often is zero, is that when it is zero, no matter what information the observation provides at the same location, the error covariance matrices will be useless. Therefore no additional information from the lightning observations will be ingested into the filter. • All lightning observations that assimilated convective rain rate have this problem

21. Statistical Analysis • Re-examine the lightning flash rate to convective rain rate relationship with data from case studies. • Is there a better relationship with a field that is continuous? • Existing Pessi/Businger relationship used as first attempt to relate model field with lightning rate for WRF-EnKF. • Can we find a model output variable parameter that always has a non-zero value thereby making full use of Kalman filter? • Convective rain values that are zero handicap the WRF-EnKF due to its use of error covariance matrices. • Examine relationship of lightning flash rate to other model output variables • Screening Linear Regressions • Precipitation fields: Convective, grid scale and total rain • Moisture fields: relative humidity • Wind fields (u, v) • Vertical winds (w) • Lapse rates, vorticity advection, Lifted Index, etc.

22. Statistical Analysis • Used the Pessi/Businger lightning rate/convective rain rate relationship as the conversion from lightning to ingestible model information to the WRF-EnKF. • Already studied and ready to use. • Works well for the Ensemble Kalman filter when there are very large amounts of lightning. • Convective rain is a problem for the Ensemble Kalman filter when the background field is zero. • Does our model output have a similar relationship during typical winter storms traversing the eastern Pacific?

23. Statistical Analysis • Relationship between convective rain rate and lightning rate breaks down when Case 2 and Case 3 are added to data analysis. • Many more lightning observations over small conv. rain model values (increase in bottom left corner of right graph) • Mean precipitation value goes down from .83 to .23 when data from Case 2 and Case 3 are added • Large amount of low flash rates and high rain rates also contaminate the relationship (upper left of right graph). • Where there is convective rain there are only small lightning densities which hurt relationship. Case 1 Case 1, 2 and 3 Flash frequency versus convective rain rate

24. Statistical Analysis • Testing the convective rain rate/lightning rate relationship • The convective rain rate/lightning rate relationship using the WRF-Ensemble Kalman Filter model output is less robust during less lightning active events in the eastern Pacific using our small sample. • Are we being fooled by the test cases? • Larger sample needed. • Examined Vaisala’s Real-time LR unfiltered lightning data from entire month of December 2007 with Real-time WRF-EnKF fields from same time period. • 9200 unique lightning points within domain • Also examined… • Size of box to determine grid density • Lightning over ocean vs. land vs. both • Background model value (interpolation, grid averages, single grid) • Examine relationship of lightning flash rate to other model output variables • Screening Linear Regressions • Precipitation fields: Convective, grid scale and total rain • Moisture fields: relative humidity • Wind fields (u, v) • Vertical winds (w) • Lapse rates, vorticity advection, Lifted Index, etc.

25. Statistical Analysis • Larger sample? • Examined Vaisala’s Real-time LR unfiltered lightning data from entire month of December 2007 with Real-time WRF-EnKF fields from same time period • Size of box to determine grid density • Optimal box size is approx 1-1.5 degrees • Lightning over land vs. ocean vs. both • Using land lightning hinders relationships with model metrics due to terrain and other complications (land v. ocean effects, orographic induced dynamics, diurnal cycles, etc.) • Background model value (interpolation, grid averages, single grid) • Little difference in results to how model value at lightning observation point is reached. • Relationships still not robust • Still see areas where lightning occurs but WRF model does not have the corresponding synoptic pattern • Can we force relationship using a convective rain filter? • Only use lighting where there are positive convective rain rate values in the model for the statistical analysis (model shows signs of correct synoptic pattern if there is precipitation there)

26. Statistical Analysis Best predictors of lightning using 1.5 degree density box Lapse rate (850-500) * RH850 R^2 = 0.172 Grid scale rain R^2 = 0.115 RH850 R^2 = 0.131 Total rain R^2 = 0.092 (T2-SST)*RH850 R^2 = 0.120 W850 R^2 = 0.113 Can we find better relationships in longer timer periods (6-,12 and 24-hr) accumulations?

27. Statistical Analysis 6-hr ltng and 6-hr Total rain 1-hr ltng and 6-hr Total rain Relationship degrades using longer assimilation time periods. Does a robust precipitation rate/lightning rate relationship exist?... 12-hr ltng and 12-hr Total rain

28. Statistical Analysis • Best model metric is Lapse Rate (850-500 hPa) * RH850 • Not very convincing metric/lightning rate relationships using UW’s Real-Time WRF-EnKF model output fields • What might be impairing finding a relationship between metric and lightning rate hurting the statistical analysis? • Model’s inability to correctly predict the synoptic pattern(most likely) • Unfiltered data: Streak contamination, bad data • Detection efficiency: Missing data over non-zero model output • Higher model resolution, more detailed microphysics… (graupel relationship)? • Same fundamental problem of model incorrectly forecasting synoptics

29. Conclusions • Analysis and forecast improvements during the abnormally high flash rate cyclone of December 2002 can be attributed to • Large non-zero values of convective rain throughout domain • Strong convective rain rate/lightning flash rate relationship during December 2002 storm • Poor performance of forecasts during Cases 2 and 3 • Poor convective rain rate/lightning flash rate relationship • Low lightning rates • Small model forecast variances in predictor (convective rain rate)

30. Conclusions • Lightning observations will not help improve forecasts in the WRF-Ensemble Kalman Filter, IF the model does not correctly predict the synoptic pattern • Without correct placement of dynamical features there will never be a statistically significant relationship between lightning rate and a model metric for the Ensemble Kalman Filter • Improvements to real time WRF-EnKF will only be useful when there are extreme amounts of lightning • Nudging may prove a better method than WRF-EnKF since you are altering the model profiles rather than depending on background forecasts • Adding lightning to the real-time system will only improve forecasts IF there are extreme amounts of lighting • Nudging doesn’t ask anything from the model only requires observations • WRF-EnKF/nudging hybrid?