A statistical framework for the analysis of large lightning and radar datasets

1. A statistical framework for the analysis of large lightning and radar datasets Timothy J. Lang and Steven A. Rutledge Department of Atmospheric Science Colorado State University, Fort Collins, CO + = ?

2. The Past Typically, in studies involving both polarimetric radar and total lightning data, emphasis has been placed on a case study approach, borne largely out of necessity, as historically these datasets have been large and unwieldy given computational limitations. The case-study approach has yielded valuable scientific results and insight. However, there is a fundamental weakness, as all case studies ultimately suffer from lack of statistical significance in their results, and from sampling issues that make it difficult to generalize findings.

3. The Future Expansion of research radar networks Polarimetric NEXRAD Expansion of VHF lightning mapping networks Regional and national lightning networks (e.g., LASA, NLDN) Satellite lightning mappers (e.g., on GOES) •Key Challenge: How do we deal with the coming deluge of polarimetric radar and lightning data?

4. The Statistical Framework Radar Volume Radar Features Echo-top height Graupel echo vol Rain rate Flashes VHF Sources Locations, Altitudes Charge distribution Environment CAPE Shear Aerosol Optical Depth Concentration Polarimetric Radar Cartesian Grid Hydrometeor ID (Dual-Doppler) Feature ID TITAN SCIT PF Lightning Mapper LMA/LDAR LASA NLDN Environmental RUC Model Sounding Network Aerosol MODIS Ground/Air Obs Modularity is key!

5. Statistical Framework Prototype: Radar Feature Identification Hybrid of TITAN and SCIT Algorithms developed in house Works on Cartesian-gridded composite reflectivity field Ellipse fitting - used for tracking and handling mergers/splits Uses two radar reflectivity thresholds - 35 and 45 dBZ Radar features analyzed in 3-D; Tessendorf et al. (2005) used for polarimetric hydrometeor identification For testing and demonstration purposes, tracking of features done visually rather than automatically (tracking algorithm still undergoing evaluation and refinement)

6. Statistical Framework Protoype: LMA Processing Process entire scene in 10-min blocks XLMA software used to classify flashes following default Thomas et al. (2003) algorithm At least 7 stations and 2 1 required for VHF sources Flash data (including all sources) exported to IDL save files Flashes and sources then matched to individual radar features using simple space/time criteria (e.g., > 50% sources contained within feature, closest radar volume)

7. Statistical Framework Protoype: Charge Analysis Simple automation developed by Kyle Wiens, runs in XLMA Based on physical principles used in subjective hand analyses (e.g., bi-directional electric field breakdown model for lightning discharges) - Kasemir (1960), Mazur and Ruhnke (1993), Rison et al. (1999), Thomas et al. (2001), Rust et al. (2005), and Wiens et al. (2005) Assumes only two layers in flash, charge sign assigned by initial leader direction, charge altitudes set by LMA density maxima Wiens claimed 99% agreement with hand analysis for 3 June 2000 STEPS storm - charge orientation and switch location Not expected to work well with complex or sloping flashes

8. Statistical Framework Test - 11 June 2000 MCS from STEPS Analyze the evolution of these two cells: “Early Hail Cells” in Lang and Rutledge (2008; JGR)

9. Lang and Rutledge (2008)

10. Statistical Framework Radar Data Lang & Rutledge (2008) Statistical Framework reproduces basic radar trends and magnitudes up until ~22:35 UTC Reason for this discussed later

11. Statistical Framework LMA Data Lang & Rutledge (2008) Statistical Framework reproduces basic LMA trends and magnitudes up until ~22:35 UTC

12. Lang and Rutledge (2008) Statistical Framework analyzing more storm than Lang & Rutledge (2008) at this time

13. Statistical Framework CG Data Missing 3 CGs (2 neg, 1 pos) Lang & Rutledge (2008) Why the missed CGs? NLDN CGs analyzed within XLMA software - sometimes does not associate CGs with parent flashes Independent NLDN analysis should obtain these CGs

14. Statistical Framework Charge ID Statistical Framework captures basic tripolar charge structure during 2200-2210 Misidentification becomes excessive afterward Lang & Rutledge (2008) (grey +, black -)

15. Statistical Framework LMA altitude histograms sorted by charge (grey +, black -) Despite mis-IDs, bimodal positive LMA source distribution indicative of normal tripole is still apparent Charge ID algorithm needs refinement, but has good potential

16. Mass Processing Demonstration Analyze all features identified in the grids used for the analysis of the Early Hail Cells (22:00-22:40) Compute and intercompare radar (e.g., storm volume, graupel echo volume) and lightning (e.g., total flash rate) statistics Ignore features on grid boundaries 85 features identified 59 not on boundaries

17. Total Flash Rate Volume/area variables show excellent TFR correlation, with graupel echo volume where T < 0 °C being best

18. LMA Sources Similar results for LMA sources as for TFR

19. Summary Statistical Framework faithfully reproduces most results from a hand-analyzed case study, in a fraction of the time Charge ID is weakest component, but shows good potential Framework generates physically reasonable results after mass processing of a small dataset Future Work Improve charge ID and fix other issues (e.g., NLDN CGs) Test Framework on other STEPS cases Add capability for enviro, aerosol, and other data inputs