Radar Data Quality Control

1. Radar Data Quality Control Warn-on-Forecast & High Impact Weather Workshop 8 February 2012 Kevin L. Manross OU/CIMMS/NOAA/NSSL

2. Importance of Radar QC • Radar data assimilation: “Garbage in – garbage out… on steroids” • “Any source of data bias will cause bias in the resulting analysis, even if it is localized in space. For example, poorly removed clutter causes problems when assimilating radial velocity, as these cause errors in the obtained winds that persist in time and may ultimately falsely trigger instabilities…” Fabry (2011 Radar Conference)

3. 2½ Radar QC Techniques • Multiple ways to skin a cat • There are a number of ways to QC Radar data • Manually • Automated • RDA vs RPG • Signal Processing • After products Generated • Control • End user • Data Collector (radar operator) • Combination

4. RDA QC • Operate on spectral data at the Radar Data Acquisition (RDA) step • (timeseries” or “Level I”) • Examples • Notch Filter • Clutter Mitigation Decision (CMD) Algorithm • Gaussian Model Adaptive Processing (GMAP) • Staggered Pulse Repetition Time (SPRT) • Phase coding • Many others • Controlled by radar operator* • *If implemented, If operator considers need, If operator trained to do so, If…

5. 2) Product Algorithms (After the RPG) • Operate on “products” (Reflectivity, Velocity, Correlation Coefficient, etc.) after the Radar Product Generation (RPG) step • End user enhancements (Control) • Operates on gridded data • Examples • Dealiasing (Legacy, Other) • Clutter removal (AP-Remove, QCNN, CREM) • This is where our focus will be

6. 2.5) Dual-Pol • New RDA • New Products • Great at discriminating non-hydrometeors / hydrometeors • IDs ground clutter and biologicals very well • Game Changer (moving forward) KMHX 21:50Z, 10/19/2011

7. Current Radar QC on Products • Efforts at CAPS • Efforts in DART • NSSL MRMS • Comparison

8. Radar QC at CAPS

9. Reflectivity Quality Control Flowchart Read Tilt Anomalous Radial Removal All Tilts Despeckle &Median Filter (opt) Assemble Volume Ground Clutter Removal For All Elev < 1.0 Despeckle Continue to Remapping

10. KDDC Sunset & Clutter Raw Obs AnomalousRadial Removed Clutter Removal

11. Radial Velocity Quality Control Flowchart Read Tilt Spectrum Width Filter All Tilts Despeckle &Median Filter (opt) Assemble Volume Ground Clutter Removal For All Elev < 1.0 Despeckle Continue to Unfolding

12. Radial Velocity Quality Control Flowchart Model Data or Sounding …Continued Calculate Mean Wind Profile Compare to mean wind Mean Wind Profile Create perturbation Vr Field Gate-to-Gate Shear Check Quadratic Check at Gates Marked Uncertain Continue to Remapping

13. KTLX 10 May 2010 6.5° Scan Raw Obs Mean Wind Shear Check Quad Fit

14. DART Radar QC

15. Radar-Data Quality Control inData Assimilation Research Testbed (DART) System Observation rejection Observation likelihoods more than a specified number of ensemble standard deviations away from the prior ensemble mean are not assimilated. Factors: observation, observation error, ensemble mean, ensemble standard deviation Doppler-velocity dealiasing (Miller et al. 1986; Dowell et al. 2010) Velocities are locally dealiased during preprocessing (e.g., objective analysis). Final dealiasing occurs within DART immediately before the observation is assimilated (i.e., the observation is unfolded into the Nyquist-velocity bin closest to the prior ensemble mean). locally-unfolded, objectively-analyzed Doppler velocity before final DART dealiasing

16. NSSL’s MR/MS

17. NMQ “Bloom/AP Removal” Flowchart Reference: Tang et al. 2011 http://ams.confex.com/ams/35Radar/webprogram/Paper191296.html For Kevin Manross

18. NMQ Bloom/AP QC example: KCRP & KBRO 06:50Z, 10/13/2011 QCNN RAW BloomAP_QC RAW BloomAP_QC For Kevin Manross

19. NMQ Remaining challenges: bloom/AP mixed with rain For Kevin Manross

20. Implementation and Comparison of Techniques • Several techniques identified and implemented to be run in realtime • Manually cleaned cases • Comparison method: • Compare • algorithm to raw (unedited) • algorithm to truth (manually edited) • Do gate-by-gate for every elevation scan available • Track gates removed/added/changed

21. Techniques Implemented

22. Cases • Using SOLOii to manually edit • Students trained • 20090605 (VORTEX2 – strongly tornadic) • KCYS (~21-00z; 40 vol. scans; 558 elev. scans) • KFTG (~21-00z; 36vol. scans; 502elev. scans) • 20090611 (VORTEX2 – weakly tornadic) • KPUX (~22-01z; 39vol. scans; 544elev. scans) • KGLD (~23-01z; 26vol. scans; 362elev. scans) • 20110524 (strongly tornadic) • KTLX✪(~20-22z; X vol. scans; Y elev. scans) • KFDR✪(~20-22z; X vol. scans; Y elev. scans) • MPAR (~20-22z; 108vol. scans*; 1512elev. Scans) ✪In progress * Up to 19.5 deg elevation

23. Reflectivity QC Algo-Raw Truth-Algo QCNN CREM/QCNN

24. Reflectivity QC

25. Velocity QC 2D Dealiasing Legacy vs 2D 0.5 deg Legacy 2D 4.0 deg

26. Velocity QC

27. Future Work • Xu’s AR-VAD method

28. vro Variational Dealiasing Method vN vr+o Alias operator: vro =Z[vrt + o, vN] First guess bfrom combined AR-VAD analysis. Analysis a minimizes J =(a-b)TB-1(a-b) + ∑i{Z[Hia- vroi, vN]}2/o2 withvroi = vro(fi) filtered by Z[Hib– vroi, vN] ≤ (1 - a)vN, where a=¾, ½, ¼ in iteration 1, 2, 3. (Xu et al. 2009a,b Tellus) -vN Illustrative example: a Ice storm case at 04:36UTC on 1/29/09 vroat 1.5ofrom KTLX with vN= 11.5 m/s b vN raw obs dealiased vro -vN Xu et al. 2011, 2012 JTech (X11, X12 hereafter)

29. Multi-Step Hybrid Dealiasing Methodfor fine-scale vortices Basic idea Use different techniques for different scales and structures as listed below: 1. Variational dealiasing of X11 for broad areas, but flag local misfit on each tilt; 2. Block-to-point continuity check of X12for local misfit, but flag discontinuities; 3. Beam-to-beam discontinuity check for small areas with discontinuities. Tornadic caseat 22:41UTC on 5/24/2011 vroat 0.5ofrom KTLXwith vN= 28 m/s Norman raw obs Dealiased in step 3 Dealiased in step 1

30. Future Work • Dual-Pol • SPRT • If/When implemented, future is bright!

31. Questions