CLEAN-AP Update Cl utter E nvironment An alysis using A daptive P rocessing

1. CLEAN-AP UpdateClutter Environment Analysis using Adaptive Processing Sebastián Torres and David Warde CIMMS/The University of Oklahoma and National Severe Storms Laboratory/NOAA NEXRAD TACNorman, OK29 August, 2012

2. what is CLEAN-AP? CLEAN-AP is a novel real-time, automatic, integrated technique for ground clutter detection and filtering that produces meteorological data with improved quality while meeting NEXRAD technical requirements.

3. what is CLEAN-AP not? While CLEAN-AP can deal with normal- (NP) and anomalous-propagation (AP) clutter, it is not a mitigation technique for moving clutter such as airplanes, cars, or wind turbines.

4. milestones (I) • Developed concept (Spring ‘08) • Filed invention disclosure with OU’s OTD • Evaluated performance using simulations (Spring ‘08) • Implemented in real time on the NWRT PAR (Fall ‘08) • Compared with current WSR-88D clutter mitigation (Fall ‘09) • KEMX (Tucson, AZ), KTLX (Oklahoma City, OK), KABX (Albuquerque, NM), KCRI (ROC testbed) • Presented at November 2009 TAC meeting • initially presented CLEAN-AP • “… the technique shows potential utility…”, “… encouraged by early results…” • “… [have not] shown the scientific details behind the algorithm…” • “… [need] case comparisons with CMD…” • Extended to dual-polarization (Spring ‘10) • KOUN (S-band) and OU’ (C-band) • Developed clutter model (Spring ‘10) • Form/Function/Fit for RVP8

5. milestones (II) • Presented at fall 2010 ROC/NSSL/NCAR TIM • presented more technical details and data cases • received positive feedback from ROC eng and agency reps • Delivered CLEAN-AP algorithm description (Fall ’10) • NSSL Report 14 (confidential attachment) • Extended to staggered PRT (Fall ‘10) • Presented at March 2011 TAC meeting • decision briefing • “The TAC members agreed to move forward with conducting an engineering evaluation on CLEAN AP.” • Documented CLEAN-AP performance (Winter ’11) • NSSL Report 15 • Delivered SPRT CLEAN-AP algorithmdescription (Spring ’12) • NSSL Report 16 (confidential)

6. clutter mitigation at the RDA Ground Clutter Detection CLEAN-AP CMD Control Ground Clutter Filtering GMAP Filtereddata Unfiltereddata

7. CLEAN-AP’s core: the ASDlag-1 autocorrelation spectral density window & DFT • For periodic signals: F1(k) = F0(k) ej2pk/M • S1(k) = F0*(k) F0(k) ej2pk/M =S (k) ej2pk/M • the ASD is the PSD with linear phase F1 V1 S1(k) = F0*(k)F1(k) 0 1 M-2 M-1 sample time V0 window & DFT F0

8. nothing is ideal… True Measured PSD ASD ASD v v v v v v Leakage Leakage Ground Clutter v v v v PSD ASD ASD Weather Spectral leakage in the measured ASD is the basis for clutter detection in CLEAN-AP

9. how does CLEAN-AP work? Integrated Detection and Filtering Estimate CNR Select appropriate data window Compute ASD Identify components with clutter contamination Phase of ASD is near zero due to leakage effect Remove clutter Reconstruct weather ASD is used to estimate meteorological variables |ASD| Detection PSD v v Arg(ASD) Filtering zero phase DC

10. Performance Summary

11. data window selectionCLEAN-AP vs. GMAP lower variance better suppression GMAP % More aggressive data windows containspectral leakage but increase variance of estimates

12. notch width selectionCLEAN-AP vs. GMAP NW determination lower bias better suppression CLEAN-AP uses magnitude and phase for improved notch width determination

13. Clutter SuppressionDoppler Mode CLEAN-AP GMAP CLEAN-AP has moreclutter suppression and lessvariance of estimates 

14. Reflectivity (Power) BiasDoppler Mode CLEAN-AP GMAP CLEAN-AP has a more robust notch widthdetermination that results in smaller biases

15. Case Examples

16. KEMX Tucson, AZCan you see the mountains? • Ground-clutter suppression • Due to the mountainous terrain close to the radar, KEMX data exhibited “hot spots” (clutter residue) after the initial release of CMD (B11). This led to a few software changes (B11 fixed)

17. KEMX Reflectivity Unfiltered CLEAN-AP CMD+GMAP (B11 fixed) CLEAN-AP mitigates both weak and strong clutter contamination

18. KEMX Tucson, AZCLEAN-AP Performance CLEAN-APPower Removed CLEAN-APWindow Used CLEAN-AP adapts to clutter environment

19. KCRI Norman, OKWhat happened to the zero-isodop? • Zero-isodop loss • Weather with narrow spectrum width and near zero velocity has nearly the same spectrum as clutter if it looks like a duck…

20. KCRIReflectivity CLEAN-AP CMD+GMAP (B11 fixed) Unfiltered Unfiltered CLEAN-AP better preserves the zero isodop

21. KCRIVelocity CLEAN-AP CMD+GMAP (B11) Unfiltered Unfiltered CLEAN-AP better preserves the zero isodop

22. Future Work • Support implementation on the ORDA • Support engineering evaluation • Investigate using dual-pol information to improve detection performance • Investigate using H and V channels to determine the filter’s notch width • This also applies to the current filter

23. Summary CLEAN-AP is a novel real-time, automatic, integrated technique for ground clutter mitigation that produces meteorological data with improved quality while meeting NEXRAD technical requirements. In March 2011, the TAC endorsedits engineering evaluation on the WSR-88D.

24. back up slides

25. the CLEAN-AP filter (I) • CLEAN-AP is automatic • no need for user intervention • real-time ground-clutter detection • no need for clutter maps • CLEAN-AP produces data withimproved quality • adaptive data windowing finds a good compromise between clutter suppression and data quality • CLEAN-AP meets NEXRAD requirements • improved clutter suppression • requirements (Z) met with as few as 8 samples • CLEAN-AP is integrated • one algorithm for ground clutter detection and filtering • gate-by-gate operation

26. the CLEAN-AP filter (II) • CLEAN-AP “sets the stage” for further spectral processing • phase provides useful information • immune to biases from circular convolution • CLEAN-AP is operational on the NWRT PAR • running in real-time since Sep 2008 • performance informally evaluated by meteorologists and forecasters (PARISE experiments) • CLEAN-AP is compatible with current and future upgrades • SZ-2, super resolution, dual pol., hybrid SW, • SPRT, range oversampling, etc.

27. KEMX Clutter Map (PwrRmvd > 10 dB) Clutter Map (PwrRmvd > 20 dB)

28. KTLX Oklahoma City, OKWhere is the zero? • Snow event with embedded storms

29. KTLXReflectivity Unfiltered CLEAN-AP Where is thezero isodop?

30. KTLXVelocity Unfiltered CLEAN-AP No detectable zero isodop loss

31. KOUNWhat about dual polarization? Differential Reflectivity Differential Phase Correlation Coefficient Reflectivity CLEAN-AP is compatible with dual polarization