The ACSPO Clear-Sky Mask (ACSM)

1. The ACSPO Clear-Sky Mask (ACSM) B. Petrenko1,2, A. Ignatov1, P. Dash1,3, Y. Kihai1,2, X. Liang1,3 1NOAA/NESDIS/STAR 2Global Science and Technology, Inc. 3Cooperative Institute for Research in the Atmosphere

2. General Concept of ACSM • The Advanced Clear-Sky Processor for Oceans (ACSPO) is developed at NESDIS to process AVHRR-like satellite infrared observations for SST, clear-sky radiances and aerosol. • The predecessor of ACSPO is “Clouds from AVHRR, extended” (CLAVR-x) - a classical set of cloud tests, based on existing notions on radiative transfer in clouds (RTC). • The ACSPO Clear-Sky Mask (ACSM) has evolved from relying on cloud tests to more extended use of online clear-sky Community Radiative Transfer Model (CRTM) and a priori information on SST and the atmosphere. • In ACSPO, the cloud masking problem is posed as a check of retrieved SST for adequacy to CRTM with a priori restrictions on retrieved SST. • In addition,reflectance–based cloud filters are used during the day.

3. Flow Chart of ACSPO GFS atmospheric data Analysis SST (e.g., Reynolds) Observed BT CRTM: simulations SST retrieval (in ALL ocean pixels) Estimation of biases in SST and BTs wrt analysis fields Clear-Sky Mask BT filter SST filters Reflectance filters (day only) • SST retrieval can use regression or RTM–based algorithms • Accounting for biases in SST and BT reduces sensitivity of ACSM to SST algorithm deficiencies and calibration trends • Several ACSM filters use SST as a predictor

4. Estimation of biases in BT and SST • Calibration trends can change biases in observed BTs and retrieved SSTs. • These biases can affect ACSM filters, which use BT and SST as predictors • ACSPO estimates the biases before ACSM as positions of maxima of histograms, accumulated over all ocean pixels during the past ≈24 hours. • This way of bias estimation prevents cross-talk between the estimated biases and results of clear-sky masking

5. Comparison of two ways of SST bias estimation (MetOp-A AVHRR GAC, August 1-7 2008) • Dashed curves - biases, estimated as average SST anomaly over “clear” pixels • Solid curves - positions of peaks of TS-TS0 histograms over all sea pixels.

6. The ACSM Filters and Output Filters using reflectance bands (daytime only): Reflectance “Gross” Contrast filter (RGCT) Reflectance “Relative” Contrast filter (RRCT) SST/Reflectance Cross-Correlation Filter Filters using thermal bands only: • BT filter • “Static” SST filter • “Adaptive” SST filter • SST Uniformity filter All ACSM filters are binary, i.e., the output is either “Clear” or “Cloudy”

7. Brightness Temperature Filter [TBi-TB0i(TS)-DBTi]cij [TBj-TB0j(TS) -DBTj]< δ ? If yes, the pixel is clear, otherwise it’s cloudy TB0i(TS)=TB0i(TS0) + Gi(TS-TS0) is Taylor expansion of CRTM BT, TB0iis CRTM BT in channel i, TS0 is first guess SST, Gi is derivative of TB0iin terms of TS TSis retrieved SST, δis tolerance(threshold), DBT is estimated bias in TB-TB0 • The BT filter was found to produce false cloud detections, due to abnormal water vapor distributions in Tropics • In the latest ACSPO version 2.2, this filter was significantly relaxed, and the emphasis was made on filters using SST as a predictor • Requires further work

8. Static SST filter (Initial Classification) TS-TS0-DSST>μ? If yes, then the pixel is “Clear”, otherwise it’s “Cloudy” TS is retrieved SST TS0 is first guess SST DSST is estimated bias in TS-TS0 μ is variable threshold • μ depends on variability of ΔTB in the neighborhood of a given pixel • ΔTB =TB11—TB12during day and ΔTB =TB3.7—TB12 at night • Higher variability of ΔTB indicates that cold SST anomalies are likely due to clouds • If the variability of ΔTB is low then μ=-2K, otherwiseμ=-4K.

9. Adaptive SST filter (Subsequent Classification) • Initial classification by the Static SST filter • The filter analyzes statistics of TS-TS0-DSSTwithin the sliding window surrounding a given pixel: • The statistics are calculated separately for “Clear” and “Cloudy” pixels and compared; • Based on this comparison, some “Clear” pixels within the window are reclassified into “Cloudy”; • The process is repeated iterationally until no new reclassifications happen • The Adaptive SST filter is a spatial filter, using iterations within a relatively big window (7×7 for AVHRR GAC, 21×21 for VIIRS). • This is one of the main ACSPO time consumers. • Recently, it was modified for processors using parallel computations.

10. 10/06/2012, Pacific Ocean near Japan, Night.VIIRS BT3.7-BT12, and SST-Reynolds, no Cloud Mask SST – Reynolds BT3.7 – BT12 • In the dynamic zones, SST anomalies are large enough to be confused with clouds. • Higher variability of BT difference between bands 3.7 and 12 μm (11 and 12 μm) is typical for cloudy areas, whereas in the clear-sky areas it is relatively low. • This feature is used to set up the threshold for the Static SST test

11. 10/06/2012, Pacific Ocean near Japan, Night.Spatial SD of VIIRS BT3.7-BT12, and SST-Reynolds with Cloud Mask SST – Reynolds, cloud mask Spatial SD of BT3.7 – BT12 μ=-4 K μ=-2 K • In the areas marked red in the left image, the static SST threshold μ=-2 K. In other areas, μ=-4 K. • This resulting cloud mask effectively masks clouds but preserves some large negative SST anomalies up to -4K.

12. MetOp-A, Feb 3, 2013, SST – Reynolds South-East off Australia, Night ACSPO, no adaptive SST filter ACSPO, with adaptive SST filter OSI-SAF • The Adaptive SST filter masks ambient clouds and some cold SST anomalies • OSI-SAF mask extensively rejects pixels neighboring to clouds

13. MetOp-A, Feb 3, 2013, SST – Reynolds in Mexican Gulf, Day ACSPO, no adaptive SST filter ACSPO, with adaptive SST filter OSI-SAF • The Adaptive SST filter masks out ambient clouds and (probably) some cold SST anomalies • OSI-SAF mask allows obvious cloud leakages

14. Uniformity SST filter • The ACSPO Uniformity SST filter is the analogue of texture filters often used to screen out subpixel clouds. mean{[X-mean(X)]2} < η? If yes then the pixel is clear; If not, it’s cloudy - mean(*) is spatial average over a sliding window - ηis threshold - X is BT or SST • In ACPO, X = SST – median(SST), median(SST) is spatial median of SST over a sliding window. • Conventional Texture filters often hide high SST gradients • The ACSPO Uniformity filter improves discrimination between subpixel clouds and SST fronts

15. Performance of Conventional SST Uniformity Filter (Aqua MODIS, Night) Predictor with X=SST Cloud Mask • Left – The predictor of conventional texture filter increases at high SST gradients • Right – This causes the filter to mask out high SST gradients

16. Performance of ACSPO Uniformity Filter (Aqua MODIS, Night) Predictor with X=SST-median(SST) Cloud Mask • Left – The predictor of ACSPO uniformity filter is insensitive to high SST gradients • Right – This preserves high SST gradients

17. Performance of ACSM Uniformity Filter (S-NPP VIIRS, 08/24/2013, California coast, Night) All filters except Uniformity All filters including Uniformity The ACSM Uniformity filter adds a lot to the cloud mask but preserves high SST gradients

18. MetOp-A, Feb 3, 2013, SST anomalies in Bay of Bengal: ACSM vs OSI-SAF, Night ACSPO OSI-SAF • The ACSM preserves high SST gradients • The OSI-SAF cloud mask does not preserve SST contrasts

19. SST/Albedo Cross-Correlation (CC) Filter • The CC filter detects subpixel cloud effects below the Uniformity filter threshold by small negative deviations of SST from spatial average, correlated with variations in albedo in one of optical bands. • The CC predictor is a part of SST variance, explained by variation in albedo: r2D <γ? If yes then the pixel is “clear”, If not then it’s “cloudy” r is spatial correlation between SST and albedo D is spatial variance of SST γis threshold

20. S-NPP VIIRS, Day:The Mask Includes all Filters except CC SST - Reynolds M7 Albedo • Residual clouds are still noticeable due to higher albedo and colder SST

21. Same scene, S-NPP VIIRS, Day:The Mask Includes all Filters including CC SST - Reynolds M7 Albedo • The CC Filter masks out small cloud effects, which could not be detected otherwise • The filter is effective in the sun glint area • CC filter is insensitive to striping noise in SST, as long as this noise not correlated with albedo

22. SQUAM Statistics of ACSPO SST and OSI-SAF SST wrt Reference SST (MetOp-A, Feb 3, 2013, Day) • ACSPO SST is more precise with respect to both Reynolds and OSTIA • ASPO SST precision is less sensitive to reference field • ACSPO produces slightly more clear pixels than OSI-SAF

23. SQUAM Statistics of ACSPO SST and OSI-SAF SST wrt Reference SST (MetOp-A, Feb 3, 2013, Night) • OSI-SAF SST is more precise, especially with respect to OSTIA • ACSPO produces 12% more clear pixels • ACSPO SD is less sensitive to reference field

24. Further work • Accelerating ACSPO • The ACSM filters extensively use spatial windows. This makes them time consuming, especially with high-resolution imagers like VIIRS. • Processing of 10 min. VIIRS granule with ACSPO v.2.1 takes 3 to 5 min. • Currently, the ACSPO algorithms are being modified for parallel processing. The processing time of 10 min. VIIRS granule is expected to reduce to 1.5 - 2 min. • Developing cloud filters based on pattern recognition • In many cases the human eye better distinguishes between cloud leakages and cold SST anomalies than the ACSM does. • Recently, we have began exploring the potential of pattern recognition techniques for cloud masking (In collaboration with Irina Gladkova from NY .

25. Backup slides

26. Example of SST anomalies images after Static and Adaptive SST filters (VIIRS, 24 August 2012, New England coast, night) Static + Adaptive SST filters No mask Static SST filter only • The Static SST filter performs initial screening of cloud SST anomalies • The Adaptive SST filter masks out ambient clouds • Some large negative SST anomalies are left unmasked based on analysis of TB11-TB12