1 / 19

SST Quality Flags in IDPS EDR, ACSPO,OSI SAF and NASA MODIS products

SST Quality Flags in IDPS EDR, ACSPO,OSI SAF and NASA MODIS products. IDPS EDR SST Quality Flags. There are four bytes of SST Quality Flags (QF) in IDPS EDR product. QF1 byte. QF2 byte. QF3 byte. QF4 byte. IDPS EDR Quality Flags (Bits 0-1 in QF1) logic in code version Mx5.3.

kolya
Download Presentation

SST Quality Flags in IDPS EDR, ACSPO,OSI SAF and NASA MODIS products

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. SST Quality Flags in IDPS EDR, ACSPO,OSI SAF and NASA MODIS products

  2. IDPS EDR SST Quality Flags There are four bytes of SST Quality Flags (QF) in IDPS EDR product QF1 byte

  3. QF2 byte

  4. QF3 byte

  5. QF4 byte IDPS EDR Quality Flags (Bits 0-1 in QF1) logic in code version Mx5.3 If Nan for M12 -> switch to split window Not Retrived If Nan M15 or Nan M16 -> SkinQF set to Not Retrived If Ice >= Ice threshold or mask > CM_NO_SNOW -> SkinQF set to Not Retrived If mask != CM_SEA_WATER -> SkinQF set to Not Retrived If mask == CM_CONF_CLOUDY -> SkinQF set to Not Retrived SST will not be caculated for Not Retrieved category High Quality If mask == CM_CONF_CLEAR -> SkinQF set to High Quality Excluded If mask == CM_PROB_CLEAR -> SkinQF set to Excluded If mask == CM_PROB_CLOUDY -> SkinQF set to Excluded • Degraded • If SkinQF == High Quality and mask!= Adjacent Conf Clear or mask == thin Cirrus -> SkinQF set to Degraded • If satzen > satzenExclThreshold -> SkinQF set to Degraded • If SkinQF == High Quality satzen > satzenThreshold and satzen <= satzenExclThreshold -> SkinQF set to Degraded • If SkinQF == High Quality and aot >= aotDegradedThreshold and aot < aotExcludeThreshold -> SkinQF set to Degraded • Excluded • If aot >= aotExcludeThreshold -> SkinQF set to Excluded • If SST < sstLowThresh or SST > sstHighThres -> set to Excluded • If SkinQF == High Quality and SST > 305 and SST <= sstHighThresh -> SkinQF set to Degraded

  6. Summary • We have 4 bytes of quality flags QF1,QF2,QF3 and QF4. All information about SST quality conditions available in four bytes,except Adjacency Cloud Confidence and Sun Glint, was used to derive aggregated Skin SST quality information, which is sitting in the first two bits of QF1 only (0-not retrieved, 1-excluded,2-degraded,3-high quality). Another useful bit in QF1 is Day/Night indicator (last bit in QF1, 0-night,1-day). Should we use Adjacency Cloud Confidence conditions for data selection or not still questionable, because this condition dramatically reduces number of retrieved observation. After error fixing, the selection conditions should be for Best Quality -> SkinQF == High Quality Largest domain with acceptable quality -> SkinQF == High Quality or SkinQF == Degraded Quality Recommendations: 1. Fix bug in QF logic mentioned above (It looks like bug already fixed in the version Mx6.0 of code) 2.The total number of Quality flags looks too large for user and may be confusing. In most cases user will use only two bits from QF1 byte. May be after testing stage we should reduce number of QF layers in product file from 4 to 2 or even 1

  7. ACSPO mask value ACSPO mask packed into 1 byte

  8. Summary ACSPO Quality Flags are placed in one byte value. Really ACSPO is using mix of cloud detection and SST quality tests and bits 6-7 aggregate not only cloud tests results, but SST and SST anomaly valid ranges tests (anomaly is calculated against Reference SST), BTs anomaly ranges tests (anomalies are calculated against CRTM simulated BTs) and ice detection test results as well. So clear sky category corresponds to water pixels passed all tests, probably clear category corresponds to water pixels passed all tests except uniformity test, cloudy category corresponds to water pixels where at least one cloud detection or quality test wasn’t passed and all other pixels included in not retrieved category. Best Quality -> Bit 6 == 0 and Bit 7 == 0 and Bit 4 == 0 and Bit 3 == 0 Largest domain with acceptable quality -> Bit 6 == 0 and Bit 7 == 0

  9. OSI SAF Proximity confidence value (PCV) PCV are dedicated on one hand to give the user a simple mean of filtering the data and on the other hand to partition the MDB in view of deriving the sensor specific error statistics (SSES) which are delivered for every SST calculation in the L2P/L3P format. It is essential here to adopt a method similar to or compatible with those of our partners in the GHRSSTPP project. For IR derived products, the normalized PCV scale shows 6 values : 0 :unprocessed, 1 : cloudy , 2: bad, 3: suspect, 4: acceptable, 5 : excellent. The list of risk indicators were used for PCV calculation is below: • Primary cloud mask indicator: derived from the MAIA quality information available. • 2. Gradient indicator: derived from the difference between the local 11 micron • brightness temperatures gradients and the corresponding maximum climatological • values calculated from the world Atlas of thermal fronts (see figure 2). The limit value • of this quantity corresponds to the T11 noise equivalent gradient value (determined to • be 0.03K/km). The critical value is a plausible margin which is reduced in the vicinity • of cloud, so that for a pixel close to a cloud the critical value is more easily reached • than far from clouds. • 3. Stratospheric aerosol indicator: in case of need, this indicator will record the intensity • of Pinatubo like stratospheric aerosol phenomena. It will be calculated according to • (1), where the test value is the aerosol optical depth.

  10. 4. Saharan dust indicator: it is based on the use of the SEVIRI derived Saharan Dust • Index (SDI, see Merchant et al., 2006) or the NAAPS Aerosols Optical Depth (AOD, • see US NAVY, 2003) where the SEVIRI information is not available. • 5. Local temperature value indicator: the calculated SST is compared to limit and critical • values of the temperature deduced from the Casey world SST climatology (see • above), by adding margins to the local value of the minimum SST climatology. These • margins are function of the interannual standard deviation of the temperatures, of the • distance to cloudiness and of the distance to coast. • 6. Ice indicator: derived from the probability of ice calculated by applying the met.no • (Eastwood and Andersen, 2006) ice probability method, based on the use of the local • value of the IR and visible AVHRR channels north (resp. south) of 50°N (resp. 50°S) . • To calculate the PCVs, a representation of the risk factors on two axis has been adopted, • one representing the cloud mask problem, the other the algorithm known risk of errors, on a • scale from 0 (no risk) to 100 (critical risk). • In practice the first axis (X) is the maskindicator and the second (Y) is derived from the • satellite zenith angle as Y=100 (q/90). X<Xa and Y<Ya determines a PCV value of 5 • (excellent) etc.. as shown in figure above. The initial values (that will likely evolve) are the • following: Xa= 20, Xb=35 and Xc=50; Ya, Yb, Yc correspond to satellite zenith angle values • of 50, 60 and 70 degrees. Yc is off the limit of the AVHRR so practically no “bad” label is • derived from the satellite zenith angle value.

  11. Summary • OSI SAF PCV aggregates cloud mask results, SST quality tests results (risk factors) and observation geometry ranges in plain integer number in the range 0-5. Selection criteria are below. • Best Quality -> PCV == 5 • Acceptable Quality -> PCV >= 4

  12. MODIS SST Quality Levels A series of quality tests are performed for each sst or sst4 retrieval. The quality tests are used to set the quality levels, which are then used to control the Level-3 binning process. For the msl12 implementation, each quality test was assigned a bit in a product-specific flag array. A separate, 16-bit flag product was created for both the short-wave (sst4) and long-wave (sst) products (flags_sst4 and flagssst, respectively). The 16 flag bits were assigned as follows:

  13. ISMASKED - Set if the SST processing is not performed because the pixel was masked prior to invocation. The msl12 code allows the user to specify a number of masking conditions. For standard SST processing, the only condition which would likely be selected for masking by msl12 at this stage is if the pixel is over land. BTBAD - Set if the observed radiances are beyond the limits of the radiance to brightness temperature tables, such that brightness temperatures can not be determined. This generally indicates saturation of one of the critical IR channels. BTRANGE - Set if one of the brightness temperatures falls outside the physically realistic range for ocean observations. The currently accepted range is -4 to 33 deg-C. BTDIFF - Set if the brightness temperature difference falls outside the physically realistic range for ocean observations. For long-wave SST, dBT = BT11 - BT12 and the currently accepted range for dBT is 0 to 3.6 deg-C. For short-wave SST, dBT = BT39 - BT40 and the currently accepted range for dBT is 0 to 8 deg-C. SSTRANGE - Set if the SST retrieval falls outside the physically realistic range for ocean observations. The currently accepted range is -2 to 45 deg-C. SSTREFDIFF - Set if the absolute difference between the SST retrieval and the sstref (oisst) value exceeds 3 deg-C. This test is problematic in regions of high spatial variability (e.g., frontal boundaries), as the sstref field is very low in spatial resolution and smoothed over time. SST4DIFF - This test is only applicable at night. Set if the absolute difference between the long-wave and short-wave SST retrieval exceeds 0.8 deg-C. SST4VDIFF - This test is only applicable at night. Set if the absolute difference between the long-wave and short-wave SST retrieval exceeds 1.0 deg-C. BTNONUNIF - Set if one of the required brightness temperatures shows evidence of spatial non-uniformity. The uniformity is determined by examination of the 3x3 pixel area around the pixel of interest. If the difference between the maximum value and the minimum value in that 9-pixel set exceeds 0.7 deg-C, the bit is set. This test does have a tendency to flag frontal boundaries and coastlines, as can be seen in this animation.

  14. BTVNONUNIF - Set if one of the required brightness temperatures shows a high degree of spatial non-uniformity. The test is identical to that of BTNONUNIF, but with a larger threshold. If the difference between the maximum value and the minimum value in the 9-pixel set exceeds 1.2 deg-C, the bit is set. • BT4REFDIFF - This test is only applicable to the short-wave SST retrieval. The test compares the brightness temperature difference (dBT=BT39-BT40) against a supplied reference temperature, where the reference is provided as a function of scan pixel (basis unknown by author). A plot of the dBTref is shown here. If the difference between dBT and dBTref falls outside a specified range, the bit is set. The currently acceptable range is -1.1 to 10.0 deg-C. • REDNONUNIF - This test is only valid for daytime, and therefore only relevant to the long-wave SST product. Top-of- atmosphere reflectance (rho_t) in the 678-nm band (MODIS band 14) is computed over the 3x3 pixel area centered on the pixel of interest, where rho_t = pi*Lt/F0/(mu0 t t0 toz) and Lt is observed TOA radiance, F0 is band-averaged solar irradiance (at day of year), mu0 is cosine of solar zenith angle, t0 and t are the diffuse transmittance through a Rayleigh atmosphere (solar path and sensor path), and toz is the ozone transmittance (inbound and outband). If the the difference between the maximum value and the minimum value of rho_t in the 9-pixel set exceeds 0.01, the bit is set. This bit is also set if 8 or more of the 9 pixels are saturated in the 678-nm band. In general, such saturation might indicate the presence of clouds, but it may also indicate the presence of sun glint. The long-wave SST is affected by clouds (SST retrieval appears colder than normal), but not by sun glint. To recover the sun glint case, the REDNONUNIF bit is only set if the retrieved SST is more than 1 deg-C colder than the reference. This secondary requirement works best in locations with temporally and spatially stable SST conditions, where the low-resolution sstref and the retrieved SST can be expected to be consistent. The saturation test is a much more stringent test than the original uniformity test. The new test is can be summarized as: set if red band reflectance in the pixel neighborhood is (saturated OR spatially nonuniform) AND SST retrieval is cold relative to the reference. • HISENZ - Set if the sensor zenith angle exceeds 55 degrees. For msl12, this is redundant with the HISATZEN bit in the l2_flag array, but with a different standard threshold • VHISENZ - Set if the sensor zenith angle exceeds 75 degrees. This is rare. • SSTREFVDIFF - Set to indicate that the difference between the retrieved SST and the reference is very large (6-deg C). The related flag, SSTREFDIFF, indicates that the difference between the retrieved SST and the reference is moderately large (3-deg C).

  15. The quality tests described above are used to set quality levels between 0 and 4, where 0 indicates best quality and 4 indicate complete failure or masked (usually land). The quality level determination varies between day and night conditions, and between the short-wave and long-wave SST products. The following tables show the quality test bits and associated quality levels. If no bits are set then the quality level is 0, but for short-wave SST retrievals in daylight the quality level is always set to 3 (bad) or 4 (failed or not computed). The quality level information for each SST product, sst and sst4, can be output by msl12 as products qual_sst and qual_sst4, respectively. Daytime Long-Wave SST

  16. After the standard assessment based on the flag bits, if the retrieved SST is more than 1 deg-C colder than the reference (oisst) an additional test is done to determine if the reflectance at 678nm is unusually high (rho_t > 0.05). If yes, and the quality level is below 3, it is incremented by 1 (i.e., quality is reduce one level). Nighttime Long-Wave SST

  17. There are two extra bits useful for users in l2_flags layer SSTFAIL - If the sst (or sst4) product is requested for output, this bit will be set in l2_flags if qual_sst (or qual_sst4) is greater than or equal to 4. SSTWARN - If the sst (or sst4) product is requested for output, this bit will be set in l2_flags if qual_sst (or qual_sst4) is greater than or equal to 1.

  18. Summary I didn’t find in NASA MODIS document how cloud mask information combined with quality tests results, but I suppose , that clear sky condition may be used with quality levels as below QL (qual_sst in data files) content values of Quality levels Best Quality -> Clear sky and QL == 0 (should be equivalent SSTWARN == 0 and SSTFAIL == 0 in l2_flags) Largest domain with acceptable quality-> Clear sky and QL <= 1

More Related