1 / 42

Enterprise Cloud Phase Provisional Maturity Review Report

This report outlines the Enterprise Cloud Phase (ECP) Provisional Maturity status as of October 2, 2018, detailing its features, evaluation, NDE status, and comparison with other sensors. It provides insights into the path forward for full validation maturity and future plans. The report summarizes conclusions and recommendations for improving performance based on findings from evaluations against NASA CALIPSO data and GOES-16 phase comparisons.

castleman
Download Presentation

Enterprise Cloud Phase Provisional Maturity Review Report

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. VIIRS Cloud PhaseTeam Michael Pavolonis(STAR),Corey Calvert (CIMSS), Jason Brunner (CIMSS) NOAA-20 VIIRS Enterprise Cloud Phase (ECP)ProvisionalMaturity October 2, 2018

  2. Outline • Description • Status in NDE • SDR Issues • Evaluation • Provisional Maturity Conclusions • Path Forward to Full Validation Maturity • Future Plans

  3. STAR ECPCal/Val Team

  4. Enterprise Cloud Phase Review

  5. Enterprise Cloud Phase • Supports many sensors and its part of the NOAA Enterprise Algorithm Suite. • It uses NASA CALIPSO data for its training. • The primary output is the cloud phase (integer values 0 – 5). • Enterprise phase is determined using multiple threshold-based radiometric tests. • The demand for one algorithm to serve many sensors drove the ECM development.

  6. How to Use the Enterprise Cloud Phase • The fundamental output of the ECP is the cloud phase. • The cloud phase output corresponds to the following cloud phase categories • 0 – Clear • 1 – Liquid water phase • 2 – Supercooled water phase • 3 – Mixed phase • 4 – Ice phase • 5 – Unknown

  7. ECP Heritage • ECP has run for years on GOES in OSPO and other sensors in STAR. • Here is an example of the NOAA Enterprise Cloud Phase applied to GOES-16.

  8. ECP Channels • ECP uses the following channels • M14 • M15 • M16

  9. Users of the ECP • Enterprise Cloud Algorithms. • VIIRS Polar Winds. • Potentially External VCM Users (we know of none).

  10. Enterprise Cloud Phase NDE Status

  11. NDE/STAR VIIRS ECP Production Status

  12. SDR Issues

  13. SDR Issues • No known issues.

  14. Evaluation of the NDE ECP

  15. Requirements • Measurement accuracy of 80% correct classification for Cloud Phase for clouds with an optical depth >1.0

  16. JPSS Data Products Maturity Definition

  17. Evaluation Methodology We have chosen independent sources of cloudiness that provide qualitative and quantitative analysis of the performance over a short time. Our Specific Evaluation Methodology applied here: • Validation against NASA CALIPSO/CALIOP • Comparison NDE ECP to GOES-16 phase from the ground system • Visual inspection of NDE ECP with ECP using S-NPP

  18. Data Used in this Analysis • NOAA-20 NDE v1r2 from June 3 and September 12, 2018 for GOES-16 and S-NPP Comparisons. • GOES-16 ground system output from June 3, 2018. • S-NPP NDE v1r2 from September 12, 2018. • 3days for CALIPSO Comparison: June 19, July 5 and Sept. 4, 2018.

  19. Comparison to CALIPSO/CALIOP

  20. CALIPSO vs NOAA-20 Comparison Description • CALIOP is a lidar, with depolarization, on board the CALIPSO satellite in the NASA A-Train. • The CALIOP 1 and 5 km vertical feature mask products are merged to derive the cloud phase of the highest cloud layer with CALIOP 532 nm optical depth > 0.01 • 3 days of CALIOP and NOAA-20 Matchup data are used • ????? (these were for GOES-16) Filters applied to NOAA-20: • Scan time difference ± 7.5 minutes, • Sensor Zenith Angle < 70.0 • NOAA-20 and CALIOP agree that a cloud is present • Validation analysis is a function of the CALIOP 532 nm cloud optical depth

  21. CALIPSO vs NOAA-20 Validation The NOAA-20 cloud top phase product meets the accuracy specification for optical depths < 1.0, but is slightly below for optical depths >1.0. Adding additional CALIOP match-ups has little impact on bulk stats (law of large numbers). Requirement

  22. NOAA-20 Liquid Phase Validation

  23. NOAA-20 Supercooled Phase Validation

  24. NOAA-20 Mixed Phase Validation

  25. NOAA-20 Ice Phase Validation

  26. NOAA-20 Single Layer Ice Phase Validation

  27. NOAA-20 Multi-Layer Ice Phase Validation

  28. Conclusions from CALIPSO Comparisons • The ECP NDE meets the 80% accuracy requirement for clouds with optical depths <1.0, but is slightly below the required specification for clouds with optical depths >1.0. • Overall, ECP NDE shows a good performance, even though it is slightly below the required specifications. • Updated threshold training specific to NOAA-20 will likely improve the performance

  29. Comparison with GOES-16

  30. NOAA-20 vs GOES-16 Comparison • NOAA-20 phase was co-located with GOES-16 phase for June 3, 2018. • Co-located pixels where both NOAA-20 and GOES-16 indicated cloudy pixels and the GOES-16 viewing angle was <60 deg were used • 152,758,528 pixels were used in this analysis

  31. NOAA-20 vs GOES-16 Comparison • Red circle highlights area where NOAA-20 phase returns a higher number of mixed phase pixels where GOES-16 phase indicates ice phase • This specific difference in phase classification between GOES-16 and NOAA-20 shows up in the confusion matrix and appears to occur most often near cloud edges

  32. NOAA-20 vs GOES-16 Comparison • The lack of ‘clear’ pixels in the NOAA-20 cloud phase was the result of a threshold issue in the ECM when determining the 4-level cloud mask (this issue has since been resolved) • The ECM team recommends a threshold-based approach using the cloud mask probability product over the 4-level cloud mask • The ECP algorithm should be updated to utilize the ECM as recommended

  33. Conclusions from GOES-16 Comparisons • Overall the NOAA-20 ECP compares well to GOES-16 • NOAA-20 ECP returns a higher amount of mixed phase pixels when GOES-16 detects ice, mostly near cloud edges • Bug in the NOAA-20 ECM 4-level product was identified and has been corrected (June 2018 DAP)

  34. S-NPP and NOAA-20 Continuity

  35. S-NPP and NOAA-20 Continuity • Animation composed of sequential cloud phase images from NOAA-20 and S-NPP for September 12, 2018 • There are still a large number of NOAA-20 granule outages • Other than the granule outages there do not appear to be any major artifacts between the two phase products

  36. Investigation of Downstream Issues

  37. Investigation of Downstream Issues • The overall Enterprise Cloud Phase Algorithm works well • However, when the cloud phase is wrong, it has a large impact on the cloud height retrieval. • There are 2 potential issues the phase team may want to track • Ice cloud phase on the edges of water cloud • Determination of ice phase when NIR imagery indicates otherwise • Previously published research has shown that these issues can be mostly mitigated by incorporating near-infrared measurements. The use of near-infrared measurements is complicated by highly variable surface reflectance and sun geometry. Thus, the mitigation strategy adds complexity to the algorithm.

  38. Provisional Maturity Conclusions • There are still issues with the NDE ECP v1r2 on NOAA-20. • The accuracy requirement of 80% accuracy of cloud classification for clouds with optical depths >1.0 has not yet been achieved • The NDE ECP accuracy for clouds with optical depth >1.0 was 78% • The NDE ECP accuracy for clouds with optical depth >2.0 was 74% • NOAA-20 specific threshold tuning should improve performance • Incorporating near-IR channels may also help, but the additional complexities this would create requires further guidance on whether to proceed • The Cloud Team recommends Beta Maturity at this time.

  39. Pathway to Full Validation • We expect to apply the same activities to be conducted for Full Validation Maturity: • We are gathering an archive of golden days where we save SDRs and EDRs over a long time period. This collection is underway. • We will work with the other teams this winter on application-specific analysis. • We will take advantage of opportunities for threshold adjustments.

  40. Risks for Full Validation Currently outstanding issues, unless fixed by handover, may prevent declaration of Full Validation Maturity: • NDE processing issues (Moderate) • Missing granules in NDE processing • Addressed in August 2018 delivery (v2r0) • Fix currently running in I&T string (as of 28 Sept 2019). Expected operations in late 2018 • Accuracy of <80% correct classification for Cloud Phase for clouds with an optical depth >1.0 (Moderate) • Update NOAA-20 specific phase test thresholds to improve performance • Update how the ECM is utilized in the ECP by using the cloud probabilities rather than the 4-level masks • NOAA-20 specific ECP Threshold Updates • Training requires data from all seasons. Expected delivery by April 2019 DAP

  41. Future Plans of ECP • Update the use of the 4-level cloud mask to a threshold-based cloud mask using the cloud probabilities • Use large NOAA-20 dataset encompassing all seasons to fine tune the cloud phase thresholds • Determine the benefit of adding near-IR channels to improve algorithm performance and minimize issues created in downstream algorithms

  42. Backup Material

More Related