Passive Microwave Data Set Management Workshop: Preliminary Results

1. Passive Microwave Data Set Management Workshop:Preliminary Results A review of the passive microwave radiometer data sets at Earth Science Data Centers supported by NASA Workshop Committee Members: Ed Armstrong, Helen Conover, Michael Goodman, Brian Krupp, Zhong Liu, John Moses, Rama Ramapriyan, Donna Scott, Deborah Smith, Ron Weaver AMSR-E Science Team Meeting Asheville, NC

2. Summary of Workshop Goals A Passive Microwave Data Set Workshop was held 17-19 May 2011 at the Global Hydrology Resource Center, sponsored by the NASA Earth Science Data and Information System (ESDIS) Project. Data producers and data distributors met in order to organize and document passive microwave radiometer data sets available at NASA Earth Science Data Centers (ESDCs) with the purpose of better informing users of data set choices. We reviewed all primary (Level 1 – Level 3) passive microwave data sets from NASA and non-NASA sensors held by ESDCs or NASA-funded data producers. Primary goals of the workshop were: • Determine if products are duplicative and, if so, why each specific product is important to a given user community. • Discuss possible changes to levels of service for any products without an active user community. For example, certain data sets may be maintained because they were inputs for generating a historical geophysical product. • Discuss ways in which we can more clearly describe and document the passive microwave data sets held by the ESDCs and other Data Providers funded by NASA. The intent is to harmonize semantics, formats, and documentation to the benefit of users. AMSR-E Science Team Meeting Asheville, NC

3. Attendees Data Centers ESDIS Data Producers * Participating remotely AMSR-E Science Team Meeting Asheville, NC

4. Data Providers Considered • The various NASA Earth Science Data Centers (ESDCs): • Atmospheric Science Data Center (ASDC) – primarily focusing on atmospheric chemistry products, but also holding some heritage passive microwave data • Global Hydrology and Climate Center (GHRC) – focusing on the atmospheric components of the hydrologic cycle • Goddard Earth Sciences Data and Information Services Center (GES DISC) – archive of record for the TRMM Microwave Imager products, among others • National Snow and Ice Data Center (NSIDC) – focusing primarily on cryospheric products, AMSR-E archive of record • Physical Oceanography DAAC (PO.DAAC) – extensive holdings of sea surface temperature and other ocean products • Other NASA-funded data production / distribution facilities: • Precipitation Processing System (PPS) – processes, analyzes and archives data from the upcoming GPM and the current TRMM mission • Remote Sensing Systems (RSS) - currently reprocessing all microwave satellite radiometers to a common calibration standard using their newest radiative transfer model • Precipitation Research Group (PRG) at Colorado State University – currently developing a Fundamental Climate Data Record (FCDR) of SSM/I and SSMIS brightness temperatures.   AMSR-E Science Team Meeting Asheville, NC

5. Data Disciplines and Services Considered Disciplines • Brightness Temperature Data Sets • Atmospheric Data Suites (Wind, Vapor, Cloud, Rain) • Additional Precipitation Data Sets • Additional Wind Data Sets • Ocean Flux Data Sets • Sea Surface Temperature Data Sets • Cryosphere / Ice Data Sets • Soil Moisture and Land Data Sets Services • Subsetting, Data Extraction, Other Data Services AMSR-E Science Team Meeting Asheville, NC

6. Summary of Data Holdings (X) Not publicly available AMSR-E Science Team Meeting Asheville, NC

7. Workshop Agenda Tuesday, 17 May 8:30a   Welcome and Introductions (M. Goodman) 9:00a   Overview of Passive Microwave Radiometer products at each data center (RSS, GHRC, NSIDC, PO.DAAC, GES DISC, ASDC) 1:00p    Introduce data review process for workshop (M. Goodman) 1:15p    Review data group 1:  Atmosphere over Oceans (moderator D. Smith; scribe R. Weaver) 3:00p    Review data groups 2-4:  Other atmosphere and/or ocean products • Wind (moderator H. Conover; scribe J. Moses) • SST  (moderator H. Conover; scribe B. Krupp) • Precipitation (moderator M. Goodman; scribe B. Krupp) Wednesday, 18 May 8:30a   Review data group 5:  Brightness Temperatures (moderator R. Weaver; scribe H. Conover)10:15a   Review data groups 6 and 7:  Cryosphere products, Land products (moderator W. Meier; scribe D. Smith) 12:45p   Other passive microwave data sources (moderator B. Krupp; scribe M. Goodman) • NASA Passive Microwave data not previously covered • NOAA Passive Microwave data holdings • International Passive Microwave data holdings 1:30p  Subsets, dynamic products, tools (moderator J. Moses; scribe M. Anderson) 3:00p  Additional resources and data stewardship activities • Data identifiers (A. Leon) • Provenance and Context Content standard for archive (J. Moses) 3:45p  Discussion - recommendations / future work (moderator M. Goodman; scribe H. Conover) • Major conclusions, Recommendations overall and for each data group • Future joint activities AMSR-E Science Team Meeting Asheville, NC

8. Summary of Meeting Outcomes:Duplication of data across data providers • We only found a few duplicate data sets; however we did find lack of documented differences between some data sets which were therefore construed to be duplicative. • Some passive microwave products are available from both the data producer and the archive of record. Generally, data producers make data available on a temporary basis to a smaller community and the archive of record provides longer-term archival, broader community distribution, and may provide higher levels of service. Examples include: • TRMM Microwave Imager products at PPS and GES DISC • MEaSUREs products from PI’s and the respective ESDCs • Draft Recommendation: Provide clear documentation and cross-referencing between these data sets. Digital Object Identifiers (DOIs) can help clarify whether data sets are identical. AMSR-E Science Team Meeting Asheville, NC

9. Summary of Meeting Outcomes:Identification of source data and version • We found several instances in which information about the source data set from which a product was derived and/or the algorithm and version used, was absent or not clear. Examples include: • Time series for which input data source or version may have changed over time (e.g., gridded SSM/I Tb from NSIDC) • Imagery or analysis products for which input data sources are not obvious (e.g., imagery produced by services like Giovanni) • Draft Recommendation: For all NASA data products, lineage should be clearly documented. In particular, • Versions of both the data product and the algorithm(s) used to generate it need to be verified, documented, and made more visible as necessary. • Source and version of data set(s) used as input to the product need to be identified, noting when the source data were published, and especially when the version of the source data has changed. AMSR-E Science Team Meeting Asheville, NC

10. Summary of Meeting Outcomes:Documentation requirements • We found a variety of levels of documentation among different PM data sets, especially between the EOS missions (AMSR-E, TMI) and some of the others (e.g., SSM/I). • While ESDIS provides guidelines for data set documentation, they are not consistently applied. Existing guidelines include: • Metadata requirements including GCMD DIF, ECHO entries, Guide documents • Templates for ESDC data product reviews • Emerging Provenance and Context Content Standard • Draft Recommendation: Compile a checklist of required documentation for data sets, based on existing requirements and guidelines. Review and update documentation for PM data sets, coordinating across ESDCs holding similar data. • Consider tying documentation requirements to levels of service • Review documentation for high value data sets first • Maintain and update regularly AMSR-E Science Team Meeting Asheville, NC

11. Summary of Meeting Outcomes:Old or out-dated products • We found several data sets which have been or will be superseded by newer products at the same data center or another data center. Examples include: • SSM/I Tb and atmospheric suite products from GHRC are based on the RSS v4 algorithm. RSS is now finalizing v7. • SSM/I ocean wind speed from DMSP-F8 available from JPL is a Pathfinder product, superseded by current data sets from RSS. • SSM/I ocean wind vector products at JPL are superseded by Cross-Calibrated Multi-Platform (CCMP) Ocean Surface Wind Components product currently being generated • Draft Recommendation: In consultation with their User Working Groups, data centers should consider common practices to handle data set retirement or sequestering. Users should be referred to the replacement product. AMSR-E Science Team Meeting Asheville, NC

12. Status and Plans • Workshop Committee is compiling a detailed report to include • Findings specific to each data discipline discussed (listed in slide 5) • Specific recommendations for consideration by each ESDC User Working Group • All participants – ESDCs, MEaSUREs, ESDIS – found this exercise to be valuable and we recommend that other instrument /discipline groups host similar workshops AMSR-E Science Team Meeting Asheville, NC

13. Backup Slides AMSR-E Science Team Meeting Asheville, NC

14. Summary of Data Holdings * Not publicly available AMSR-E Science Team Meeting Asheville, NC