Report and Progress on WGCV Cooperation with WGISS

Report and Progress on WGCV Cooperation with WGISS • K. Thome/C. Ong/A. Kuze • NASA/CSIRO/JAXA • WGISS #47 • NOAA, Silver Spring, Maryland, USA • April 29, 2019

WGCV overview WGCV includes members from CEOS agencies as well as the Chairs of six subgroups Subgroups allow inclusion of a wider range of researchers Atmospheric Composition Land Product Validation Infrared and Visible Optical Sensors Microwave SAR Terrain Mapping Task groups address cross-cutting topics covered by multiple sub-groups • General topics include • Validation metrics • Protocols • Traceability • Prelaunch and post-launch characterization WGCV-44 meeting hosted by EUMETSAT in Darmstadt, Germany

WGCV leadership WGCV Vice Chair • Akihiko Kuze • GOSAT project team, JAXA • Greenhouse gases Observing Satellite (GOSAT) project for 15 years developing on-board instruments, calibrations and operation • Level 0-1 processing of GOSAT data and the GOSAT Level 1 products WGCV Chair • Cindy Ong • Principal Research Scientist @ CSIRO • Expertise in ground-based and airborne-basedremote sensing (imagers & optical sensors) • Greenhouse Gas detection including sensor development • Quantification and monitoring of methane and other environmental impacts related to the onshore gas industry

Error Propagation Achievement: Review paper on the Importance of Biomass Product Validation. Accepted in Surveys in Geophysics 1. TLS and Field Data for plot biomass estimates 3. Generate local biomass maps at desired (spaceborne product) resolution Expected outcome: Acommunity-driven good practices guidebook for EO biomass product validation. External review expected in summer 2019 • Chapters • Good practices for biomass estimation in the field • Linking remote sensing observations to field estimates • Error Propagation • Using existing in situ data for map validation • Independent Validation and Reporting • Utility of Protocol for Other Communities • Knowledge Gaps Calibrate Airborne lidar with in situ data 4. Report accuracy over geographic domain of interest given available data • CV19: contributing towards • CEOSStrategyforCarbonObservationsfromSpace • CARB16: Cal/Val and production of biomass products from CEOS missions • CARB-23: Forest Biomass measurements for GFOI countries; • AGRI-13: Iteratively respond to GEOGLAM EO Data Coordination team’s definitions of “Applications Ready Data” (ARD+) and “Essential Agricultural Variables for GEOGLAM” WGCV Current Activities - Biomass Validation Protocol

Achievement: Global Surface Albedo ProductValidationBestPracticesProtocolcompleted, ready to be endorsed at the WGCV-45 in Julyin Perth, Australia; LPV SuperSites Contributing to global products ECVs from a variety of satellite sensors per Action Items outlined in the 2010 GCOS Implementation Plan in Support of the UNFCCC (GCOS-138, 2010, aka IP-10) and the 2011 update [GCOS-154, 2011] WGCV Current Activities - Surface Albedo Validation Protocol

Expected Outcome: A list of reference standards for the validation of CO2and CH4 (and CO, N2O) products suitable for use in inter-comparison of multiple GHG missions • Identify the current shortcomings/gaps/sustainability in GHG Cal/Val, and, formulate recommendations on the medium- to long-term way forward, that is, specific focus on GHG Fiducial Reference Measurement (FRM) CV-18: Drivers/contributing to CEOS-32-05 • CEOS AC-VC GHG Whitepaper “A Constellation Architecture For Monitoring Carbon Dioxide And Methane From Space”; • Past CEOS chair initiative on “an international coordinated Greenhouse Gases monitoring system”; WGCV Current Activities - Greenhouse Gas Reference Standards

* SAR workshop at CONAE with 50 participants from JAXA, NASA/JPL, DLR, ESA, CSIRO, CCRS, INTA, KARI, GA, CONAE, University of Zurich , and companies Aresys, MDA, and SoloEO WGCV Current Activities - Analysis Ready Data • Support LSI-VC on the CARD4L Product Alignment Assessment Process through peer-review of products generated by data providers adopting these standards (VC-40); • Contributed paper “Role Of CEOS Working Group On Calibration And Validation In Analysis Ready Data Products”. IGARSS invited session organised by LSI-VC; • Support CARD4L SAR PFS Development and Product Assessment (VC-33). • Support LSI-VC with the development of CARD4L SAR Product Family Specifications and support of the CARD4L Product Alignment Assessment Process through peer-review of products generated by data providers adopting these standards; • Future CV: SAR Radiometric Calibration and Geolocation Supersites. • Update the calibration target and sites database and augment where needed; • Produce a standards and requirements document for a multi-mission SAR image Cal/Val supersite including the characterization of targets and sites. This Cal/Val supersite may include a virtual array of active radar and passive calibration devices.

WGCV-43 meeting hosted by INPE and joint with WGISS-45 WGCV Current Activities - Synergies amongst teams • CV-14: Report on application of approaches for cloud masking will involve understanding impacts on CARD4L (LSI-VC) • CV-17: Surface reflectance validation also will prompt interactions on CARD4L • CV-18: Greenhouse gas reference standards for interoperability will lead to work with AC-VC • CV-19: Biomass validation protocols will involve WGClimate • Formalize those interactions already occurring at the subgroup levels • Fiducial reference measurements (FRMs) for validation of surface temperature is a collaboration between WGCV IVOS/LPV and SST-VC • Ocean Colour FRM comparisons for OCR-VC and IOCCG

Joint WGISS/WGCV Session Reminder Discussions cover common areas and topics in the ongoing/planned WGs activities which could benefit from cooperation between the two groups Ways to describe and share available facilities, expertise, resources, technical information and documentation Topics of common interest with natural synergies INPE meeting emphasized two key elements to help completion of activities in a reasonable time period Well-defined tasks to define a way forward Agreed to monitor progress through periodic joint calls – all three to date were organized by M. Albani/M. Piepgrass WGISS and WGCV have held joint meetings including the most recent joint session at INPE in April 2018

Joint WGISS/WGCV Session Reminder Data Formats and Interoperability in the framework of FDA Robert Woodcock (WGISS), MedhavyThankappan (WGCV) Quality Indicators in Discovery Metadata Michael Morahan (WGISS), Nigel Fox (WGCV) CEOS Data Cubes and CEOS Test Sites Data Access in support of WGCV Activities Robert Woodcock (WGISS), Greg Stensaas (WGCV) Standardization and Best Practices (e.g. ISO 19159-3) Richard Moreno (WGISS), Cindy Ong (WGCV) Four topics discussed in April 2018 with leads identified from WGISS and WGCV

Joint WGISS/WGCV -Activity summary to date Teleconon June 18, 2018 to develop concrete tasks Discussed overall goals of the tasks with long-term concepts Developed short term actions to show progress by Fall 2018 SIT Technical Workshop Updates were provided during WGCV-44 Telecon September 6, 2018 to discuss progress Provided updates during SIT Technical Workshop in Darmstadt last September Telecon February 7, 2019 WGISS/WGCV joint session at WGISS-45 with follow-on telecons

1) Data Formats and Interoperability in the frame of FDA Example case for surface reflectance showing the range of possible challenges in creating a consistent time series across multiple processing schemes Data format is not the biggest issue to solve related to interoperatbility Format conversion is pretty well understood and WGISS already provides that service Recognize that the definition of ARD is necessarily broad and we should not let this prevent progress on understanding how to communicate uncertainties and data quality Objective: Ensure users a seamless access and usability of data from multiple sensors including tools to use data from multiple sensors

1) Data Formats and Interoperability in the frame of FDA Analysis Ready Data (ARD) clear definition Definition of quality/uncertainty parameters to be inserted into ARD Determine a method to incorporate these along with their provenance into the data stream Definition of a user manual for data community on how to provide provenance information Tasks - ARD uncertainty: Work to define a set of quality/uncertainty parameters to include within ARD and determine an initial method to incorporate these along with their provenance into the data stream, including education of the data community on how to provide provenance information

1) Data Formats and Interoperability in the frame of FDA WGCV concentrating on Surface Reflectance as ARD test case to be studied in more detail Thermal and SAR ARD products recognized as important but would be completed at a later stage. Efforts to concentrate on Landsat 8 and Sentinel 2 investigations Goal was to complete work by August 2018 ARD team will also include a Cal/Val function to improve collaboration with WGCV defining the assessment framework Approach (defined at June Telcon):

1) Data Formats and Interoperability in the frame of FDA WGCV was to provide an initial list of uncertainties in late September Discussion with WGISS to define what uncertainty information to make available to users and how WGISS to investigate how field sites data (e.g. for Surface Reflectance) can be made available through Data Cube technology Workflow for this is in Digital Earth Australia - starts with field data, convolution to match satellite sensor surface reflectance, then comparison report mechanism Reviewing USGS land product characterisation website for further use-case insight Work with the WGISS Connected Data Assets team for an ARD data cube on demand September and February updates:

2) Quality Indicators in Discovery Metadata Measurement equation must be embedded somehow Per pixel uncertainty too large to handle There exist a set of key quality metrics that are useful as indicators QI should be tagged at all stages with a link to the earlier stages and is locked into the future Discussion on where to go forward – QI is well defined but concrete cases will always have different levels and flags Describing QI is not easy and more difficult to do it for the user readability Keep in mind that quality is defined by end user – fitness for purpose is defined by end user Objective: Ensure quality and uncertainty information availability (discovery and access) for users

2) Quality Indicators in Discovery Metadata Objective: Ensure quality and uncertainty information availability (discovery and access) for users Or Making things like this transparent to the non-calibration expert

2) Quality Indicators in Discovery Metadata Qualityassessment matrix

2) Quality Indicators in Discovery Metadata Survey on QI currently used by different sensor families Rely on dedicated small group teleconferences (Task Team) SST to be confirmed as a suitable test case for QI development WGCV to survey SST QIs currently used by different sensor families at different agencies. WGCV Vice-Chair to ask SST Virtual Constellation Chair to help develop a case study on QI WGISS to define approach for representing and including QIs for the test case in discovery metadata searchable by end users Embedding measurement equation Link later stages to earlier stages and processing Readability for end users Tasks - QI Test case: Evaluate whether SST would provide a suitable test case for QI development that can act as an expressive case for QI access and implement this or another one as determined through this activity

2) Quality Indicators in Discovery Metadata Project was slow to start due to Northern Hemisphere summer Telecon of the Task Team has yet to be organized A. Kuze has confirmed that SST Virtual Constellation will collaborate N. Fox working with GHRSST to survey SST QIs currently used by different sensor families at different agencies G. Corlettof GHRSST has started to compile SST QIs Review current reported meta-data on SST and how QI information made accessible/viewable For individual sensors and from GHRSST product (if different) Value/usefulness/adequacy for current users Links to Validation data and preservation of information Develop measurement Equation, traceability visualisation of process (from L1 data sets) September and February Updates:

3) CEOS Data Cubes and CEOS Test Sites Data Access in support to WGCV Activities Exist a range of data sets already available through the cal/val portal such as Landnet; plus QA4EO activities; and LPCS Need a way to get those data more available to users Recommendation – Focus on IVOS and LPV and key GEO applications including obtaining data over key test sites (WGCV) and pushing this process to CEOS SIT/LSI VC/WGCV/WGISS Also need to simplify the problem from a WGISS view and concentrate on analyzable variables and get an on-demand analytics environment to do what users want Objective: Data Cubes (or similar stacking of data over time for specific locations) test cases

Pre-configured list of sites • Data Discovery • Direct access to the selected site • In-situ Data Visualization and pixel access; • In-situ / EO Data Discovery, Visualization and Pixel Access • Atmospheric Correction Inter-Comparison Exercise (ACIX) (CV-14) • ACIX II-land, 115 sites were selected based on AERONET stations’ location and measurements availability for one-year period starting from October 2017. A total of around 7000 image scenes acquired by Sentinel-2A, -2B and Landsat-8; • ACIX-Aqua, the exercise is split into phases: Phase I, processors over coastal waters (20 AERONET-OC sites) and Phase II, processors over inland waters (15 sites with in-situ measurements) • Radiometric Calibration Network (RadCalNet) (CV-09) • 108 users since portal opening 3) Joint WGISS/WGCV CEOS Data Cubes and CEOS Test Sites Data Access

3) CEOS Data Cubes and CEOS Test Sites Data Access in support to WGCV Activities Definition of the status of the activity concerning the identification of the key supersites in the frame of CEOS Land Product Validation (LPV) Definition of the analytics tools needed Definition of how to improve the user access to the LPV supersites Tasks - LPV supersite data access test case: LPV supersites are a good test case related to validation with a broad user community. Objective is to provide an improved user access to the LPV supersite data sets coupled with analytics tools to provide improved results for the users

3) CEOS Data Cubes and CEOS Test Sites Data Access in support to WGCV Activities Document the cal/val test sites Information on Land calibration sites for optical sensors Define list of validation sites with LPV Ensure documented sites are into the IDN with a link to the CEOS Cal/Val portal for site definition where available Follow-on task is for WGCV to update test site catalog on CEOS Cal/Val porta Include links later for those not ready yet Focus for initial datacube/catalog population be on previous tasks for surface reflectance Ensure data are available from CEOS agencies Start with Landsat and Sentinel -2 Work with LSI-VC on data population Approach defined at June Telcon

3) CEOS Data Cubes and CEOS Test Sites Data Access in support to WGCV Activities Test site catalog list - Note using work being developed for Land Product Characterization Systems (LPCS) Used LPV site list CEOS test site list NEON test sites Need to get new sites added New RadCalNet New Australia sites https://lta.cr.usgs.gov/Calibration_Validation_Test_Sites Test site catalog Cal/val portal – http://calvalportal.ceos.org/ USGS – test site catalog Document the cal/val test sites

3) CEOS Data Cubes and CEOS Test Sites Data Access in support to WGCV Activities Document the cal/val test sites

3) CEOS Data Cubes and CEOS Test Sites Data Access in support to WGCV Activities WGCV continues update the test sites catalogue on the CAL/VAL Portal Goal is to provide a list of sites to M. Morahanwho will begin a test case to define how to get this information into the IDN for discover and access WGCV to update info on USGS test data to allow access from WGISS Connected Data Infrastructure R. Woodcock (WGISS) in cooperation with M. Thankappan (WGCV) are investigating methods make test sites data (e.g. for Surface Reflectance) available through Data Cubes WGCV GA team has supplied some CV jupyter notebooks which can be used with the Data Cube technology Operate over some Australian test sites and need to be ported to use the CEOS test site data Information has been supplied by the WGISS CDA team regarding how to discover the test data Challenges to overcome in the discovery, metadata and data access pipeline to load the necessary data into a DC Bigger challenge competing work priorities September and February updates:

4) Standardization and Best Practices Discussion from 2016 is still valid today Work on actions from 2016 still remains Support evolution of Data Management principles (Quality principles) (WGISS-41) Support WGISS to provide guidelines for the metadata for ARD to ensure the information needed to describe a measurement is available (WGISS-41) Objective: Contribution on CEOS Best Practice/White Paper

4) Standardization and Best Practices Objective: Contribution on CEOS Best Practice/White Paper • Develop Best Practices on Data Reprocessing – including current and future practices (i.e. on-demand ARD Processing) (WGISS-41) • Support and collaboration on ISO 19159-3 “Geographic information -- Calibration and validation of remote sensing imagery sensors and data – Part 3: SAR and InSAR.” (WGISS-43) • ISO 19159-3: Geographic information -- Calibration and validation of remote sensing imagery sensors -- Part: 3: SAR/InSAR passed DIS vote in April 2017. It will automatically become ISO Technical Specification in this year.

4) Standardization and Best Practices Definition of points of contact Review of Data Management and Stewardship Maturity Matrix Collection of the RIDs Issuing new version of the Data Management and Stewardship Maturity Matrix Provision of new input to GEO Data Management Principles Implementation Guidelines (DMP IG) Tasks - Revisit and update WGISS-41/WGCV-40 actions: Define a point of contact within WGCV to work with WGISS to reinvigorate the actions of WGISS-41 in terms of more recent CEOS activities and move towards closure of these items to provide a good practice white paper for inclusion on the CEOS website

4) Standardization and Best Practices Approach defined at June Telcon WGISS point of contacts changed to I. Maggio and G. Stensaas with C. Ong also a contact within WGCV WGCV to provide comments on Maturity Matrix to WGISS Review actions from 2016 and reformulate as needed Status prior to September telecon • I. Maggio provided updated actions list • G. Stensaas responded that actions are still relevant to CARD4L and MRI, but recommended that an ARD/MRI, WGCV IVOS and LPV, WGISS team is needed to determine how best to close the current actions while leading to an updated set of actions to support for ARD and MRI

4) Standardization and Best Practices Maturity matrix concept helps address quality indicators topic Maturity Matrix developed in WGISS is very much data access and "data preservation" oriented Efforts at ESA are working to include data quality LPV work at the validation stage is being evaluated as a test bed for inserting maturity matrix information There has been quite a bit of activity related to maturity matrix within WGISS and WGCV in parallel to current joint activity Efforts are underway to coordinate those parallel activities Maturity Matrix is on the agenda for WGCV-45 in Perth Maturity Matrix: Becomes more important as products/missions increase and comparison tools are needed

4) Standardization and Best Practices WGISS Data Management and StewardshipMaturity Matrix

4) Standardization and Best Practices CEOS Maturity Matrix

4) Standardization and Best Practices CCI Maturity Matrix (courtesy ESA)

Summary Initial telecons were successful at organizing the activities after the joint meeting in Brazil in April 2018 WGCV and WGISS meetings in Fall 2018 helped to spur further action as did CEOS SIT meeting in September 2018 Current WGISS meeting and July WGCV meeting are helping to remind activity leads of next steps Key next steps will be To have a WGCV-organized telecon of the activity leads on both working groups prior to July WGCV meeting Assess the work loads of the activity leads and determine if additional help is needed since work loads can change in 12 months Prepare for the next WGCV/WGISS joint meeting WGISS/WGCV joint activities are making progress even in an environment of competing deadlines and projects

Backup - WGISS/WGCV joint actions

Report and Progress on WGCV Cooperation with WGISS