1 / 15

GEOVIQUA workshop

GEOVIQUA workshop. February, the 18th, 2011 Barcelona S&T experiences on data quality in Remote Sensing Joost Smeets. Contents. Introduction to S&T background- experiences Overall approach to data quality control in Remote Sensing Types of data quality information Data quality concepts

tasha-vang
Download Presentation

GEOVIQUA workshop

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. GEOVIQUA workshop February, the 18th, 2011 Barcelona S&T experiences on data quality in Remote Sensing Joost Smeets QUAlity aware VIsualisation for the Global Earth Observation system of systems

  2. Contents • Introduction to S&T background- experiences • Overall approach to data quality control in Remote Sensing • Types of data quality information • Data quality concepts • Quality Information and Action Protocol Kick off meeting. February 17th, 2011

  3. Relevant S&T data quality projects • Development of routine calibration and quality monitoring systems for ESA Earth Observation satellite missions • Atmospheric sensors on ENVISAT (GOMOS, MIPAS, SCIAMACHY) • Earth Explorer missions: AEOLUS (wind speed), SWARM (magnetic field), CRYOSAT-2 (ice monitoring) • IDEAS project • Harmonizing quality monitoring of different missions • Initial work on standardization for quality monitoring • GECA project: Generic Environment for Cal/Val Analysis • Intercomparison tool chain of satellite data vs. correlative data • Development of Quality Information and Action Protocol (QIAP) • Contribution to major evolution of Generic Earth Observation Metadata Standard (GEOMS) Kick off meeting. February 17th, 2011

  4. Product QC Product QC Anomaly investigation Calibration processing Calibration monitoring Long-term monitoring Configuration monitoring Data flow monitoring PDGS performance Product QC Quality Monitoring Organisation of data quality control for remote sensing Mission planning Mission operations Image: ESA Processing Archiving Processing Archiving Processing Archiving Processing Archiving Kick off meeting. February 17th, 2011

  5. Data quality control loop Image: ESA Kick off meeting. February 17th, 2011

  6. Description of quality information • Examples of quality information • Uncertainty represented by a standard deviation • A data validity flag that flags measurement data as valid/invalid • A combination of flags that signify special conditions on the data or the processing algorithms Kick off meeting. February 17th, 2011

  7. Description of quality information • Examples of quality information (continued) • Secondary measurements and corresponding rules to derive quality of primary measurement • Discard measurement if cloud detected (filtering) • Measurements at solar zenith angles>80° should be discarded • Model based uncertainty Ɛx=f(x,y,z,...) Kick off meeting. February 17th, 2011

  8. Description of quality information • Examples of quality information (continued) • Disclaimers • Inform users on data quality aspects that are not in the product (non-integrated, typically becomes available only after data product has been delivered to users) • Disclaimer example • Plots (example from GOMOS monthly report) Kick off meeting. February 17th, 2011

  9. Quality concepts – processing level • Quality issues for different processing levels: • Level 0: Accumulation of downlinked data into “Raw” instrument data products • Corrupted data issues (communication), Saturation of measurements , spikes, Signal-to-Noise Ratio, telemetry data • Level 1: Processing into calibrated (intermediate) physical quantities • Application of calibration parameters and processing algorithms to raw data • Updating of calibration parameters from special measurements (on-board calibration) • Level 2+: Processing into derived physical quantities that the user is interested in • All kinds of processing, combination of data products • Lower-level data quality indicators influence higher level data quality in a non-trivial way Kick off meeting. February 17th, 2011

  10. Quality concepts – data version • Several values for a measurement at a specific time and place may exist due to different processing • Near real time processing: faster delivery of data to users at the expense of data quality • Consolidated processing: after an additional period, more accurate data become available due to more information about instrument characterisation (calibration) • Reprocessing: after even longer periods, intercomparison/ground based validation may lead to additional knowledge that is used to reprocess raw data into a new version of the measurement. • A concept is needed to identify different versions of measurement data Kick off meeting. February 17th, 2011

  11. Quality concepts – data version • Possible definition: “A data-version associated with measurement data is a textual identifier indicating a common configuration for these measurement data, for example because the data have been generated using the same data processor within the same processing centre and using the same procedure” Kick off meeting. February 17th, 2011

  12. Quality concepts -- metadata • Different levels of specification of (quality) metadata • At measurement level (requires access to actual product data) • At product level (can be provided through catalogues) • At dataset level (e.g. for dataset selection) Kick off meeting. February 17th, 2011

  13. Quality concepts -- QIAP • QIAP= Quality Information and Action Protocol • QIAP study and implementation has been part of GECA project (Generic Environment for Cal/Val Analysis) • Concept: moment of discovery of quality information with respect to data availability: • Integrated quality information: quality information that is included in the data processing and delivered with the data • Non-integrated: data quality information that becomes available only after a data product has been delivered to users • QIAP is a means of: • Informing users about non-integrated data quality • Providing users with a correction/filtering mechanism to be applied to data products for improving data quality. Note: the correction should be able to be performed automatically by SW. Kick off meeting. February 17th, 2011

  14. Quality concepts -- QIAP • Issues found during QIAP design: • Different notions of what quality information is/should be • As a consequence the scope of QIAP was initially unclear • Therefore the QIAP design included a study that: • Captured different views on quality information • Defined clear vocabulary for describing quality information concepts that guided the final decision on the scope of QIAP • Meanwhile QIAP has been successfully implemented as: • A defined protocol • A (pilot) Kick off meeting. February 17th, 2011

  15. Quality concepts QIAP User Product Product Product Product Initial products Corrected products (data) Product Data Quality information Quality information Correction/filtering info QIAP-server Quality Information database Kick off meeting. February 17th, 2011

More Related