1 / 25

Data Aggregation & Products Generation - Results, Strengths, and Weaknesses - EMODnet Chemistry WP2

Summary of results on nutrients, data inventory, regional data buffer, DIVA maps generation, results on other parameters (oxygen and chlorophyll a), contaminants inventory, and strengths and weaknesses.

samanthaellis
Download Presentation

Data Aggregation & Products Generation - Results, Strengths, and Weaknesses - EMODnet Chemistry WP2

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. EMODnet Chemistry WP2: Data aggregation and products generation - results, strengths and weaknesses - Atlantic Sea Annual Meeting, June 8-9 2016, Helsinki, Finland

  2. Outline • Summary of results on Nutrients : • Data inventory • Regional data buffer : time/space distribution • DIVA mapsgeneration • Results on Otherparameters (Oxygen and Chlorophylla) • Data inventory / aggregation / QC • Regional data buffer : space/time distribution by parameter • DIVA mapsgeneration • Contaminants : Data inventory • Strengths and weaknesses Annual Meeting, June 8-9 2016, Helsinki, Finland

  3. Nutrients inventory First harvesting Second harvesting Enlarged data coverage Distribution by partners Distribution by parameters Annual Meeting, June 8-9 2016, Helsinki, Finland

  4. Regional data buffer - Nutrients - Time distribution Time histograms 1970 1990 1990 1990 Season histograms Annual Meeting, June 8-9 2016, Helsinki, Finland

  5. Regional data buffer - Nutrients - Space distribution Station maps Vertical plots 3000m Annual Meeting, June 8-9 2016, Helsinki, Finland

  6. DIVA maps generation Nutrients at surface (0m) – summer (0709) - 20002009 Nitrate_plus_nitrite Phosphate Silicate Ammonium Annual Meeting, June 15-16 2015, Istanbul, Turkey

  7. Other parameters inventory Distribution by EDMO code Annual Meeting, June 8-9 2016, Helsinki, Finland

  8. Other parameters inventory Distribution by parameters Annual Meeting, June 8-9 2016, Helsinki, Finland

  9. Other parameters - metadata QC - • ODV files harvested : problemsfaced (1) • 116 empty files • semantic and column headers but no data • 4026 errors on P01 for PH : white spacecharacterinstead of XX • //<subject>SDN:LOCAL:pH</subject><object>SDN:P011::PH ZZXX</object> • 3errorsin columnheader : white spacemissing • Depth [m] QV:SEADATANET DOXY_DOXSEB_NOT_NDT[ml/l] • 33 errors in cruisename • CruiseStation • UPWELLING V CAP 7402 "NAN 00010 • 9 wrongLocal_cdi_id • LIVRA_00400_G02 instead of 898LV12840015 • 44 errors in Local_cdi_id : hyphenmissing • 486PE15101A336881 instead of 486PE151-01A336881 • 92 wrong EDMO codes • 2124 instead of 3288 Annual Meeting, June 8-9 2016, Helsinki, Finland

  10. Other parameters - metadata QC - • ODV files harvested : problemsfaced (2) • >1500 Incorrect primary variables (P01) : • - For sediment data, ADEPZZ01 instead of COREDIST • - For ocean data, RECORD numberinstead of ADEPZZ01 • //SDN_parameter_mapping • //<subject>SDN:LOCAL:Record_No</subject><object>SDN:P01::ACYCAA01> • Many Warnings on SDN vocabulary version : • //<subject>SDN:LOCAL:pH</subject><object>SDN:P011::PH ZZXX</object> Use the SDN tool : Change_Vocab_V1toV2 http://www.seadatanet.org/Standards-Software/Software/Change_Vocab_V1toV2 • Solution : feedback loop with CDI partners • Files corrected locally by RL to gain time • Contact with the CDI partner to update the CDI in SDN/Emodnet infrastructure (MARIS CDI support) Annual Meeting, June 8-9 2016, Helsinki, Finland

  11. Data aggregation and QC - Other parameters - • Perform QC with ODV • Use only data with Quality Flag = 0, 1, 2, 6 as initial collection to use only good data • Unit conversion and aggregation (P35 vocabs) • Search for out of range values with Flag = 1 • Set a QC for all measurements with Flag = 0 • Set default values with Flag = 0, 1 to Flag = 9 Flag 0 Wrong Flag 9 Write a report on QC and send results to data providers and MARIS Wrong Flag 1 Annual Meeting, June 8-9 2016, Helsinki, Finland

  12. Data aggregation Oxygen data SDN:P35::EPC00002: Water body dissolved oxygen concentration [umol/l] Mix of uncalibrated and calibrated measurements

  13. Data aggregation Chlorophyll-a data SDN:P35::EPC00105: Water body chlorophyll-a [mg/m^3] Mix of uncalibrated and calibrated measurements Annual Meeting, June 8-9 2016, Helsinki, Finland

  14. Data aggregation and QC Oxygen quality checks – problems faced • LowOxygen profiles • e.g. flaggedsome BODC data to 4 • StrangeOxygen data • e.g. delete PANGAEA profiles because of not calibratedOxygen • Low historic data • French SHOM data flagged to 3 • Oxygen profiles decrease to zero • Russian data flagged to 3/4 Annual Meeting, June 8-9 2016, Helsinki, Finland

  15. Regional data buffer Oxygen data Better time coverage 1963 c) Time histogram a) Station map b) Vertical plot d) Data bins e) X/Y distribution f) Season histogram Annual Meeting, June 8-9 2016, Helsinki, Finland

  16. Data aggregation and QC Chloroplyll-a quality checks – problems faced • High Chla data at depth Feedback to the CDI partner : calibrated data? correction to apply? Flag to 3/4 ? Annual Meeting, June 8-9 2016, Helsinki, Finland

  17. DIVA maps production DIVA maps settings : Annual Meeting, June 15-16 2015, Istanbul, Turkey

  18. DIVA maps production Oxygen at surface (0m) – summer (0709) DIVA – 1997/2006 Annual Meeting, June 15-16 2015, Istanbul, Turkey

  19. DIVA maps production Oxygen at surface (0m) – summer (0709) DIVA – 1994/2003 M36/5 cruise – sept 1996 • Feedback to the CDI partner : calibrated data? shift to apply? Flag to 3 ? • Run DIVA after corrrection/flag of these 57 profiles Annual Meeting, June 15-16 2015, Istanbul, Turkey

  20. Contaminants inventory Annual Meeting, June 8-9 2016, Helsinki, Finland

  21. Contaminants - metadata QC - • ODV files harvested : problemsfaced • 319 wrong file extension : .txtinstead of .zip • 163 wrongsemantic header • Wrongprimary variable : ADEPZZ01 instead of COREDIST • Warnings on SDN vocabulary version Update of P01/P02/P35 vocabularies before data aggregation Annual Meeting, June 8-9 2016, Helsinki, Finland

  22. Strengths and weaknesses • Strengths : • the robust CDI infrastructure • the validation loop for metadata and data QC • improve the data archives by detecting the problems with the metadata and the data, by fixing them and by giving to scientists "good" collections • Weaknesses : • Large number of sources/instruments to measure a parameter makes challenging to obtain consistent aggregated dataset • Time delay in the harvesting and the validation loop • Need more Scientific experts feedback on the products Annual Meeting, June 8-9 2016, Helsinki, Finland

  23. Ongoing work • Finish QC on Chlorophyll, PH, … • Re-run DIVA for Oxygen after additional QC • DIVA for Chlorophyll • Contaminants collections • Import aggregated Qced collections and DIVA maps in Oceanbrowser and Sextant catalogue • Send report on metadata and data QC to CDI partners and to MARIS Annual Meeting, June 8-9 2016, Helsinki, Finland

  24. Thanks! Annual Meeting, June 8-9 2016, Helsinki, Finland

  25. DIVA validation loop • DIVA maps production Data extraction (and weighting) Parameters optimization Run analysis Additional QC Phosphate Silicate NO3+NO2 Ammonium Oxygen 4D Netcdf files ready Chlorophyll If needed Annual Meeting, June 8-9 2016, Helsinki, Finland

More Related