1 / 40

Resulting Technical Specification

Resulting Technical Specification. Technical approach and discussion Development of the processing system System design Targets: Maximum re-use and integration of existing processing / data handling package. Flexibility to activate one merging technique or the other.

willem
Download Presentation

Resulting Technical Specification

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Resulting Technical Specification

  2. Technical approach and discussion • Development of the processing system • System design • Targets: • Maximum re-use and integration of existing processing / data handling package. • Flexibility to activate one merging technique or the other

  3. Technical approach and discussion Development of the processing system System overview Acceptance of any data sources Stand alone processing system (PC based HW / under GNU/linux oper. syst.) Intermediate individual sensors level 3 products Dissemination of the DDS, PPS, FPS Link with BEAM

  4. Technical approach and discussion • Development of the processing system • GLOBCOLOUR • processing system Main modules: • Pre-processing • Spatial binning • Temporal binning • Merging

  5. Technical approach and discussion Update of the methodology GLOBCOLOUR processing system • L3 products at track level • daily L3 products per instrument (temporal binning +data-day) • merged daily L3 products (merging) • 8-days and monthly L3 products (temporal binning)

  6. Globcolour processor development

  7. Technical approach and discussion • Development of the processing system • GLOBCOLOUR processing system • Pre-processing components: • Application of a cross calibration LUT • Computations of Fully Normalised Water Leaving Radiance • Computation of a common Kd(490)

  8. L2 grid L3 grid Technical approach and discussion Development of the processing system GLOBCOLOUR processing system Spatial binning Application of a flux-conservative resampling algorithm (using the Sutherland- Hodgeman area Clipping technique)

  9. Technical approach and discussion • Development of the processing system • GLOBCOLOUR processing system • Spatial binning • Main software component developed: • Product reader for all types of Level-2 input products; MERIS L2 (Envisat PDS format), SeaWiFS and MODIS /Aqua Ocean Colour Level 2 in (HDF-EOS), POLDER Ocean Colour Level 2 (HDF format) • Resampling Component • DDS Level-2/Level-3 Extraction Component • DDS Level-2/Level 3 granule netCDF writer • Product writer for the spatially binned, intermediate level-3 netCDF products

  10. Technical approach and discussion • Development of the processing system • GLOBCOLOUR processing system • Merging • Several software components for merging are currently under IDL breadboard implementation: • One or more implementations for the merging method to be used, e.g. • Weighted Averaging • Optimal Interpolation • Blended Analysis • GMS • An ocean colour retrieval processor starting from merged sensor spectra

  11. Technical approach and discussion • Development of the processing system • GLOBCOLOUR processing system • Temporal binning • The core software components have been developed: • A component performing the weighted averaging • DDS Level-3 extraction component • DDS Level-3 granule netCDF writer • Level-3 data product writer using netCDF format • Components not yet consolidated: • A component used for computing the error statistics • Data-day algorithm for all instruments (breadboard under validation)

  12. Technical approach and discussion Development of the processing system Input Products Overview Uncompressed data volume

  13. Technical approach and discussion Development of the processing system Output Products Overview

  14. Technical approach and discussion Development of the processing system Output parameters (1/3)

  15. Technical approach and discussion Development of the processing system Output parameters (2/3)

  16. Technical approach and discussion Development of the processing system Output parameters (3/3)

  17. Technical approach and discussion Development of the processing system ISIN grid main characteristics

  18. Technical approach and discussion Development of the processing system Output products variables - definition (1/2)

  19. Technical approach and discussion Development of the processing system Output products variables – storage (2/2) + 15 attributes: scaling factors, names, unit, etc…

  20. Technical approach and discussion Development of the processing system Format modification since TS 1.1 (globCOLOUR processor v1.0) Variables are no more scaled due to hot points occurrence hot points The hot points are due to a high coding quantisation error at very small weight bins Impact on final product size has been assessed and taken into account in the re-estimation of the required resources

  21. Technical approach and discussion Development of the processing system DDS/PPS/FPS products – overall list (1/2)

  22. Technical approach and discussion Development of the processing system DDS/PPS/FPS products – content (2/2)

  23. Technical approach and discussion • Production of the Full Product Set • Processing requirements • one month of data from the 4 sensors processed in 5 days • ten years of data processed in 6 months • NRT L3 processed in less than 24 hours Preliminary data flow requirements and system sizing has been re-assessed using the globCOLOUR processor v1.0 installed on the APADIC system at ACRI-ST

  24. Technical approach and discussion Production of the Full Product Set APADIC system in ACRI-ST

  25. Technical approach and discussion Production of the Full Product Set Resources requirements analysis

  26. Technical approach and discussion Production of the Full Product Set Resources requirements analysis

  27. Technical approach and discussion Production of the Full Product Set Resources requirements analysis

  28. Technical approach and discussion Production of the Full Product Set Resources requirements analysis

  29. Technical approach and discussion Production of the Full Product Set Resources requirements analysis

  30. Technical approach and discussion Production of the Full Product Set Resources requirements analysis

  31. Technical approach and discussion Production of the Full Product Set Resources requirements analysis

  32. Technical approach and discussion Diagnostic Data Set DDS objectives The processing system creates DDS granules for a minimum of 20 predefined diagnostic sites. DDS granule are extracted from Level-2 input products and Level-3 output products A DDS granule covers approximately 100km x 100km and comprises all layers of information of the input or output products. The granules are provided in the netCDF format following the globCOLOUR DDS format (following CF 1.0 rules)..

  33. Technical approach and discussion Diagnostic Data Set DDS important characteristics DDS are defined by center lon-lat and temporal period. Close DDS lead to almost identical DDS L2 products. DDS bins are computed using a nearest interpolation approach. One pixel of the L2 track is projected on a single DDS bin: holes exist in the built DDS map (see next slides). Identified by flags. DDS from L2 input products: PC 1 km DDS from L3 output products: ISIN 4.63 km (automatic reprojection on PC map performed by DDS tools)

  34. Technical approach and discussion Diagnostic Data Set DDS holes MERIS MODIS

  35. Technical approach and discussion Diagnostic Data Set DDS holes - flags

  36. Technical approach and discussion • Diagnostic Data Set • Diagnostic Sites • Two purposes • Sensors characterisation against in situ data • Cross-characterisation for specific biogeographical areas • The DDS location and spatial extent are selected from the existing validation sites, e.g. IOCCG/SIMBIOS DDS, ESA validation sites, using the same protocol of selection as the one used in SIMBIOS. .

  37. Technical approach and discussion Diagnostic Data Set Diagnostic Sites First pre-selection (1/2) .

  38. Technical approach and discussion Diagnostic Data Set Diagnostic Sites First pre-selection (2/2) GlobCOLOUR Diagnostic Data Set locations

  39. Technical approach and discussion Diagnostic Data Set Already generated DDS L2 products

  40. Globcolour processor status Version 1.0: used at ACRI and at UoP to generate DDS regression tests Version 2.0: to be released 1st of September

More Related