190 likes | 333 Views
Quality control and dissemination of real time data - methodology and advanced available technological tools the Coriolis data centre experience Gilbert Maudire, Loïc Petit de la Villéon, Catherine Maillard SEAMER 2000. Pre-operational and Operational Oceanographic Programmes.
E N D
Quality control and dissemination of real time data - methodology andadvanced available technological tools the Coriolis data centreexperience Gilbert Maudire, Loïc Petit de la Villéon, Catherine MaillardSEAMER 2000
Pre-operational and Operational Oceanographic Programmes • Pre-operational data management TOGA-WOCE-CLIVAR Global Subsurface Data Centre • CORIOLIS-ARGO real-time data processing • Pommier, Pomme • GYROSCOPE (Atlantic) • MFSPP (Mediterranean)
New Questions Asked • Access to observational data of temperature within 12 hours after their production • Access to other types of data within 48 hours within their production • Access to statistics, gridded data, combinations with delayed mode data, graphs, maps and products • Assess a standardised level of quality to the data
Sensors currently producing time data • Lagrangian profilers : PROVOR, APEX.. • XBt, Xctd • Thermosalinographs
CORIOLIS Data Centre • to process European floats • provor • other floats • to combine various in-situ measurements • to insure interoperability with argo data centres • data formats • common quality procedures • real time data exchanges • to gather global data sets for European users Partners : IFREMER, SHOM, IRD (ORSTOM), METEO-FRANCE, CLS-ARGOS
data centre functions Interoperability with argo gts (meteo-france) control data quality controltechnical monitoring of equipments WWW (full data set) data collect mail internetgtscd-rompaper... data base (RDBS)catalogue - archive models (Mercator, Soprane) disseminationpublic / private mailmailing lists productsgraphics and chartsstatisticsanalyses cd-romor other supporton request quality insurance for exploitation hardware & software management - backup
a provor cycle surface drift profile10 hours of argos transmission descending profile10 hours ascending profile5 hours immersion drift profile5 to 15 days
provor incoming data argos internet ifremer cls-argos ads service mail data processing store the message check the address sender generate the cycles activate decoding • internet mail major issues • data processing naturally performed « on the fly » • data integrity (no loss or errors on data) • smtp protocol includes re-transmission in case of trouble • strategic service for ifremer • standard service from telecom operators
quality control • complies with international recommendations • IOC, ICES, MAST • two levels • automatic control • visual control for dubious profiles • gtspp quality flag scale
quality control qc1 qc2
*Provor profiler : <Identificateur flotteur PROVOR> *Float cycle : <n° de cycle> *Descending vertical profile *Date : yyyy-mm-dd hh24:mi (QC=9), Estimated position : <Latitude>, <Longitude> (QC=9) Date Time Pressure Temperature Conductivity Salinity (yyyy-mm-dd) (hh24:mi) (dBar) (°C) (mS/cm) (P.S.U.) 1999/10/07 23:58 2 27.52 44.90 1999/10/08 00:04 56 18.21 44.81 *Ascending vertical profile *Date : yyyy-mm-dd hh24:mi (QC=9), Estimated position : <Latitude>, <Longitude> (QC=9) Date Time Pressure Temperature Conductivity Salinity (yyyy-mm-dd) (hh24:mi) (dBar) (°C) (mS/cm) (P.S.U.) 1999/10/15 10:30 3980 2.52 44.90... 1999/10/15 10:40 2378 5.21 44.81 *Immersion drift *Date : <Heure de fin de descente> Date Time Pressure Temperature Conductivity Salinity (yyyy-mm-dd) (hh24:mi) (dBar) (°C) (mS/cm) (P.S.U) 1999/10/08 04:55 1200 2.22... *Surface drift Date Time Latitude Longitude Position quality Surface temperature (yyyy-mm-dd) (hh24:mi) (HDD MI.mmm) (HDD MI.mmm) (Argos scale) (°C) provordecoded cycle ascii file *Technical status *Probes : Internal time : hh24 :mi Pressure offset : 99.99 dBars Internal pressure : 99.99 dBars CTD drift measurements : 999 *Engineering : Valve motions : Surface : 99, Descending : 99 Pump motions : Surface : xx, Descending : xx, Immersion : xx First stabilization : dd/mm *Data reduction : Descending profile : PRSEP : 9999 dBars, NTS : 9999, ETS : 9.99°C, NCS : 9999, ECS : 9.99 mS/cm NTF : 9999, ETF : 9.99°C, NCF : 9999, ECF : 9.99 mS/cm Ascending profile : PRSEP : 9999 dBars, NTS : 9999, ETS : 9.99°C, NCS : 9999, ECS : 9.99 mS/cm NTF : 9999, ETF : 9.99°C, NCF : 9999, ECF : 9.99 mS/cm *Transmission : Received frames : 9999, complete : 9999, CRC OK : 9999, recombined frames : 9999 *Positionning : Class 1 : 9999, Class 2 : 9999, Class 3 : 9999, Rejected locations: 9999
formats • dissemination • ascii : medatlas/coriolis • binary : netcdf • gts/tesac • others on request (gtspp, tsdc) • data collect • provor, sippican mk12 • gts (bathy, tesac) • gtspp • medatlas/coriolis
www.ifremer.fr/coriolis provor on the web
atalante xbt and ts data processing quality control gtsdissemination coriolis data base oracle8derived from toga/woce webinterface atalante mail inmarsat uploading disseminationto models
tables of coriolis data-base temp-vproftvp_station_id tvp_immersion tvp_value tvp_value_qc temp-vprof-headertvh_station_id tvh_profile_qc tvh_level_number tvh_max_level tvh_digitization tvh_validation plateform pl_plateform_code pl_plateform_name pl_plateform_nodc_code pl_plateform_type pl_institution_code provor-config pr_plateform_code pr_type pr_version pr_crc_type pr_cycle_period pr_immersion_drift_period pr_launch_datetime pr_launch_latitude pr_launch_longitude pr_reference_datetime pr_reference_internal_date pr_probe_code pr_experiment_id recorder re_recorder_code re_recorder_description probe pb_probe_code pb_probe_description pb_speed_coef_a pb_speed_coef_b psal-vprofpvp_station_id pvp_immersion pvp_value pvp_value_qc psal-vprof-header pvh_station_id pvh_profile_qc pvh_level_number pvh_max_level pvh_digitization pvh_validation station st_station_id st_station_type st_station_date st_date_qc st_station_latitude st_station_longitude st_position_qc st_ocean st_marsden st_experiment_id st_station_number st_plateform_code st_plateform_motion st_plateform_nodc_flag st_institution_code st_institution_gtspp_flag st_archiving_date st_probe_code st_recorder_code st_archiving_status st_dupliate_flag st_duplicate_station_id st_nature st_qc_status provor_programming pg_plateform_code pg_prog_parameter pg_prog_value cndc-vprofcvp_station_id cvp_immersion cvp_value cvp_value_qc cndc-vprof-headercvh_station_id cvh_profile_qc cvh_level_number cvh_max_level cvh_digitization cvh_validation provor_tech_parametertp_tech_parameter tp_description tp_programing tp_status 2 2 immersion_driftid_plateform_code id_cycle_number id_start_station_code id_end_station_code id_date_time id_date_qc id_latitude id_longitude id_location_qc id_immersion id_immersion_qc id_temperature id_temperature_qc id_conductivity id_conductivity_qc id_salinity id_salinity_qc surface_drift sd_plateform_code sd_cycle_number sd_start_station_code sd_end_station_code sd_datetime sd_date_qc sd_latitude sd_longitude sd_location_class sd_location_qc sd_temperature sd_temperature_qc sd_conductivity sd_conductivity_qc sd_salinity sd_salinity_qc provor_tech_status pg_plateform_code pg_cycle_number pg_prog_parameter pg_prog_value beaconsbc_beacon_code bc_plateform_code
provor cycling cycle period cycle period launching reference date reference date + 1 period argos argos argos cycle 0 does not match the period cycle 1 cycle 2