GODAE Quality Control Pilot Project

1. GODAE Quality Control Pilot Project Status Report August 2007 IGST Meeting Department Fisheries Ocean St. John’s, New Foundland, Canada Outline • New Data Sources • Output File and Reference Data • Plans and Issues • Argo FSI Float QC Outcomes

2. GODAE Quality Control Pilot Project New Data Sources • UKMO • long list form of UKMO QC output now available in real-time • UKMO back filled long list data to 2003 – thanks to Matt Martin • long list provides additional QC outcomes and both in situ and potential temperatures (short list had only potential temperatures) • convention is to use in situ temperature in QC databases • in situ calculated from potential when salinity reported • not always the case in the UKMO short list files • BMRC • pushing QC data from BoM system to GODAE server • currently have data for 2005 and 2006 (nothing from 2007 yet) • working on BMRC data decoder

3. GODAE Quality Control Pilot Project New Output File Format and Reference Data • Call sign database conversion to NetCDF format complete • based on Coriolis/Argo multiple profile NetCDF file structure • easily expanded with new QC variables or new QC data providers • sorted by observation time and data center for ease of use • Navy GDEM climate and climate variability profiles added as reference profiles in call sign database • temperature and salinity climatology • global, monthly, 0.25 degree resolution • 78 vertical levels (47 in upper 1000 m of water column) • climate mean and standard deviations interpolated to observed levels for month in which observation taken

4. GODAE Quality Control Pilot Project New MEDS QC Variable in Call Sign Database • decoded MEDS hex bit codes on QC tests performed Meds QC Tests (binary string bit position from the RIGHT): Bit # - Test 0 - Platform Identification 1 - Impossible Date/Time 2 - Impossible Location 3 - Position on Land 4 - Impossible Speed 5 - Impossible Sounding 6 - Global Impossible Parameter Values 7 - Regional Impossible Parameter Values 8 - Increasing Depth 9 - Profile Envelop 10 - Constant Profile 11 - Freezing Point 12 – Spike 13 - Top and Bottom Spike 14 – Gradient 15 - Density Inversion 16 - Levitus Seasonal Clim 17 - Emery and Dewar Clim 18 - Asheville Clim 19 - Levitus Monthly Clim 20 – Waterfall 21 - Cruise Track 22 – Profiles 23 – Bottom 24 - Temperature Inversion • experience indicates only two tests flagged as typically performed • Levitus seasonal climate check and temperature inversion

5. GODAE Quality Control Pilot Project Plans • initialize call sign databases with 2003-2006 production run • NetCDF call sign databases will be hosted on US GODAE server • target date for completion Oct 2007 Issues • US GODAE server hardware upgrade has never taken place • IBM X-346 machines plus 30 TB storage purchased years ago by ONR • never installed due to security issues – completion date still unknown • GODAE QC application cannot run on existing server hardware • cannot do real-time updates of call sign databases as planned • in process of purchasing additional disk space for personal Linux workstation to support GODAE QC function • using my Linux workstation (at best) only monthly updates of NetCDF call sign databases on GODAE server possible

6. GODAE Quality Control Pilot Project FSI Argo Floats Pressure Offset Errors • discovered in February, 2007 that Argo profiles from SOLO floats with FSI CTD (Argo Program WHOI) have incorrect pressures • profile data may be offset upward by one or more pressure levels, resulting in a significant cold bias for the data • a data denial list was posted on the Argo web site (287 WMO IDs) • data no longer distributed on GTS • Argo science team asked if bad FSI float data had been accepted into operational assimilation systems • FSI float QC outcomes from the NRL/FNMOC real-time QC system were evaluated from 2004 through 2006 • instrumentation error checks • climate background field check • cross validation (buddy) check

7. float track error probability of entire profile (reject if p > 0.95) GDEM climate anomaly cross validation anomaly error probability of profile level (reject if p > 0.95) NRL FSI Float QC Outcomes Display

11. GODAE Quality Control Pilot Project Summary NRL/FNMOC QC of FSI Argo Float Pressure Errors • rejected 11% of the float temperature profiles  • rejected 5% of the float temperature levels • 185 float profiles rejected due to inadequate vertical sampling as a result of the vertical level rejections (created gaps in profiles) Positive feedback from Howard Freeland on call sign databases • From: Freeland, Howard [mailto:FreelandHj@pac.dfo-mpo.gc.ca] Sent: Wednesday, August 01, 2007 2:58 PMTo: Cummings, James [cummings@nrlmry.navy.mil]Subject: Rejected floatsI think the call sign files are exactly what I'd like to look at.  I've been doing some simple investigations about data being rejected by less sophisticated users than GODAE people and have some interesting results...

12. END

13. Call Sign Database Structure

14. NRL MEDS UKMO Coriolis BMRC GODAE QC Application Program Interface (API) read different QC data formats, check for valid call sign, match profiles (lat, lon, time, call sign), populate database structure with observation, QC codes, QC flags, and supplemental information, sort by observation time, update NetCDF call sign database 52007 TAO Mooring 1900305 Argo Float A8ES9 Ship XBT Call Sign Databases