TES GDS Instrument Team Ground Data System Reports Scott Gluck Jet Propulsion Laboratory California Institute of Techno

1. TES GDS Instrument Team Ground Data System Reports Scott Gluck Jet Propulsion Laboratory California Institute of Technology Aura DSWG September 27, 2010

2. Overview • Current State of SIPS Data Production • Current Data Product Version • Current and Upcoming Releases • Issues and Other Items Aura DSWG

3. Current State of SIPS Data Production • TES GDS software Release R11.3 (L2 ESDT Version 4) completed reprocessing January ’10 • TES GDS software version: ‘R11.3’, active at SIPS from February ’09 (R11.0 from August ‘08) • L1B ESDTs • Version 003, File version F03_04 • L2 ESDTs • Standard products: Version 004, File Version F05_07 • Summary/Ancillary products: Version 004, File Version F02_07 • L3 ESDTs • L3 products (daily, monthly grid and browse): Version 002, File Version F01_07 • Global Surveys to date: 901, 100% processed with L2 ESDT Version 4 - 6+ Years of data! • Special Observations to date: 2479, 100% processed with L2 ESDT Version 4 Aura DSWG

4. SIPS R11 Reprocessing – Completed Aura DSWG

5. Data Processing Configuration • Three processing centers • SIPS: processes nominal Global Survey and Special Observations • SPC: ‘special’ processing: full filter, larger scale science investigations • SCF: supports development and small scale investigations • SIPS • 120 nodes with dual Intel Xeon quad-core CPUs with 16GB RAM and 2TB storage • 190 nodes with dual AMD Opteron single-core CPUs with 6GB RAM and 500GB data storage • Total of 310 nodes with 335TB data storage • SPC • 40 nodes with dual AMD Opteron single-core CPUs with 6GB RAM and 500GB storage • Total of 20TB data storage • SCF: 40 nodes with 16 GB memory per compute node, 120 TB data storage Aura DSWG

6. Hardware/processing upgrades • Additional cluster is scheduled to arrive on 9/22/10 and to be implemented in SIPS OPS in October 2010 • 60 nodes with dual Intel Nehalem Xeon quad-core CPUs with 24GB RAM and 2TB storage • Total of 120TB data storage • Noted processing timings • The new nodes have ~30% improved processing time • Data throughput bottleneck in Level 2 processing • Implementation of new file server data access methodology is being tested to alleviate PGE data bottleneck. Potential to provide significant improvement over current data staging on individual processing nodes. Aura DSWG

7. SIPS Data Throughput • TES safe-mode event and calibration scheme modification will reduced data volume by ~40% for 2010. • In the latter half of 2009, the TES pointing control system (PCS) started showing errors and late in 2009 the errors resulted in a degradation in TES data so data collection was halted. The cause was determined to most likely be shorting in one of the telemetry lines in the flex cable that supplies power and control signals to the TES PCS. In order to minimize wear and tear of the cable an alternate calibration methodology was derived and implemented. The TES instrument resumed data collection in April 2009. • TES Level 1B software required calibration and phase alignment updates, delivered July 2010 • SIPS has completed the processing of Global Survey and Special Observation data collected while software upgrades to accommodate the calibration schema change were in progress. SIPS was delivering 15-24 Global Surveys/week during this period while keeping up with forwarding processing of TES data. • Last month SIPS delivered the following volume to ASDC under a rate of 3x; where 1x is the volume for 3.5 Global Surveys: • L1B: 86 GB L2: 26 GB L3: 1 GB Aura DSWG

8. Calibration changes & Impact on file naming convention • To account for the TES modification to the on-board data collection techniques and changes to the calibration during telemetry processing there is now two output file naming conventions. The ground data software will calibrate the data with different algorithms but they will be scientifically compatible. To distinguish between the calibration methods a designator has been incorporated into the output product names and information will be written to the file metadata. • Standard calibration scheme with embedded blackbody and space view measurements • Global surveys with Run IDs <= 11124 => TES-Aura_L2-CO2-Nadir_r<RunId>_Fff_cc.he5 • Special Observations with Run IDs <= 11188 => TES-Aura_L2-CO2-SO-Nadir_r<RunId>_Fff_cc.he5 • New calibration scheme with split-cal blackbody and space view measurements • Global surveys with Run IDs >= 11125 => TES-Aura_L2-CO2-Nadir_r<RunId>_C01_Fff_cc.he5 • Special Observations with Run IDs >= 11189 => TES-Aura_L2-CO2-SO-Nadir_r<RunId>_C01_Fff_cc.he5 Aura DSWG

9. TES Level 2 Data Version 4 The Version 4 TES L2 standard product data includes: Improvements to the temperature retrievals (updated CO2 spectroscopy) Improvements to the methane retrievals Species dependent quality control information Validation analyses are underway; Validation report will be provided late this year. A TES Global Survey quality statement has been released detailing almost no systematic differences between the new versus old calibration strategy. Data Quality Statement and Data Users Guide provided to ASDC for Version 4 (http://eosweb.larc.nasa.gov/PRODOCS/tes/table_tes.html) Aura DSWG

10. Future State of SIPS Data Production • TES GDS software Release R12 reprocessing scheduled, October 2010 - January 2012 • ESDT updates for Release 12 • L1B ESDTs • Version 004, File version F04_06 • L2 ESDTs • Standard products: Version 005, File Version F06_08 • Summary/Ancillary products: Version 005, File Version F03_08 • L3 ESDTs • L3 products (daily, monthly grid and browse): Version 003, File Version F01_08 Aura DSWG

11. Release 12 and Upcoming Releases • Release 12 product enhancements • New standard product files: Ammonia (NH3 ), Carbon Dioxide (CO2 ), CO2 Lite (reduced pressure grid) • Ozone (O3 ) tropospheric column from ancillary to all standard products • Add Instantaneous Radiative Forcing Kernel (IRFK) of outgoing longwave radiation (OLR) computation of Ozone (O3 ) L2 standard products • Update Ocean/Ice emissivity algorithm to accurately determine whether water is frozen or not. • Updates to Surface Temperature algorithm for cases when a scene is a freshwater target surrounded by land. • Updates to daily and monthly Level 3 plots to help differentiation based on pressure levels and color scales. • Future Product enhancements • NetCDF compatibility • GEOS-5 update incorporation • Joint TES-MLS Carbon Monoxide (CO) and TES-OMI products (delivery dates TBD) • New standard products for Methanol (CH3OH) and Formic Acid (HCO2 H) (delivery dates TBD) Aura DSWG

12. Accessing TES data • ASDC Data Pool & WIST Data access; all V3 and V4 data available • Prototype ESDT Version 005 Level 2 standard products have been successfully ingested at ASDC in preparation for Release 12 processing. • Documentation & IDL data readers available from • http://eosweb.larc.nasa.gov/PRODOCS/tes/table_tes.html • IDL readers provided early in each release, and are backward compatible • Documentation provided via ASDC site • Data Product Specification • Level 2 & Level 3 User’s Guides • Data Quality Statements • Data Versioning • Validation Reports • ATBDs Aura DSWG

13. TES Web Page Update • New look and feel using “JPL common” attributes • The color theme shall be based on the following criteria: • Atmosphere: Purple • Ocean: Blue • Land: Green • Other: Amber • Updated Science team bios • Access to all publications, presentations and posters generated since launch Aura DSWG

14. Issues & Other Items • Naming convention change (due to calibration schema modification) could affect product reader tools • Looking for a firm schedule for GEOS-5 delivery Aura DSWG