1 / 8

DSN Raw Data Incompatible Binary Formats

DSN Raw Data Incompatible Binary Formats. PDS4 Operational Readiness Review and Acceptance Review 30 November 2011. DSN Radio Science Data Flow. 1980 ANTENNA. 2012 ANTENNA. ATDF (TRK-2-25). TNF (TRK-2-34). RSR (0159-SCIENCE). ODR (RSC-11-3). ARCHIVED. TDM 0212-TRACKING. ODF

jethro
Download Presentation

DSN Raw Data Incompatible Binary Formats

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. DSN Raw DataIncompatible Binary Formats PDS4 Operational Readiness Review and Acceptance Review 30 November 2011

  2. DSN Radio Science Data Flow 1980 ANTENNA 2012 ANTENNA ATDF (TRK-2-25) TNF (TRK-2-34) RSR (0159-SCIENCE) ODR (RSC-11-3) ARCHIVED TDM 0212-TRACKING ODF (TRK-2-18) ODF (TRK-2-18) ANALYSIS SPK FILES MASS/DENSITY GRAVITY MODELS GRAVITATIONAL WAVES ATMOSPHERIC DRAG ATMOSPHERIC TEMP/PRESSURE ELECTRON DENSITY PROFILES RING OPACITY/PARTICLE DENSITY SURFACE ROUGHNESS/DENSITY BINARY ARCHIVED ASCII

  3. Case 1:ODF Data Record Format(36 8-bit bytes = 288 bits) Time Tag fits in 32-bit word Time Frac + D/L Delay fit in 32-bit word but don’t align with 8-bit bytes 1, 2, 3, 6, and 7-bit fields RF H/P wraps over 32-bit boundary and does not align

  4. Case 2: RSR Data Sample Packing Real and Imaginary parts of complex samples are segregated Time order of samples within 16-bit words is reversed One of these structures is repeated thousands of times per record

  5. Option 1 • Allow arbitrary bit field binary formats (allow ‘descriptive’) • Data provider decides bit field lengths • Data provider decides start bits • Data provider describes interpretations • Pros and Cons: • Preserves instrument output • Violates prescriptive vs descriptive philosophy • Applicability to other data providers (MIPL, MSSS, …)? • Makes PDS4 tool development harder/impossible • No changes needed by DSN or RS community • Discourages ‘new investigator’ browsing

  6. Option 2 • Require conversion to compliant formats (current default) • Ad hoc PDS conversions (dangerous?) • Work with DSN on conversion tools (unlikely) • ASCII, FITS, HDF5, or TBD which could be judged compliant • Pros and Cons • DSNandRS community unlikely to use archive • Improved browsing capabilities • Major conversion effort • Software • 5 binary products (ATDF, ODF, ODR, TNF, and RSR) • TNF hasover a dozenrecord types (SIS includes 77 pages of descriptive tables) • ODF is in its 6th generation (6 versions of SIS); each ODF version has multiple record types • Who will validate, how, and when? High probability of conversion error • ‘Inverter’ software, needed for at least some validation, could be public? • Operations • Virtually every radio science raw data archive is built around these products • DSN has been generating some products for more than 30 years • Deliveries of TNF and RSR will continue indefinitely (ODF limited to ‘old’ missions)

  7. Option 3 • DSN binaries accepted as encoded byte streams • PDS points to DSN documentation • Pros and Cons • Preserves instrument output • Applicability to other data providers (MIPL, MSSS, …)? • No PDS4 tool development required (?) • No changes needed by DSN or RS community • Discourages ‘new investigator’ browsing • Raw data will be less accessible than under PDS3

  8. Recommendation • Accept arbitrary bit field formats for instrument output only (the most raw data) • Notes: • This does not allow ODF, the most popular radio tracking data type • But the ODF is being phased out. • Convert non-compliant binaries to compliant formats as interest demands and resources allow • Requires PDS to develop, maintain, and make available conversion software (but only for the PDS4 ‘platform’ by EN programmers) • Expect ‘browsing’ users to work with the converted products • Pros and Cons • Preserves instrument output • Makes ‘usable’ products available, but only as interest dictates • PDS can choose format of converted data (such as ASCII, FITS, or HDF5) • Meshes with current DSN and RS community needs • Doubles(?) volume of converted data sets

More Related