1 / 8

NDE XML Tailoring Service

Develop a service that reduces labor effort to create user-specific tailored products from HDF and HDF-EOS files by providing routine data manipulation and configurable output options.

lewisdonald
Download Presentation

NDE XML Tailoring Service

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. NDE XML Tailoring Service HDF and HDF-EOS Workshop XIII November 4, 2009 Tom Feroli, Peter MacHarrie

  2. Goal • Reduce the labor effort to create user-specific tailored products (I.e., image, gridded, map reprojected, and other specific file formats, etc.) • Objective • Develop (InputFormat + Output Format) software applications instead of (Input Format * Output Format) software applications. • Approach • Develop a service that provides routine data manipulation (i.e., sub-setting, sub-sampling, filtering, aggregation) and places output into a well-defined intermediate format. • Create tailoring applications that accept a well-defined intermediate format and configurable output options to create a specific end-user product.

  3. Data Selection Driver XML Reads Writes Data Selection Driver Uses Binary Format (Subsample, Subset, Aggregate, Filter, Scale/Offset) HDF5+ netCDF4 netCDF4 netCDF3 • Modular design, accepts drivers to accommodate additional input • and output file formats (e.g., HDF-EOS5, netCDF3, etc.)

  4. Enterprise “Measure” • Enterprise Measure is a logical construct • It is the concatenation of all related arrays in all files in an “enterprise” • File 1 – Array1 (Scan = 12, Sample=96, Channel=22) • File 2 – Array1 (Scan =12, Sample = 96 , Channel=22) • Enterprise Measure1 (Scan=24, Sample=96, Channel=22) • Enterprise Measure reference “Enterprise Dimensions” • A file, is a storage unit, for a partition, of the Enterprise Measure • The partition is a range of values of one or more Enterprise Dimensions • File 1 – Scan Dimension 1 -12 • File 2 – Scan Dimensions 13-24

  5. Enterprise “Dimension” • Enterprise Dimension is a unique logical mapping to dimensions within all data files. • In VIIRS, Image and Moderate product profiles, the dimension AcrossTrack is defined as 6400 and 3200 • Enterprise dimension are named uniquely for each • AcrossTrack-I 6400 • AcrossTrack-M 3200

  6. Enterprise Data Cube • Data Cube is a structure of one or more Enterprise Measures. Within a data cube: • Measures share the same dimensions • Measure names are unique • Measures may have different data types • In VIIRS SDRs all arrays are named Radiance • Enterprise measure uniquely name each array • Example • Radiance_I1 -> Product I1->Radiance Array • Radiance_I2 -> Product I2->Radiance Array

  7. Enterprise Data Structures Tailoring Service XML References Enterprise Data Structures

  8. Measures (additional information) • Named Subset • Array: AntennaTemperature(12, 96, 22) • Measure: AntennaTemperature, references entire AntennaTemperature array • Measure: AntennaTemp-Channel_1, references array AntennaTemperature, with sub-sample specification of Start=(1,1,1), Stride=(1,1,1), Count=(12,96,1) • Named BitMask • Array: QF1_VIIRSCMIP(768, 3200) • Measure: CloudConfidencePixel • Has Integer DataType • References array QF1_VIIRSCMIP starting at bit 2 for 2 bits • Scale and Offset • Measure defined as float, references an integer array and its scale and offset are applied • Measures can be referenced in the <select> or <filter> elements of the tailoring XML. • <where><filter><measure name=‘CloudConfidencePixel’> • <operation value=‘LT’><operand value=‘2’/></filter></where> • (i.e., CloudConfidencePixel < 2)

More Related