1 / 15

Overview of NOAA/NESDIS GOES-R Hyperspectral Sounder Data Compression Study

Overview of NOAA/NESDIS GOES-R Hyperspectral Sounder Data Compression Study. Bormin Huang, Allen Huang, Alok Ahuja Cooperative Institute for Meteorological Satellite Studies University of Wisconsin-Madison 4th MURI Workshop, April 27-28. What is hyperspectral sounding data?

presta
Download Presentation

Overview of NOAA/NESDIS GOES-R Hyperspectral Sounder Data Compression Study

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Overview of NOAA/NESDIS GOES-R Hyperspectral Sounder Data Compression Study Bormin Huang, Allen Huang, Alok Ahuja Cooperative Institute for Meteorological Satellite Studies University of Wisconsin-Madison 4th MURI Workshop, April 27-28

  2. What is hyperspectral sounding data? • It is generated from an interferometer (e.g. HIS, AERI, CrIS, • IASI) or a grating sounder (e.g. AIRS). • It consists of several thousand spectral channels that span the • infrared region on the order of one wavenumber or less • What is hyperspectral sounding data for? • to retrieve • - atmospheric temperature, water vapor, and trace gases profiles; • - cloud & aerosol properties, • - surface temperature, emissivities, etc., • to derive wind from radiance or retrieved water vapor fields, • for better weather and climate prediction. • Why does it require compression? • Unprecedented volume of 3D data that consists of one spectral • and two spatial dimensions (~10-100 GB per day) ; • Beneficial to efficient data transfer and archive. • What is new for the data compression society? • High correlation among remote disjoint channels due to the • absorption of the same absorbing gases. • Lossy compression needs retrieval impact studies, i.e. • interdisciplinary knowledge in data compression and • remote sensing is needed! • Why is lossless or near-lossless compression desired? • Physical retrieval of atmospheric temperature and • absorbing gases is a mathematically ill-posed problem, • i.e. sensitive to data error and noise!

  3. Lossless Compression Study • 2D Wavelet-Based Compression Scheme JPEG2000: 2D IWT → Bitplane Coding → Entropy Coding • 3D Wavelet-Based Compression Schemes 3D IWT →3D EZW→ Entropy Coding 3D IWT →3D SPIHT→ Entropy Coding 3D IWT → 1D BWT → Entropy Coding • 2D Predictor-Based Compression Schemes CALIC: 2D Gradient-adjusted Prediction → Entropy Coding JPEG-LS: 2D Nonlinear Prediction → Entropy Coding

  4. 2D Wavelet Transform Integer Wavelet Transform (Lifting Scheme)

  5. Wavelet based Schemes • JPEG2000 • A new ISO/IEC (International Organization for Standardization/International Electrotechnical Commission) compression standard. • Successor to the DCT (discrete cosine transform)-based JPEG algorithm. IWT with 3 stages (Taubman et. al. 2000)

  6. 3D Wavelet Tree Coding 3D EZW: It uses the spatial hierarchical tree relationship of the wavelet transform coefficients for efficient compression. 3D SPIHT: Refinement of the EZW scheme that provides better compression while having faster encoding and decoding times. Parent-child interband relationship and locations for EZW and SPIHT coding

  7. i nn nne nw n ne ww w ? j Predictor-Based Schemes • 2D Context-based Adaptive Lossless Image Codec (CALIC) • Among the nine proposals in the initial ISO/JPEG evaluation in July 1995, CALIC was ranked first. • It is considered the benchmark for lossless compression of continuous-tone images. Neighboring pixels used in prediction (Wu et. al. 1997) Schematic description of the CALIC encoder

  8. c b d a x • 2D JPEG-LS • Published in 1999 as a lossless compression standard of the ISO/IEC. Neighborhood of JPEG-LS used in prediction Schematic description of the JPEG-LS encoder

  9. Burrows Wheeler Transform • Block-sorting compression scheme [Burrows et al, 1994] • Rearranges the positions of the data such that the few distinct values under the same previous context are grouped together in position. tennessee*tennessee* ennessee*t          *tennessee nnessee*te          ssee*tenne nessee*ten          e*tennesse essee*tenn          nnessee*te ssee*tenne          nessee*ten see*tennes          essee*tenn ee*tenness          see*tennes e*tennesse          ee*tenness *tennessee          ennessee*t An example of the Burrows-Wheeler transform. bwt(tennessee*) = t*sennesee. The matrix on the right is obtained by sorting the rows of the left matrix in right-to-left lexicographic order. * denotes end of the data block and can be considered as the smallest symbol.

  10. Ten selected AIRS granules on Sept. 6, 2002 AIRS radiance field at wavenumber 900.3cm-1 for the selected granules

  11. AIRS radiance field at wavenumber 900.3cm-1 for the selected granules

  12. Compression ratios of different algorithms for the 10 selected AIRS granules

  13. Bias-Adjusted Reordering (BAR)* Scheme • for Data Preprocessing • Hyperspectral sounder data features strong correlations in disjoint spectral regions affected by the same type of absorbing gases at various altitudes. • The Bias-Adjusted Reordering (BAR) scheme is used for exploring the correlation among remote disjoint channels. • The technique can be used to improve the compression ratio of any existing scheme. • The BAR scheme paper is accepted to be published in Optical Engineering. • We are in the process of patent application.

  14. Effect of the BAR scheme on various compression algorithms for the 10 selected AIRS granules

  15. Summary • In support of the NOAA/NESDIS GOES-R data processing studies, we investigated lossless compression of 3D hyperspectral sounding data using wavelet-based schemes (3D EZW, 3D SPIHT, JPEG2000) and predictor-based schemes (CALIC, JPEG-LS). • The performance rank from best to worst in terms of compression ratios before the BAR scheme is given in the order of JPEG-LS, 3D SPIHT, JPEG2000, CALIC, BWT and 3D EZW. • The performance rank from best to worst in terms of compression ratios after the BAR scheme is given in the order of JPEG-LS, JPEG2000, CALIC, 3D SPIHT, BWT and 3D EZW. • To take advantage of the spectral correlations, we applied the BAR scheme to significantly improve the compression performance of all the compression algorithms. Acknowledgement: This research is supported by NOAA NESDIS OSD under grant NA07EC0676.

More Related