1 / 40

IEEC Data Compression Group: Presentation of activities

Explore the activities of IEEC Data Compression Group in developing advanced data compression techniques for space missions, including Gaia project optimization and cutting-edge algorithms like PEC and FAPEC.

kbruner
Download Presentation

IEEC Data Compression Group: Presentation of activities

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. IEEC Data Compression Group:Presentation of activities Enrique García-Berro (UPC) Jordi Portell (UB) Alberto G. Villafranca (IEEC) IEEC Data Compression Group | Presentation of activities | GTD, Barcelona, 25 January 2010

  2. Outline • The IEEC Data Compression Group (DCG) • Experience • Gaia PDHS • On-board clock simulation • Optimization of Gaia science TM downlink • TM CODEC • Optimum data compression for Gaia (GOCA) • Current activities • PEC and FAPEC • New entropy coders • Applications of PEC and FAPEC • Hardware implementation of FAPEC • Forthcoming work, outreach and collaborations

  3. The IEEC DCG • Members of the Team: • 2 Doctors in Physics • 1 Electronics Engineer • 2 Telecommunication Engineer • External support: • ESA scientists and engineers • UB scientists

  4. Outline The IEEC Data Compression Group (DCG) Experience Gaia PDHS On-board clock simulation Optimization of Gaia science TM downlink TM CODEC Optimum data compression for Gaia (GOCA) Current activities PEC and FAPEC New entropy coders Applications of PEC and FAPEC Hardware implementation of FAPEC Forthcoming work, outreach and collaborations

  5. Gaia PDHS

  6. Gaia clock simulation

  7. Gaia downlink optimization • Optimized telemetry formats and timing schemes(CCSDS-compliant)

  8. Gaia downlink optimization • Optimization of Groundstation contact periods

  9. Telemetry CODEC

  10. Preliminary Gaia compression • Two-stage strategy • Tailored pre-processing • No CPU considerations

  11. Preliminary Gaia compression • Further refinements (improved pre-processing) • First flight-realistic considerations (packet size) • Even better ratios

  12. GOCA (IEEC/GTD) • Algorithm: Pre-processing and coding stages • Creation of the PEC entropy coder

  13. GOCA (IEEC/GTD) • Flight-realisticsoftware prototyping • Representative operationand CPU consumption

  14. GOCA (IEEC/GTD) • Detailed algorithm testing

  15. Outline The IEEC Data Compression Group (DCG) Experience Gaia PDHS On-board clock simulation Optimization of Gaia science TM downlink TM CODEC Optimum data compression for Gaia (GOCA) Current activities PEC and FAPEC New entropy coders Applications of PEC and FAPEC Hardware implementation of FAPEC Forthcoming work, outreach and collaborations

  16. Prediction Error Coder (PEC) • Current standard for lossless data compression in space (CCSDS 121.0): • Too sensitive to outliers (cosmic rays, noise, etc) • Caused by the Rice core

  17. Prediction Error Coder (PEC) • Goals: • Very quick operation • Ratios as uniform (and high) as possible • Three coding options + segmented coding • Input: signed integers (use of “–0” code) • Automatic calibrator • Representative histogram of the data

  18. Prediction Error Coder (PEC) • Nearly-optimal performancewith clean distributions(two-sided Geometric) • Almost the same performancewith 10% of outliers! (see Rice with 0.1%...)  Yes, PEC is robust!

  19. Fully Adaptive PEC (FAPEC) • Adaptive stage of PEC: • Data accumulator (typ. 100-1000 samples) • Optimized histogram calculated on-the-fly • Histogram analysis  nearly-optimal PEC configuration • Computational cost:similar (slightly lower) than CCSDS 121.0! • Patented

  20. Tests with synthetic data Discrete Laplacian, 16-bit samples 0.1% flat noise added: FAPEC almost identical to Best PEC  OK! FAPEC very close to CCSDS 121.0  OK!

  21. Tests with synthetic data Discrete Laplacian, 16-bit samples 1% flat noise added: CCSDS 121.0 starts decreasing its efficiency… FAPEC efficiency is almost identical as before!

  22. Tests with synthetic data Discrete Laplacian, 16-bit samples 10% flat noise added: CCSDS 121.0 is not optimal anymore FAPEC still above 80%-90% in most cases!

  23. FAPEC real performance • Results assessed on real (and simulated) data: • Larger symbol sizes seem to benefit PEC/FAPEC (interleaving) • Better pre-processing benefits PEC/FAPEC • Overall, PEC/FAPEC perform at least as well as CCSDS 121.0

  24. Sub-Exponential + CCSDS • Simple improvement to CCSDS 121.0 • Substitute Rice by Sub-Exponential codes • (used e.g. in FELICS) • Resiliency improvement! (close to FAPEC) • …but more CPU-expensive

  25. HyPER Coder • Rice good for clean and compressible data • PEC/FAPEC good for very noisy data • Could we join them?  HyPER: Hybrid PEC/Rice coder • Still experimental, but promising!

  26. PEC and HDF5 • On-ground processing of Gaia (DPAC) • Massive parallelization in MareNostrum (>1000 CPU) • I/O  bottleneck • HDF5: • File format for optimum access and efficient storage • Inclusion of PEC compression (quick and stable) • Still in development • Preliminary tests:HDF5+PEC leads to similar sizes as Java Serialized + Zip • If successful, may be used in the final Gaia pipeline

  27. GPS compression with FAPEC • WHY? • Remote applications relying on GPS present communication problems Bandwidth is usually very small • The best compressor available could be improved thanks to FAPEC

  28. GPS compression with FAPEC • Improvements in the pre-compression stage…

  29. GPS compression with FAPEC • … and in the compression

  30. Embedded FAPEC (FPGA) • WHY? • Half of the > 20 missions with the CCSDS 121.0 have used hardware implementations Embedded implementation is mandatory to cover all the requirements of the missions • Optimal power performance • High throughput

  31. Embedded FAPEC (FPGA) • New version of the algorithm, more HW-friendly

  32. Embedded FAPEC (FPGA) • SCENARIO • 16 bits samples @ 2 MHz → 32 Mbps • Parallel input (16 bits) + Serial Output (up to 32 MHz) • < 10 Kbits of internal memory required • TARGET ACTEL RTAX (antifuse and rad-hardened) • Representative PROTOTYPE  PROASIC3L

  33. Embedded FAPEC (FPGA)

  34. Embedded FAPEC (FPGA)

  35. Embedded FAPEC (FPGA)

  36. Embedded FAPEC (FPGA) • 35 mW • ~ 100 usec. LATENCY • USAGE • ~ 10 % cell • ~ 15 % of internal memory • RTAX250S → 2 x FAPEC (64 Mbps) • RTAX4000S → > 30 x FAPEC (> 1 Gbps) • FUTURE  48 Mbps @ 3 MHz / FAPEC core

  37. Embedded FAPEC (FPGA) • Real Data Performance

  38. Outline The IEEC Data Compression Group (DCG) Experience Gaia PDHS On-board clock simulation Optimization of Gaia science TM downlink TM CODEC Optimum data compression for Gaia (GOCA) Current activities PEC and FAPEC New entropy coders Applications of PEC and FAPEC Hardware implementation of FAPEC Forthcoming work, outreach and collaborations

  39. Outreach and forthcoming work • PEC/FAPEC presented at SPIE OP’09 • High interest: Meteosat-3, Wisconsin (US), Xidian (China)… • Sub-Exp CCSDS + FAPEC@GPS towards SPIE OP’10 • Several papers and proceedings • Further improve FAPEC hardware implementation • Implementation in ASIC?  Complete product! (patent) • Data compression for future missions • LISA, PLATO, Meteosat-3?, etc. • Finish study on alternative entropy coders • SubExp real alternative to 121.0? • HyPER even better than FAPEC? • Finish study on HDF5+PEC • Study other applications of PEC/FAPEC

  40. IEEC Data Compression Group:Presentation of activities Enrique García-Berro (UPC) Jordi Portell (UB) Alberto G. Villafranca (IEEC) IEEC Data Compression Group | Presentation of activities | GTD, Barcelona, 25 January 2010

More Related