1 / 21

GGSPS overview and development status

GGSPS overview and development status. Andy Smith RAL. Contents. The RAL GERB Ground Segment Processing System System overview Context: the GERB ground segment What does the GGSPS do? How does it work? Development status Current functionality Recent problems and updates

tierney
Download Presentation

GGSPS overview and development status

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. GGSPS overview and development status Andy Smith RAL

  2. Contents • The RAL GERB Ground Segment Processing System • System overview • Context: the GERB ground segment • What does the GGSPS do? • How does it work? • Development status • Current functionality • Recent problems and updates • Major areas of work remaining

  3. GGSPS context

  4. Responsibilities • The RAL GERB Ground Segment Processing System • Receives and archives L0 data from EUMETSAT • Processes L0 data to L1.5 • Archives L1.5 data and sends the L1.5N to RMIB • Archives L2 fluxes produced by RMIB • Creates monthly averages from L2 fluxes • Provides a user interface to the data archive

  5. Logical model

  6. Sub-systems (1) • Data Reception • ingests new data from external sources (L0, SV, L2) • Archive / Maintain User Areas • catalogues and archives data products • Select Data for Processing • searches catalogue for data to process (L0, Eng, L2) • manages queue of processing jobs • Product Processing • processes L0 data to L1.5 and Eng Products in NRT • processes EPs to daily report and monthly trend • generates eng. plots • and processes L2 fluxes to monthly averages

  7. Sub-systems (2) • Process User Requests • web interface: catalogue queries and product download • Operator Console • controls and monitors software operations • System Error Logger • consistent error reporting across all components • Data Re-Ordering • detects missing data, places orders with UMARF • Product Re-Processing • bulk reprocessing of L0 data to L1.5

  8. NRT L1.5 processing

  9. NRT L1.5 processing • Data Reception (NRT GL0 and SV) • establishes & maintains network connection to EUM • receives raw packets, writes GL0 files • new GL0 file -> message to Archive • looks for new SEVIRI headers sent by RMIB • new SV -> message to Archive • messages: UNIX Inter-Process Communication library • Archive “listens” for messages • reads new products, writes catalogue entries • stores products in online archive • new L0/SV product -> message to Select Data

  10. NRT L1.5 processing • Catalogue: INGRES relational database • user queries via web: selection of products • system: tracking location and processing of data • Select Data for Processing (L0): • searches catalogue for groups of unprocessed L0 products • matches to SV header • maintains a queue of processing jobs for PPL15 • new job -> message to PPL15

  11. NRT L1.5 processing • Product Processing (L1.5) • processes L0 products to EP, L15N, L15A • more details? Ask Martin! • new products -> message to Archive • job finished -> message to SDPL0 • Archive • catalogues and archives new L1.5 products • L1.5 NANRG product copied to RMIB (via scp) • RMIB process L1.5 NANRG to L2 flux • send L2 flux file to GGSPS • Data Reception (L2) detects arrival of L2 flux • message to Archive…

  12. Data access http://ggsps.rl.ac.uk/

  13. Current state • In place and working well: • ingest of L0, SV header and L2 flux products • archiving of all current products • selection of L0/SV products for processing • processing of L0 products to L1.5 – see Martin’s talk • “Could do better”: • eng. processing: several (minor) problems • INGRES database (see later) • user interface / data access: handling of download jobs

  14. Recent changes • Software updates since GIST 20: • Data Reception: introduced “keep-alive” to maintain network connection (see next slide) • Data Reception: improved commanding to minimise data loss when switching between IMPF servers. • Selection of data based on scan count: better handling of missed data, repeatability when re-processing. • From mid-January 2004: real SEVIRI header files sent from RMIB (every 15 mins rather than daily). • CSOL: interim solution for transfer of corrected SOL in NRT is under way. • Plus several other changes to improve robustness etc.

  15. Known problems • Problems reported at GIST 20: • If network connection to IMPF drops out, GGSPS does not trigger switch to back-up line: now fixed – last line outage resulted in loss of only 2 GERB packets while the back-up line started up. • INGRES hangs, preventing processing: patch to INGRES tested successfully, install on operational system in April(?).

  16. Known problems • Hardware and other outages: • BT exchange problem in Nov/Dec. led to “patchy” line performance, alarms at EUMETSAT. Line re-routed by BT. • Network time-outs between the GGSPS processing machine and disk archive. Led to data loss when software detected errors writing files. Fix applied at system level.

  17. Data loss: recent Catalogued L0 packets vs expected (at 99.72 / minute)

  18. Work remaining • Major areas of development work remaining: • Data Re-Ordering • Product Re-Processing • L2 monthly averages • Product validation and data release • Data access via BADC • Hardware, systems etc • INGRES hangs: apply the latest patch • INGRES replication: • use 2nd machine to separate product download from • NRT processing

  19. Future work (1) • NERC have agreed to fund Item 1 of our bid: • Geolocation, Stray light • All L1.5 updates -> mid-June? • Validation: June onwards? • Re-ordering & re-processing: ~0.5 year -> Sept/Oct? • Plus other NCR fixes as priority demands

  20. Future work (2) • Item 2 bid to be reviewed July 2004 • Requests funding for monthly averages of L2 fluxes: • archiving of the RMIB L2 “BARG” product • addition of clear sky fluxes to the L2 means • averaging studies (Imperial) • Plus non-L2 work: • non-urgent (Eng) NCRs • acceptance activity

  21. Future work (3) • MSG-2 launch in March 2005? • GGSPS-2: set-up should start mid-2004 • purchase, install and configure machines • organise network lines • install GGSPS software and GERB-1 tables • System/Interface Tests & commissioning • Set-up period overlaps Level 2 work

More Related