1 / 16

EUMETSAT Unified Meteorological Archive and Retrieval Facility and Services (U-MARF)

EUMETSAT Unified Meteorological Archive and Retrieval Facility and Services (U-MARF). Franck Cade (U-MARF Project Manager) (cade@eumetsat.de). Outline. An introduction to U-MARF: What is U-MARF ? Project drivers and objectives Development approach & schedule overview

armand
Download Presentation

EUMETSAT Unified Meteorological Archive and Retrieval Facility and Services (U-MARF)

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. EUMETSAT Unified Meteorological Archive and Retrieval Facility and Services (U-MARF) Franck Cade (U-MARF Project Manager)(cade@eumetsat.de)

  2. Outline • An introduction to U-MARF: What is U-MARF ? • Project drivers and objectives • Development approach & schedule overview • U-MARF technical overview • Performances issues • U-MARF standard services : How can users gain access to it ? • U-MARF standard services overview • Access to U-MARF data: Product search and ordering • U-MARF distribution formats • User documents • Conclusions

  3. 1. What is the U-MARF ? • An introduction to U-MARF: what is U-MARF ? • Project drivers and objectives • Development approach & schedule overview • U-MARF technical overview

  4. U-MARF Project drivers and objectives • EUMETSAT decision in 1997 to develop a multi-mission, archiving and retrieval facility, including on-line web user access capability, supporting all EUMETSAT programmes: the Unified Meteorological Archive and Retrieval Facility “U-MARF” • Offer integrated user access to the whole of EUMETSAT data products and services: • GEO data : • Meteosat Transition Programme (MTP) (including historical archive from METEOSAT 1 to 6), • Meteosat Second Generation (MSG-1 to MSG-3) • LEO data : EUMETSAT Polar System (EPS) (METOP satellites), • Plus multi-mission products ( Level 2 and above) from Satellite Application Facilities (SAF) • Exploit and take advantage of new technologies (e.g. archiving, on-line web access) • Built in flexibility and scalability to accommodate future new data and archive growth potential • Objective is to provide access to a wide range of external users (from novice users, to expert meteorological and climate communities)

  5. U-MARF Development approach • U-MARF procured from industry : contract allocated to Consortium including Matra Systemes & Information (F) as Prime, with ACS (I) approved by EUMETSAT Council in November 1998 • Incremental development performed in two major versions: • U-MARF V1: • phased on MSG development schedule • providing common multi-mission functions and services for MSG and MTP data and products • MTP data to be migrated from current MARF to U-MARF in 3 years minimum • U-MARF V2 : • phased on EPS development schedule • extending services to EPS data sets and implementing connectivity with the SAF

  6. U-MARF Schedule Overview • U-MARF Development Milestones:: • Project start Early January 1998 • Invitation To Tender July 1998 • V1 Industrial development kick-off Mid-December 1998 • End of V1 development Planned December 2001 • End of integration with GS Planned May 2002 • V2 Industrial development kick-off Planned January 2002 • End of V2 development Planned March 2004 • U-MARF services milestones: • Ingestion, archiving and cataloguing MSG data: operational for MSG-1 launch • Standard user services : fully operational after end of MSG-1 commissioning • End of migration of MTP data: MSG-1 launch + 3 years (TBC)

  7. U-MARF Technical Overview (1/3) Monitoring & Control MSG Front-End MTP Front-End MSG Ground Segment EPS Ground Segment MTP MARF EPS Front-End SAF SAF Local User Services SAF Local Archive &Catalogue U-MARF Facility External Users UMARF Catalogue Management UMARF Storage Management Product Formatting & Delivery UMARF User services External Users External Users • User Services • Product formatting & delivery • Monitoring & Control • Ground Segment Front-End • Storage management • Catalogue management

  8. U-MARF Technical Overview (2/3) • U-MARF Functions: • Ingestion - Archiving & cataloguing chain: • Ingestion from Ground Segments (processing facilities), and Remote Centres • Archiving & Cataloguing • Retrieval - Ordering & Delivery chain : • User Services • Product Formatting and Delivery • Monitoring & Control & Administration: • Monitoring & Control • User administration • Excluded functions: • Processing, [Re-processing]

  9. Performances Issues • Data ingestion: • For V1: capacity to ingest 3 MSG data stream in parallel : 180 Gbytes/day - 2.4 Mbytes/s • For V2 :capacity to ingest 2.2 EPS data stream in parallel: 420 Gbytes/day - 5.5 Mbytes/s • User data retrieval : • For V1: 100 Gbytes/day • For V2: 600 Gbytes/day then 1.2 Tbytes/day • Capacity requirement: • see next • On-line user connections: • For V1: capacity to support at least 10 simultaneous users for on-line data retrieval for V1 • For V2: capacity to support at least 30 simultaneous users for on-line data retrieval • Availability of data for user retrieval • Data accessible to users 2 hours after ingestion into the U-MARF • SAF catalogue records accessible to users 3 hours after ingestion

  10. U-MARF Archive ingestion and accumulation • MTP: ~ 15 Terabytes/year => total of ~ 45 Terabytes • MSG: ~25 Terabytes/year => total of ~300 Terabytes • EPS: total of ~100 Terabytes/year => total of ~ 1.5 Petabytes

  11. U-MARF Technical Overview (3/3)Summary • Characteristics & constraints summary: • Technical features: • Near-real time ingestion constraints from processing facilities • Throughput performances: ingestion + retrieval • Operational aspects: • Drastic availability & reliability constraints: 10 minutes downtime • Automated archive and automated production for retrieval including for media delivery to users (small robots/stackers providing 48 autonomy of operations) • Open points: • Inter-operability features with other EO providers I.e. CEOS,… • User feedback on user services definition and performances

  12. 2. How can users gain access to U-MARF? • U-MARF standard services presentation : How can users gain access to U-MARF data? • U-MARF standard services overview • Access to U-MARF data: Product search (Inventory & Browse) and ordering • U-MARF distribution formats • User documents

  13. U-MARF standard User Services • U-MARF services will be integrated with EUMETSAT user services and accessed from the EUMETSAT home page (www.eumetsat.de) • U-MARF Core Services : • Information and Guide: aiming at providing users with general information and documents (about U-MARF, products characteristics, file formats, …) • Product search (Inventory and Browse): allowing users to search for archived products (simple or advanced queries) and to obtain information (metadata , thumbnails, browse images) on the selected products • Ordering and Follow-up: allowing users to order selected products (further to Inventory search services) • U-MARF Supporting Auxiliary Services: • User Registration : allowing a user to log-in as known user or to request for a registration (user account and user profile) • User Services Administration: users account management, EUM data policy implementation, invoicing management... • Help Desk: comprehensive help desk services supported by automated functions

  14. Access to U-MARF data: product search & ordering • Inventory search • V1/V2 search features => GEO and common search features: • Temporal search criteria • Metadata search criteria :satellite, instrument, product type, processing level, version of algorithm, quality indicators • V2 additional LEO specific features: orbit, geographical search • Ordering features: • Elementary orders (on archived products) or Standing Orders (subscription on future selected products in a pre-defined time range) • Product subsetting capabilities: • spectral bands subsetting • geographical subsetting (ROI for GEO image products, orbit subset for LEO orbit based products) • specific subsetting (header and/or trailer extraction) • Delivery options : • Electronic (ftp) • Physical media delivery : media includes : CD-ROM, [DVD], DAT, Exabyte, DLT • High quality printouts

  15. U-MARF Distribution Formats • U-MARF V1 Distribution Format: • Native format (as generated by MTP or MSG Ground Segment ) • Band Sequential Format for SEVIRI Level 1.5 • Open MTP for: • Level 1.0 and Level 1.5 • Cloud Analysis, Cloud Motion Winds, Sea Surface Temperature, Upper Tropospheric Humidity, Climate Data Set • WMO / BUFR for MTP MET Products and for the following MSG MET Products : • Atmospheric Motion Vectors (AMV), Cloud Analysis (CLA) , Clear Sky Radiance (CSR), Tropospheric Humidity (TH), Climate Data Set (CDS), Total Ozone (TOZ) • JPEG for METEOSAT and SEVIRI level 1.5 • Other Distribution Formats (TBD): including formats requested by last STG OPS WG :McIdas, and other evolutions of the MTP MARF • Additional U-MARF V2 Distribution Formats: • Native format (as generated by EPS Ground Segment ) • WMO formats (BUFR/GBRIB) for all EPS products • HDF (Hierarchical Data Format) for MTP, MSG and EPS products • PNG (Portable Network Graphics) for MTP, MSG and EPS products

  16. Conclusions • U-MARF is part of a EUMETSAT effort to : • build an affordable and maintainable multimission infrastructure by exploiting commonalty between the different programmes • Enhance user access to EUMETSAT data products and services, as they become relevant to a wider range of user communities

More Related