180 likes | 200 Views
PEPS. French A ccess to Copernicus S pace Data. PEPS – 04 - 12 - 2018. What’s PEPS today. 3 main functions : " Mirror site " of ESA hubs providing access to Sentinel 1, 2 and 3 products Automatic download to multiple ESA hubs in parallel (delay <48h )
E N D
PEPS French Access to Copernicus Space Data
PEPS – 04 - 12 - 2018 What’s PEPS today • 3 main functions: • "Mirror site" of ESA hubs providing access to Sentinel 1, 2 and 3 products • Automatic download to multiple ESA hubs in parallel (delay <48h) • Full archive – except S3: only up to 15th April 2019 • Processing workshop allowing "first level" treatments, results visualization and products downloading • New version releasedin June 2018 and progressivelyenriched • Incubation of relatively "mature" processing chains for "scale" validation • 5 companies hosted on the CNES computer center + 2 science laboratories Direction du Numérique, de l'exploitation et des Opérations
PEPS – 04 - 12 - 2018 Active treatments in PEPS • Several standard tools: streaming visualization, NDVI, color compositions… • Dedicatedtreatmentsrequired by someusers • Flagshiptreatments(based on CNES tools) • Orthorectification of S1 products vs the S2 grid • S2 groundreflectanceusing MAJA • Unitary but also multidate processing(full Ortho and full Maja…) • Use of WPS protocol • In progress: • S2 correlation • S1 interferometry • Soil moisture product mixing S1 and S2 data Direction du Numérique, de l'exploitation et des Opérations
Near Data Processing PEPS provide online processingfacilities Applications are provided by CNES or by users This featuresimplify the discovery and the use of the information in the spatial data Direction du Numérique, de l'exploitation et des Opérations
User uptake and Incubation • After agreement, PEPS host companieswhowant to develop and improve a spatial service • The hosting infrastructure isbased on WPS, DOCKER and PROACTIVE metascheduler • The data are stored close to the HPC/HPDA 9.5 PB Direction du Numérique, de l'exploitation et des Opérations
PEPS – 04 - 12 - 2018 PEPS linkwithThematic Data Centers / hubs • MUSCATEenvironmentuses PEPS catalog and accessdirectly to the products • For Theia (Land Surface Thematic Data and Services Center) • systematic MAJA processing over France, northAfrica… • + snowdetection, monthly L3A synthesis, land cover… • http://www.cesbio.ups-tlse.fr/multitemp/?p=14600 • For Form@ter(Solid EarthThematicData and Services Center) • systematicinterferometryprocessing over large areas • SPOT archive • Provideaccess to SPOT data to completeSentinel2 time series(SWH processing) Direction du Numérique, de l'exploitation et des Opérations
PEPS – 04 - 12 - 2018 Data storage • Volume of stored data • ≈ 9.5 PB - around 4 PB in line • 14,175,000 products • Availablestoragecapacity: 14 PB • Logic for storage: • Oldest data are transferred to tape storagedevices (longer delay for data retrieving) • Sentinel-3 products are systematicallytransferredafter a few days • Easy upgrade for storagecapacity if needed • Good margin in the capacitiesdedicated for usersprocessing DIrection du Numérique, de l'exploitation et des Opérations
PEPS Somestatistics • septembre 2018
PEPS statistics– 04/12/2018 Stable increase of registeredusers Users registration evolution 4713 registeredusers (french users: 59.8%) Users countries repartition DIrection du Numérique, de l'exploitation et des Opérations
PEPS statistics– 04/12/2018 Increase of the distributed data volume (but veryirregular) - 2019 vs 2017: X2 DIrection du Numérique, de l'exploitation et des Opérations
PEPS statistics– 04/12/2018 Whichis the ‘top’ Sentinel? Number of distributedproducts for eachSentinel • 60% S1 – 40% S2 - <1% S3 (hugedownload of S2 productsby Google in april/may 2018!!) DIrection du Numérique, de l'exploitation et des Opérations
PEPS statistics– 04/12/2018 Archive products / ‘fresh’ products ≈1/5 Google archive in april/may DIrection du Numérique, de l'exploitation et des Opérations
PEPS statistics– 04/12/2018 Stronginterest for archive data Number of distributedproducts per acquisition date DIrection du Numérique, de l'exploitation et des Opérations
PEPS statistics– 04/12/2018 How to access the data ? Volume of downloaded data script vs interactive Number of users script vs interactive DIrection du Numérique, de l'exploitation et des Opérations
PEPS statistics– 04/12/2018 Users help desk exppeps@cnes.fr Type of request Number of usersrequests * Information Help request Request for new needs Averagedelay - 48H *Requestnumberincrease due to deactivation of the downloading of storedproducts (not on line). Temporarynecessity for manualoperator activation. DIrection du Numérique, de l'exploitation et des Opérations
PEPS statistics– 04/12/2018 Number of treatmentsperformed • MAJA • S1 ortho. DIrection du Numérique, de l'exploitation et des Opérations
PEPS – 04 - 12 - 2018 PEPS in 2019 • Exploitation runssmoothly • Regular upgrades • Optimisation of data downloadfunctions • Users data access interface improvements • Verylowinterest for Sentinel-3 data • Someusersask for S2_L1B products • Downloading data from ESA hubs • ESA distribution relatively mature • Strong and efficient ESA supportboth for information requests and for back log retrieving in case of maintenance/crash or data gaps filling • Whatwillbe PEPS tomorrow? Direction du Numérique, de l'exploitation et des Opérations