Data and Data Requirements

1. Data and Data Requirements Wouter Los University of Amsterdam

2. Environmental Science • Earth as a single complex and coupled system W. Los - ENVRI @ EUDAT

3. ESFRI Environmental Research Infrastructures W. Losi- ENVRI @ EUDAT

4. -ENVRI @ EUDAT

6. Gas (CO2 etc) fluxes ∂ (concentration) Radar interference data Areal and satellite observation Species data, distributions, abundance, biomass, etc. Observations, sensor data, collection data, DNA, etc Plate tectonics Seismic data, satellite data, sensors, etc Marine sensors Currents, salinity, deposition, etc Pasquale Pagano - ENVRI @ EGI CF 2012

7. Goal Enable multidisciplinary scientists to accessand study data from multiple domains for “system level” research by providing solutions and guidelines for the RIs common needs Multiple data producers Multiple data consumers W. Los- ENVRI @ EUDAT

8. Approach • Provide software tools to W. Los - ENVRI @ EUDAT

9. Data infrastructure requirements • Integrated data discovery across various catalogues • (Near) real-time data handling • Federation over infrastructures/services • Persistent identifiers mechanism • Metadata definition and assignment • Attribution / crediting author/ownership • Quality control of data • Provenance and preservation • Archiving vs. regeneration of data and/or results (processed data) • Single sign-on, delegated authorisation • Running complex models • Data staging or moving computation to data W. Los- ENVRI @ EUDAT

10. First steps - priority areas W. Los - ENVRI @ EUDAT

11. 1: Integrated data discovery • Integrated data discovery across various centres / catalogues • The challenge of being able to easily discover data which are heterogeneous (in format, content, and metadata description) and which are stored at different places • ENVRI partners ESA, CNR, UvA and CSC are tackling this • Does EUDAT see a role to contribute?

12. Geospatial Data Services P1 P2 P.. OGC OpenSearch Linked Open Data gCube Data staging OGC WMS, WFS OGC WCS OGC WPS • Hadoop Cluster Data Access Data Pub. /Vis. Data Discovery Data Process HDFS • WPS Hadoop • GeoServer • WPS 52N Catalogue Services • THREDDS Geospatial Repositories by courtesy of P. Pagano

13. 2: (Near) Real Time Data Handling • (near) Real-time data handling • The challenge(s) of being able to handle real-time data • Challenges include: • collecting, storing and cataloguing data as it arrives in real-time from sensors • processing data into derived data products in real-time • analysing data in real-time It was suggested that EUDAT might take this up

14. 3: Federation • Federation over existing (national and international infrastructures / services • The challenge of bringing together existing infrastructure components / services as contributions to the construction of an Research Infrastructure • The challenge of bringing about interoperability (syntactic and semantic) between separately owned and operated facilities that each contribute to the Research Infrastructure • Is EUDAT also interested?

15. Data infrastructure requirements • Integrated data discovery across various catalogues • (Near) real-time data handling • Federation over infrastructures/services • Persistent identifiers mechanism • Metadata definition and assignment • Attribution / crediting author/ownership • Quality control of data • Provenance and preservation • Archiving vs. regeneration of data and/or results (processed data) • Single sign-on, delegated authorisation • Running complex models • Data staging or moving computation to data W. Los- ENVRI @ EUDAT

16. Need a Common Data Infrastructure • Managing the growing amount of data • Improving interoperability between infrastructures and across disciplines • Promoting collaboration and clarifying roles and responsibilities EUDAT contributions to the ENVRI consortium is welcome! W. Los- ENVRI @ EUDAT

17. Thank you http://envri.eu/ W. Los – ENVRI @ EUDAT