SeaDataNet

1. SeaDataNet A Pan-European Infrastructure for Ocean and Marine Data Management by Dick M.A. Schaap (MARIS) Technical coordinator NATO ARW September 2007

2. Background • Data availability of vital importance for marine research • European data set is fragmented • Not always quality controlled • Not always easily accessible • In 35 countries bordering European seas • more than 1.000 scientific laboratories from government organizations and private industry collect data • use various sensors on board of research vessels, submarines, fixed and drifting platforms, airplanes and satellites • measure physical, geophysical, geological, biological and chemical parameters, biological species, etc. NATO ARW September 2007

3. What is SeaDataNet? • EU Sixth Framework Programme • Integrated Infrastructure Initiative (I3) • 1 April 2006 to 31 March 2011 • Objectives • develop an efficient distributed pan-European marine data management Infrastructure for managing large and diverse data sets • network existing professional data centres of 35 countries, active in data collection • provide integrated databases of standardised quality on-line NATO ARW September 2007

4. SeaDataNet partners • 49 partners including: • Major oceanographic institutes of 35 countries • National Oceanographic Data Centres (NODCs) • Divisions of major national marine research institutes • 3 satellite data centres • 3 scientific modelling centres • 2 SMEs experts in software development • 3 International organisations (IOC, ICES, JRC) NATO ARW September 2007

5. Building on experience • Leaders of Data Management Concerted Actions • MEDAR/MEDATLAS II - Mediterranean Data Archaeology and Rescue, aiming to make available a comprehensive data product of temperature, salinity and bio-chemical data in the Mediterranean and Black Sea through a wide international cooperation • Sea-Search – The gateway to Oceanographic and Marine Data & Information in Europe, to provide an effective navigation tool to oceanographic and marine data and information sources for European marine waters and to centres in Europe with expertise and skills in oceanographic and marine data & information management. • Data management of scientific projects: • MTPII-MATER, CANIGO, OMEX, MERSEA, etc. • Bringing in new partners – satellite data centres NATO ARW September 2007

6. SeaDataNet components NATO ARW September 2007

7. Infrastructure, versions & planning • Version 0 Continuation, maintenance and further population of existing Sea-Search systems with minor modifications • Version 1 All data centres involved in upgraded meta-databases (ISO compliant XML metadata-profiles and web services) Technical Task Team partners involved in transparent data access, security, and monitoring of use = trans-national access via downloading of datasets in standard formats (ODV ASCII; NetCDF; MedAtlas) Launch by March 2008 • Version 2 All data centres involved in trans-national access and upgrading of data access to include spatial data infrastructure features (adopting OGC; ISO and INSPIRE standards) and using XML data profiles Operational by 2010 NATO ARW September 2007

8. SeaDataNet Data Policy • Metadata • free and open access, no registration required • each data centre should provide the meta-data in standardized format to populate the discovery services (metadata directories) • Data and products services • in general free and open access, but mandatory registration • a “SeaDataNet role” (partner, academic, commercial etc.) is attributed to each user • Each NODC attributes the roles to the users of its of country • Out of the partnership, the roles are assigned by SeaDataNet user-desk • each data centre node delivers data according to the user’s role and its local regulation • each data centre should provide freely the data sets necessary to develop the common products NATO ARW September 2007

9. Discovery services • Marine environmental data sets (EDMED) • Marine environmental research projects (EDMERP) • Observing systems (EDIOS) • Research Cruise Summary Reports (CSR) • Common Data Index (CDI) • Directory of marine organisations (EDMO) NATO ARW September 2007

10. Cruise Summary Reports • Conceived by IOC • ICES acted as focal point • Currently over than 37000 cruises • Documenting data collected on scientific research cruises operated by European vessels • Cruise track charts included • Online entry form • Online searchable database NATO ARW September 2007

11. European Directory of the Ocean-observing System (EDIOS) • Initiative of the European Global Ocean Observing System (EuroGOOS) • Regular repeated measurements • On-line searchable directory of observing, measuring, and monitoring systems • Over 10,000 observation entries • Links to on-line real-time and archive data • Tool to aid design of observing systems NATO ARW September 2007

12. European Directory of Marine Organisations (EDMO) • Over 1.000 organisations: addresses, profiles and relations in other directories • Online maintenance per country by CMS • On-line user interface with combined alphanumeric – geographical searching • Export in .csv files • Also available as Web service (SOAP based) NATO ARW September 2007

13. Common Data Index • Index to individual data sets • Now >250000 entries from 20 partners • Mid Oct 2007 entries from 35 countries • ISO19115 metadata standard has been adopted • On-line searchable directory • Direct access to on-line data sets, on-line order facility or e-mail request NATO ARW September 2007

14. Technical developments • Streamlining of metadata formats for interrelationships between the directories • Establishment and governance of common vocabularies • ISO 19115 content model as basis for the XML formats and exchange schemas (XSD) with SeaDataNet XML approach • Innovating maintenance modalities: • XML exchange using local software component (MIKADO) or own application • XML validation services • Online Content Management System (CMS) • Introducing Web services • Vocabularies and Directory of Marine Organisations (EDMO) • Coming: other metadata directories NATO ARW September 2007

15. Interrelationships between the directories, using vocabularies CSR EDMED XML … … … EDMERP Platf Param Country EDIOS CDI EDMO NATO ARW September 2007

16. Common Vocabularies • Controlled vocabularies containing keys, terms, abbreviations and essential (but often missing) definitions • Lists for platforms, countries, sea areas, etc… • International content governance by SeaVox list server • Web service • On-line user client • Export in .csv files NATO ARW September 2007

17. SeaDataNet standards • Interoperability – key to distributed data management system success • Common understanding of terms used via common vocabularies • Standard data exchange formats (ODV ASCII, NetCDF, MedAtlas) • Common quality control protocols and flag scale • Standard Web services • Schemes for handling duplicate data • Ocean Data View software (ODV) adapted for SeaDataNet NATO ARW September 2007

18. SeaDataNet : Delivery Services V1 Primary function • Users will be able to download selected data sets CDI User Interface • Searching and browsing of metadata of data sets NATO ARW September 2007

19. SeaDataNet : Delivery Services V1 Shopping basket • Preparing a user request of multiple data sets Central User Register • Validation of users and their roles Business matrix • Combining user role and data access restriction(s) to decide on modality for processing each data request NATO ARW September 2007

20. Central User Register • Single Sign On system required for access to distributed system • User’s authentication information based on personal login / password • User must register in order to get login • Web form to provide necessary information • Online user agreement “SeaDataNet General Licence” • Registration processed per country via online CMS • After processing, login/password sent by e-mail (e-mail check) • User gets ‘Role’ • SeaDataNet Data Policy • Data access based upon combination of User role - Access restriction(s) as indicated in CDI record and business logic NATO ARW September 2007

21. SeaDataNet : Delivery Services V1 Request Status Manager • Processing of all requests and data deliveries • Enables users to follow progress • Enables DC’s to oversee transactions Download manager • Local component for communication with portal and preparing data deliveries NATO ARW September 2007

22. Status of request User registration Data download General request Metadata request Data request IFREMER Database Entry point for access hits services for metadata Data downloading services Visualisation services (WMS) for V2 SeaDataNet User’s portal schema V1 Project info metadata data Registr. Standards Cross search My transact. Shopping basket CDI Requestsstatus manager Software EDIOS EDMED AAA Vocabularies CSR EDMERP EDMO Organisation +data source id User Register CSR EDIOS CDI EDMO EDMED BODC Database + EDMERP at BSH at BODC at MARIS BSH Database Metadata & Data catalogues ... Download managers in data centres NATO ARW September 2007

23. Data products • SeaDataNet will provided added-value products to be created by regional centres • Mediterranean Sea • Black Sea • Baltic Sea • Arctic and North Sea • Atlantic and global ocean • First products => gridded climatologies • based on already available data-sets (mostly hydrographic data) • updated during the project life • To make products comparable, all regional data centers will use a common tool for spatial mapping • For this a new version of DIVA (Data Inverse Variational Analysis) was developed (University of Liege) NATO ARW September 2007

24. SeaDataNet wider involvement • SeaDataNet partners are involved in many other European and international projects, contributing and safeguarding good data management. Examples: • Services for the European marine geological institutes • EDIOS Services for EuroGOOS • Partners in the MERSEA project (GMES) for MIM • Partners in the HUMBOLDT project (GMES) for developing a framework for a European Spatial Data Infrastructure • Partners in SIMORC for oil & gas industry metocean data • Partners in ECOOP for data provision and management • Component of the GMES Marine Core Services NATO ARW September 2007

25. Marine geology data Catalogue of > 300.000 seafloor samples and >2.5 million line kilometres of marine seismic and sonar survey from the ocean basins and continental shelves, held at European institutions www.eu-seased.net NATO ARW September 2007

26. Industry metocean datasets - SIMORC • System of Industry Metocean data for the Offshore and Research Communities • Metadatabase and Database of datasets from major oil & gas companies At present ca 870 data sets (Shell, BP, Total), more underway www.simorc.org NATO ARW September 2007

27. SeaDataNet - BlackSeaScene • BSS is an Integrating Activity of EU-Research Infrastructures programme and runs from 1 Dec 2005 – 30 Nov 2008 • BSS has 32 partners with 25 from Black Sea region and is coordinated by MARIS • BSS is a Black Sea Scientific Network • analysing the present quality of Black Sea datasets and applied DQC procedures • developing a Black Sea regional portal for metadata and data access • BSS is adopting the SeaDataNet standards and building upon the SeaDataNet basic services • BSS engages 25 Black Sea institutes, strengthening and expanding the compilation of metadata and access to data for the Black Sea region NATO ARW September 2007

28. SeaDataNet capacity building • The training program has two main objectives: • to ensure that the data and metadata to be integrated in the SeaDataNet system will be formatted, checked for quality and disseminated according to the common protocols developed • to transfer expertise and to train IT experts of the data centers in the basics, installation and operation of the SeaDataNet technical components NATO ARW September 2007

29. International Conference IMDIS 2008 • Second international conference “International Marine Data Information Systems” (IMDIS) • March 31 - April 2, 2008, Athens, Greece • Joint organisers: SeaDataNet, Hellenic Centre for Marine Research, IOC/IODE • Four sessions: • Marine environmental data bases: infrastructures and data access systems • Data Services in ocean science: standards and interoperability • Visualisation tools • Services for Users and Education NATO ARW September 2007

30. www.seadatanet.org NATO ARW September 2007