1 / 31

SeaDataNet Pan-European Infrastructure for Ocean and Marine Data Management

SeaDataNet mid-term review – Bologna – September 2008, 19. SeaDataNet Pan-European Infrastructure for Ocean and Marine Data Management. Gilbert Maudire and the SeaDataNet Consortium. Data is of vital importance for marine research and support to public decision.

carson
Download Presentation

SeaDataNet Pan-European Infrastructure for Ocean and Marine Data Management

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. SeaDataNet mid-term review – Bologna – September 2008, 19 SeaDataNetPan-European Infrastructure for Ocean and Marine Data Management Gilbert Maudire and the SeaDataNet Consortium

  2. SeaDataNet mid-term review – Bologna – September 2008, 19 Data is of vital importance for marine researchand support to public decision • Marine observation and data sets status • observation conducted by more than 1100 active scientific laboratories in the countries of partners, using various sensors and platforms • not always quality controlled using comparable procedures • not always easily accessible • Necessity of large background work in order to : • secure them in long term archives • document available observations and data sets (metadata) • improve coherence of data sets (references, quality, redundancy) • make data sets accessible easily (formats, on line distribution)

  3. SeaDataNet mid-term review – Bologna – September 2008, 19 What is SeaDataNet? • EU Sixth Framework Programme • Integrated Research Infrastructure Initiative (I3) • 1 April 2006 to 31 March 2011 (5 years) • Objectives • To network existing oceanographic data centres already nationally funded • To develop an efficient distributed pan-European marine data management Infrastructure (a “unique Virtual Data Centre”) • To provide on-line access to distributed databases of standardised quality by using adapted communication & information technology • This project integrates and develops several EU past initiatives and international cooperation, in particular Medar/Medatlas, Edmed, Edios, Sea-Search etc.

  4. SeaDataNet mid-term review – Bologna – September 2008, 19 49 Partners • National Oceanographic Data Centres • divisions of major national marine research institutes in 35 countries • experienced as leaders of data Management of many scientific projects and of concerted actions (e.g. MTPII-MATER, CANIGO, OMEX, EDIOS, MERSEA, MEDAR/MEDATLAS, SEA-SEARCH) • 3 satellite data centres • 3 scientific modelling centres • 2 SMEs experts in software development • 3 International organisations with strong regional interest/cooperation

  5. SeaDataNet mid-term review – Bologna – September 2008, 19 SeaDataNet managed data • Primary focus on water column parameters • Physical • Chemical • Bio-chemical (non species dependant) • Projects will be proposed for other domains by involved communities in strong relationships with SeaDataNet Several SeaDataNet partners are already involved in those domains • Geophysical and geological data • Biology (species dependant)

  6. SeaDataNet mid-term review – Bologna – September 2008, 19 SeaDataNet services • Online services : web portal and web services • Discovery services • Metadata directories • Common vocabularies : parameters, units, … • Delivery services • Security services : Authentication, Authorization & Administration (AAA) • Data download and access including reformatting in standardized formats • Viewing services • Web map services, data visualization. • Offline services : « back office » services • Safeguarding within the national repositories (NODCs) • Quality Control and desktop viewing services • ODV analysis software • Product generation adapted to each regions • Monitoring and statistics : usage & system performance

  7. SeaDataNet mid-term review – Bologna – September 2008, 19 Infrastructure versions & planning • Version 0 – 2006-2007 Continuation and maintenance of existing Sea-Search system : the data access needs several different requests to each data centres and the data sets are delivered in different formats • Operated at the start of the project • Version 1 – 2008-2010 Setup of the integrated online data services to users : networking of 10 “interoperable” data centres of the Technical Task Team unique request to the interconnected data centres and the data sets are delivered with a unique format • Presently under test and progressive integration of 10 data centres during 2008 • Version 2 – 2010-2011 Improved version of the system (ease of use, data visualisation, data access)and by step, all data centres connected for providing online trans-national access to the data, metadata and products • Operational in 2010

  8. SeaDataNet mid-term review – Bologna – September 2008, 19 Discovery services All catalogues(European metadata directories) • are compliant with ISO19115 standard family for spatial metadata • make use of common vocabularies and references • are accessible and linked through web services • ease data access using various selection criterions adapted to different users (scientists, observatory and fleet managers, …)

  9. SeaDataNet mid-term review – Bologna – September 2008, 19 • Common data index : gate to data access • online searchable directory where?, when?, what?, … • tool for managing user’s“Shopping Basket”, that is : • To prepare a list of distributed datasetsmatching user’s request • To ask concerned data centres to serve the user, • To display the status of user’s request • To route the requested data from the distributed data centres to the user

  10. SeaDataNet mid-term review – Bologna – September 2008, 19 SeaDataNet data policy and data licence inspired by INSPIRE Directive to facilitate the data flow takes into account the local rules and the SeaDataNet users needs • Metadata • free and open access, no registration required • each data centre should provide the meta-data in standardized format to populate the catalogue services • Data and products • visualisation freely available • data download, in general, free for academic purposes with mandatory registration • a “SeaDataNet role” (partner, academic, commercial etc.) is attributed to individual user using the Authentication, Authorization and Administration (AAA) Service • Each NODC attributes the roles to the users of its of country • Out of the partnership, the roles are assigned by SeaDataNet user-desk • a common data licence has been adopted and users must agree on when registering • each data centre node delivers data according to the user’s role and its local regulation • each data centre should provide the data sets necessary to develop the common products

  11. SeaDataNet mid-term review – Bologna – September 2008, 19 QC procedures and tools • SeaDataNet protocol : • Coherent automatic and visual checks are performed on the data by each NODC • A unique quality flag scale has been adopted • The automatic checks and the visualization could be performed using either by Ocean Data View (ODV) or existing partner software (if the same protocol and checks are applied) Example:Using derivatives for spike & discontinuity detection in ODV

  12. SeaDataNet mid-term review – Bologna – September 2008, 19 Data products • SeaDataNet develops added-value products at regional scales: • Mediterranean Sea • Black Sea • Baltic Sea • Arctic Seas • Atlantic and global ocean The preparation of these products allows thecheckingof the compatibility of the QCs performedon the data in each NODC. • First products => gridded climatologies and trends • The comparability of the different regional products is assessed by using a new version of DIVA (Data Inverse Variational Analysis) developed (University of Liege)

  13. SeaDataNet mid-term review – Bologna – September 2008, 19 • Training program main objectives • to ensure that management procedures are coherent within the consortium • to transfer expertise and to train IT experts of the SeaDataNet data centers in the installation and operation of the SeaDataNet technical components Capacity Building is a main issue

  14. SeaDataNet mid-term review – Bologna – September 2008, 19 www.seadatanet.org Please contact : sdn-userdesk@seadatanet.org

  15. SeaDataNet mid-term review – Bologna – September 2008, 19 SeaDataNetProject management

  16. SeaDataNet mid-term review – Bologna – September 2008, 19 Role of partners (1/4) • Coordination • Ifremer : project coordinator • Maris : technical coordinator animator of the Technical Task Group with the help of : • NERC/ BODC : standard development • AWI : data quality control and desktop viewing services • University of Liege : scientific coordinator in charge of the regional products development with the help of : • JRC : Product harmonization • INGV : coordinator of Mediterranean products • METU : coordinator of Black Sea products • NERI : coordinator of Baltic products • IMR & VLIZ : coordinators of Artic & North Seas products • Ifremer : coordinators of Atlantic & Global Ocean products

  17. SeaDataNet mid-term review – Bologna – September 2008, 19 Role of partners (2/4) • Coordination (cont.) • HCMR : Coordinator of the networking activities i.e. metadata and data management, system monitoring with the help for coordinating the distributed network of • Maris : management of Common Data Index (CDI) and of Directories of Marine Projects (EDMERP) and Marine Organizations (EDMO) • NERC/ BODC : management of Directories of Marine Environmental Datasets (EDMED) and of Ocean Observing System (EDIOS) • BSH : management of Cruise summary reports • Ifremer : Management of SeaDataNet users

  18. SeaDataNet mid-term review – Bologna – September 2008, 19 Role of partners (3/4) • Technical task team (11 partners : Western Europe + Russia) • Technical specifications, • Check of the defined standards • Integration and system setup, test and validation of technical components • Running of System V1 • Capacity building and training • UNESCO/ IOC Project office with the help of Technical Task Team • Information and communication • ENEA

  19. SeaDataNet mid-term review – Bologna – September 2008, 19 Role of partners (4/4) • All partners which are mostly NODC’s (except international organizations) have to • generate metadata : organizations, projects, cruises, observing systems, data sets • manage data from their country : quality, format, … according to SeaDataNet agreed standards • provide access to their data repository, automatically using SeaDataNet system or manually as a first stage • manage users of their respective countries by providing information and help

  20. SeaDataNet mid-term review – Bologna – September 2008, 19 External relationships / Collaborations • Advisory board • MEDS Canada(represents IODE/WMO JCOMM, GTSPP and Argo international programs) • US – NODC (represents North America initiatives like DMAC) • EuroGOOS(represents Operational Oceanography community) • Standard summit (last January on behalf of COI) • North America, Europe, Japan • “Memorandum of understanding” with other consortiums • Unidata, Mersea-MyOcean (GMES), EuroBis-Marbef (FP6), GENESI (FP7) • International conference on Marine Data and Information Systems (IMDIS) • April 2008, more than 200 participants from about 50 countries.

  21. SeaDataNet mid-term review – Bologna – September 2008, 19 Project management tools • SeaDataNet web sitewww.seadatanet.org • Distributed management, based on a Content Management system (EZPublish) • General information : Description of Work, Meetings, Conferences, Education (several languages) • Project services (discovery, data access, …) • Finalised documents (applicable version of standards) and software tools • Mailing lists • Based on Sympa software • One list per group (technical, …) owned by the chair person • + user desk (SeaDataNet users point of contact) • Document management • Based on BSCW software (Basic Support for Cooperative Work - Germany) • Cooperative work on documents (non finalised versions) and software • Software management (versionning tool) • Based on GForge

  22. SeaDataNet mid-term review – Bologna – September 2008, 19 System monitoring  focused on system “availability” • Availability of SeaDataNet user services • availability of web pages and web services general information, metadata browsing (EDMED, CSR, CDI), data selection users’ interfaces, web services • availability of web data access servicesdata centre download services • Service “available” if monitoring system get an answer • Search of keywords within web pages, • Web services can be reached : WSDL returned • Monitoring • done by IFREMER (coordinator) and HCMRin order to avoid biases due to network troubles • Using NAGIOS Open Source software • Polling of services (web pages, web services)

  23. SeaDataNet mid-term review – Bologna – September 2008, 19 Polling of system components

  24. SeaDataNet mid-term review – Bologna – September 2008, 19 Monitoring board • Example of one-month report for Ifremer http access Electrical failure during 6 hours

  25. SeaDataNet mid-term review – Bologna – September 2008, 19 SeaDataNetAdministrative and financial aspects

  26. SeaDataNet mid-term review – Bologna – September 2008, 19 Known issues • IOC Project Office partner • In charge of training courses and capacity building • Report at 0 up to now even if work done (3 training courses organised yet)no agreement on US $ / € change rate between UNESCO and EC) • IOC is an « Additional Cost partner »not well adapted to training course organization (additional personnel) • Transnational activities report • Partners with very small web site and/or very little audience • Audience measurment : KiloPageViews unit difficult to report accurately • 80% of partners report less than expected for TA

  27. SeaDataNet mid-term review – Bologna – September 2008, 19 Transnational Activities amendment • Change reporting unit from KiloPageViews to KiloHits • To take into account all hits on online services :web pages (for any usages), web services, ftp servers, … • But, cost / unit must decrease (Project Officer comment) • However, most audiences will remain far less than expected • Cost are not linearly proportional to the audience« Entry costs » to set up a web site funds transfer to JRA (web site maintenance) • Who? How much? • All must remain in the same budget envelope

  28. SeaDataNet mid-term review – Bologna – September 2008, 19 SeaDataNetPlanning

  29. SeaDataNet mid-term review – Bologna – September 2008, 19 Infrastructure versions & planning • Version 0 – 2006-2007 Continuation and maintenance of existing Sea-Search system : the data access needs several different requests to each data centres and the data sets are delivered in different formats • Operated at the start of the project • Version 1 – 2008-2010 Setup of the integrated online data services to users : networking of 10 “interoperable” data centres of the Technical Task Team unique request to the interconnected data centres and the data sets are delivered with a unique format • Presently under test and progressive integration of 10 data centres during 2008 • Version 2 – 2010-2011 Improved version of the system (ease of use, data visualisation, data access)and by step, all data centres connected for providing online trans-national access to the data, metadata and products • Operational by 2010

  30. SeaDataNet mid-term review – Bologna – September 2008, 19 Project planning and budget • High level of effort during the first three years(80% of the budget) • Common standard adoption, set up and training, • Development / Adaptation of common tools • System specifications, development and setup (V1 and then V2) • Development of first release of regional products and initial harmonization • Lower level of effort during the last two years • Continuous data management • Exploitation of the system, system maintenance and upgrade • Upgrade of regional products

  31. SeaDataNet mid-term review – Bologna – September 2008, 19 More information about budget • Funds breakdown / year / partner after second report • Funds breakdown : https://www.ifremer.fr/bscw/bscw.cgi/0/110042 • Transnational access audience • Evolution for TA- Seadatanet.xls

More Related