IODE Ocean Data Portal: Enhancing Marine Data Exchange

1. Third Session of the Joint WMO-IOC Technical Commission for Oceanography and Marine Meteorology (JCOMM) IODE Ocean Data Portal Nikolay Mikhaylov, JCOMM/IODE ETDMP Chair Marrakech, Morocco, 4-11 November 2009

2. IODE Ocean Data Portal Project Objective The objective of the Ocean Data Portal is to facilitate and promote the exchange and dissemination marine data and services. Participants:ODP-WIGOS for JCOMM Steering Group, JCOMM/IODE ETDMP, IODE Project Office, IODE NODC/DNA’s Work packages: • Project coordination • ODP standards development • Portal implementation Project implementation period: • 2008 – 2009 (IODE-XX/JCOMM-3) – ODP (V.1) • 2009 – 2011 ODP (V.2)

3. IODE Ocean Data Portal Concept • ODP is a standards based infrastructure for integration of data and services from IODE and other IOC/JCOMM data centres • ODP provides on-line access and dissemination: • operational and delayed-mode data • data and services from the oceanographic and marine meteorological domains • from multiple data formats and local data systems (DBMS, data files, map services, electronic documents) • from multiple data and service providers in different geographic regions • Data/service providers need to accept and implement a set of interoperability arrangements • It will not required to change their local data structures and formats for distributed data integration • ODP will provide the interactions with other systems (WIS, SDN,..) • ODP is build on the existing components: ISO and OGC standards, DiGIR, OpenDAP, Russian system – ESIMO, Australian IMOS and other solutions

4. IODE Ocean Data Portal Stages The ODP development is a complex task andtherefore it was defined that two versions of the portal would be developed one afteranother: version 1- with initial capabilities operating on the basis of theE2EDM technology version 2- with full capabilities which should be fully based on ISO/OGC interoperability standards and tools During the reported period the ODP (version 1) has been developed (http://www.oceandataportal.org).

5. IODE ODP Architecture Service Provider Service Catalogue Interface Interfaces Interface Processing System Interaction Interfaces Interfaces Interfaces Interfaces Interface Interfaces Interfaces Interfaces Interfaces User service layer Administrating, Discovery, Viewing, Analysis, Download Integration layer End-User Data/Service registration, Metadata management, Data delivery, Monitoring, … External System Metadata and Data Internet-network Response Request Data Provider Data Catalogue Data System Interaction Layer of Distributed Data Net

6. ODP V.1 Operation Framework User access ODP Services Integration Server 1. Integration of Distributed Data Sets Data centers install the Data Provider software or use the light DP to serve as a data source for the distributed system according to the commitments of the centers. The IODE Project Office uses the Integration Server for the metadata management and other operations for the distributed data system support Each ODP data provider is responsible for provided data quality and data storage. There is an Integration Server function for “cashing” and storage of the data sets and this function can be implemented according to the agreements (driven by requirements).

7. ODP V.1 Operation Framework 2. Access to distributed data sets and end-user services Discovery servicedisseminates a data source catalogue with descriptions of resources. The data source catalogue can be accessed from external systems directly or after reformatting into other metadata structures.  Viewing serviceis based on Web-application and provides: (i) data search using a map, time period, resource and etc.; (ii) access to remote data sources and tabular-graphic/map visualization of data. Analysis servicehas been developed recently – the GIS applications for presentation of multidisciplinary data and products on the map and WMS/WFS services. Download service - the loading the data on user computer via search or a time schedule (in this case it is required to register a point for downloading, list of required data sets and sampling criteria). Transport data file formats: NetCDF, ASCII and XML.

8. ODP V.1 Discovery Service Resource catalogue Parameters Additional search options

9. ODP V.1 Search Service Search status

10. ODP V.1 Viewing Service Point data profile data grid data • ODP Web sitewww.oceandataportal.org • ODP applications - IODE Project Office (VLIZ)www.oceandataportal.net - RIHMI-WDC (mirror) http://data.meteo.ru/e2edm/ NODC, Russia

11. ODP V.1 GIS Analysis Service

12. IODE ODP V.1 Operation Framework • 3. Administrative management • The administrative section of the ODP provides authorization, authentication and audit against user roles and rights on particular data and tools. • Service is provided according to the IOC Data Exchange Policy. The following access rules were defined: • (i) metadata are freely accessible without any condition. • (iI) data and products require: • - registration • - acceptance of user license conditions: referencing of data sources • Statistics on data and metadata usage, requests and ranking of data products are based on the actions of authorized users only.

13. IODE ODP V.1 Operation Framework WIS GISC ODP Services Integration Server SDN Portal GISC catalogue / registry OAI Client (GISC) WMO «Request» Schema « B » «Register» «Response» OAI Server VGISC protocol communication E2EDM records are reformatted into WMO standardrecords are migrated from (A) to (B) DCPC Obninsk « A » Data Providers Integration Server WMO Core Metadata records (v.0,2, v.1.0) Resource catalogue 4. Interaction with other systems OBIS Interexchange rules 1. Intersystem’s connection is provided 2. Data exchange list is agreed 3. Discovery metadata catalogue is used 4. Metadata reformatting is provided JCOMM DCPC Prototype (Obninsk) – 2007, WIS prototype project

14. IODE ODP V.1 Operation Framework • ODP Documentation • Overview of the IODE/JCOMM Ocean Data Portal (April 2009); • E2EDM Technology Overview (V.1.0, July 2008); • The Checklist of technical requirements for data providers for E2E technology implementation (V.1.0, August 2008) • E2EDM technical specification (V.1.0, January 2007) • E2EDM Concept XML Schema (V.1.1, April 2009); • E2EDM Global XML Schema (V.1.0, May 2007); • E2EDM Integration Server User Guide (V.1.0, March 2007); • E2EDM Data Provider User Guide - Installation and setup (V.1.1, July 2008); • E2EDM Data Provider User Guide • Ocean Data Portal. User Guide (V.1.1, April 2009)

15. IODE ODP V.1 Implementation for ODIN’s ODINCARSA ODINCINDIA ODINECET ODIN-BlackSea ODINAFRICA ODIN-WESTPAC ODP for IOC/IODE Ocean Data Infrastructures (ODIN’s) Training courses on ODP: • the IODE Project Office (Oostende, Belgium, October 2007), • RIHMI-WDC/Russian NODC (Obninsk, Russia, March 2009) • NODC of Korea (Seoul, Republic of Korea, August, 2009)

16. IODE ODP V.1 Implementation for JCOMM JCOMM Pilot Project for WIGOS: (i) define and agree on common standards to manage the instrument/platform metadata in cooperation with JCOMM ET (META‐T, VOS/VOSC and etc.) (ii) build a distributed data system and provide its interoperability with WIS. (iii) prepare and submit the ocean and marine meteo standards to be adopted via IODE/JCOMM Standards Process. ODP has been designated as a WIS Data Collection and Processing Centre (DCPC) and will be a resource that can be discovered through the WIS Global Information System Centre (GISCs).

17. WIS Compliance Specifications • Existing centres within WMO Member States may apply for designation as one of the functional centres forming the core infrastructure of WIS: • Global Information System Centres (GISCs) • Data Collection or Production Centres (DCPCs) • National Centres (NCs) • Designation requires a statement of compliance with WIS requirements, compiled by the Inter-Commission Coordination Group on WIS (ICG-WIS) • The WIS Compliance Specifications document is the authoritative source for specifications applicable to WIS GISCs, DCPCs, and NCs

18. WIS and ODP Interoperability WIS-TechSpec-1, Uploading of Metadata for Data and Products. WIS-TechSpec-2, Uploading of Data and Products. WIS-TechSpec-3, Centralization of Globally Distributed Data. Status – not required . WIS-TechSpec-4, Maintenance of User Identification and Role Information. WIS-TechSpec-5, Consolidated View of Distributed Identification and Role Information. WIS-TechSpec-6, User Authentication WIS-TechSpec-7, User Role Authorization WIS-TechSpec-8, DAR Catalogue Search and Retrieval WIS-TechSpec-9, Consolidated View of Distributed DAR Metadata Catalogues WIS-TechSpec-10, Downloading Files via Dedicated Networks WIS-TechSpec-11, Downloading Files via Non-dedicated Networks WIS-TechSpec-12, Downloading Files via Other Methods WIS-TechSpec-13, Maintenance of Dissemination Metadata WIS-TechSpec-14, Consolidated View of Distributed Dissemination Metadata Catalogues WIS-TechSpec-15, Reporting of Quality of Service

19. IODE ODP V.1 Implementation for JCOMM Пользователь Запрос Интерфейсное взаимодействие Ответ Сервер Интеграции Транспортный Транспортный файл данных файл данных Сообщения - ответы Сообщения - запрос Транспортный файл данных Поставщик Поставщик Поставщик Данных Данных Данных ODP will make appropriate ocean and marine meteo datasets to WMO and IOC applications. A number of organizations and programmes have been identified as potential partners and data providers for ODP. Functional view Technical view

20. IODE ODP V.1 Implementation for JCOMM Core JCOMM for WIGOS partners 1) US NODC: World Ocean Atlas; World Ocean Database; US NODC GTSPP; Surface currents from HF radar; 2) Russian Federation NODC: GTS observation data, analysis and forecast data gridded fields 3) Canada, ISDM: Upper-ocean T & S gridded fields; and Ocean currents derived from surface drifters; 4) BODC GLOSS Sea-level data 5) UK Met Office or DWD via Virtual GISC or direct impact to ODP Marine Climatological Summaries and Global Collecting Centres (GCCs) Potential partners 1) In situ data sets from the JCOMM Observations Programme Area (Argo, OceanSITES, TAO, DBCP, SOT, GOSUD 2) Model output fields (e.g., GODAE); 3) Metadata about the platforms / instruments (e.g., META-T, ODAS); 4) Fast delivery sea level data (University of Hawaii Sea Level Center);

21. IODE ODP for JCOMM – current main issue User access ODP Services Integration Server ODP implementation processes are strongly connected with the expansion of data providers How to become the ODP data provider? Data Provider Data Catalogue Data System Interaction

22. IODE ODP for JCOMM – looking forward ODP profile for Climate ODP profile for Marine Hazards ODP profile for Maritime Safety ODP profile…. IODE/JCOMM distributed data system – E2Etechnology realization

23. Thank you for attention Marrakech, Morocco, 4-11 November 2009