Update on IOC data strategy and policy and the Ocean Information Technology (OIT) Pilot Project

1. Update on IOC data strategy and policy and the Ocean Information Technology(OIT) Pilot Project Neville Smith BMRC, Melbourne, Australia

2. Synopsis • Why? • The international community is keenly interested in NVODS/OPeNDAP and the development of DMACS. • Work out how we should work with these initiatives; things we can use, things we can do • Data exchange policy • Aspects of international arrangements • Key groups and relevant parts of their agendas • Major drivers and influences • The OIT Project NVODS Workshop 2003

3. IOC Oceanographic Data Exchange Policy Clause 1: timely, free and unrestricted access to all data, associated metadata and products from IOC Programs. Clause 2: as 1, but for essential data from non-IOC programs Clause 3: non-commercial use Clause 4: acknowledges the right of Member States and data originators to determine the terms of exchange Clause 5: to the best practicable degree, use data centres linked to IODE’s NODC and WDC network Clause 6: capacity enhancement in developing countries NVODS Workshop 2003

4. How are international data management activities coordinated? NVODS Workshop 2003

5. Review of joint IOC/JCOMM DM • Carry out an assessment of data and data-product requirements of existing oceanography and marine meteorology programmes/projects, and evaluate whether these are currently met by the various groups of IODE data centres; • Take into consideration existing data management plans such as the GOOS data management plan and relevant WMO data management plans; • Draft an IOC Strategic Plan for Oceanographic Data and Information Management, taking into consideration the requirements for such a plan in the framework of JCOMM; NVODS Workshop 2003 “A comprehensive and integrated ocean data and information system, serving the broad and diverse needs of IOC Member States, for both routine and scientific use.”

6. Specific Objectives • Support for scientific research and resource assessments at a national, regional, and global level (both the regional and global aspects have international contexts). o We must have active scientific participation in the development and execution of the DM Plan, including in the governance structure. • Collaborate with other inter-governmental bodies to ensure greater flexibility in timely and cost-effective access to data and information; • Support IOC’s commitment to its member countries and international organizations (wide spectrum of users). • To provide reliable, cost effective and affordable access for developing- as well as developed-country Members, o Technologically sustainable and appropriate to local expertise; o Modular and scalable; o Flexible, able to adjust to changing requirements and allow dissemination of products from diverse data sources. • Embrace capacity building activities NVODS Workshop 2003

7. regional and specialised data centres (i) To serve the data and information management requirements of a GOOS and Regional Alliances; (ii) To satisfy the requirements of an IOC-defined region or Regional Subsidiary Body. (iii) To satisfy the D&IM requirements of other regional programs, e.g. an LME or Regional Seas program; (iv) To satisfy a specialist requirement, e.g. a science program, a specific data service (e.g., sea level) (v) To satisfy capacity building requirements, such as through the Ocean Data and Information Networks (ODIN); (vi) Geopolitical, geographic or other forms of regional affinity (as noted in the UNESCO approach); (vii) WMO regional associations; NVODS Workshop 2003

8. Organisational Structure(future) WMO and the WWW NVODS Workshop 2003

9. Organisational Structure(future) WMO and the WWW NVODS Workshop 2003

10. Ideas and Concepts of the Future WMO Information System (FWIS) Excerpts from a presentation byDr. Robert STANEK

11. Source: Report by R Stanek of the Inter-Programme Task Team on FWIS TRENDS IN INFORMATION TECHNOLOGIES The Internet has many capabilities not possessed by the GTS. One example, realtime monitoring of its health (including data loss and latency) Increasing data volumes from numerical weather prediction systems NVODS Workshop 2003

12. FWIS REQUIREMENTS • To provide an integrated approach to meeting the requirements of: • Routine collection of observed data; • Automatic dissemination of scheduled products, both real‑ and non-realtime; • Ad hoc, non-routine applications (e.g. requests for non-routine data and products). NVODS Workshop 2003 The system will need to be: • Reliable; • Cost effective and affordable for developing- as well as developed-country Members; • Technologically sustainable and appropriate to local expertise; • Modular and scalable; • Flexible, able to adjust to changing requirements and allow dissemination of products from diverse data sources. Source: Report by R Stanek of the Inter-Programme Task Team on FWIS

13. FWIS REQUIREMENTS The system will also need to support: • Different user groups and access policies; • Integration of diverse data sets; • Data as well as network security; • Ad hoc, as well as routine requests for data and products ("pull" as well as "push"); • Timely delivery of data and products (appropriate to requirements). NVODS Workshop 2003 Source: Report by R Stanek of the Inter-Programme Task Team on FWIS

14. FWIS Communication Topologies NVODS Workshop 2003 Source: Report by R Stanek of the Inter-Programme Task Team on FWIS

15. DATA AND PRODUCT COLLECTION The collection of observations, and aggregation of products in national, regional and global centres is a critical activity. NVODS Workshop 2003 Data transport will be mostly regularised and standardised, not general Source: Report by R Stanek of the Inter-Programme Task Team on FWIS

16. DATA AND PRODUCT DISTRIBUTION The distribution of data and products will occur both through broadcast and ad hoc request. NVODS Workshop 2003 Source: Report by R Stanek of the Inter-Programme Task Team on FWIS

17. Development Path in Response to Increasing Requirements NVODS Workshop 2003 Source: Report by R Stanek of the Inter-Programme Task Team on FWIS

18. ETDMP (/ GETADE) • Chaired by Nic Mikhailov and closest analogue of DMACS (several here involved next week) • Focus is end-to-end DM system NVODS Workshop 2003 Source: Draft ETDMP Strategy

19. USERS E2EDM Web-portal Data system 1 Data sources • Thematic DM functions: • Collection; • Assembling; • Archiving; • Processing/product generation Data system … Data system 2 E2EDM Integration function Data cetntre E2EDM “vision” • Best practice DM components • Telecommunications, assembly, etc. • Agreed rules, protocols, standards • Linked through “integration technology” Source: Draft ETDMP Strategy (Mikhailov) NVODS Workshop 2003

20. E2EDM integration technology “architectural components” • Somewhat analogous to DMACS data transport element (NVODS) Source: Draft ETDMP Technology (Mikhailov) NVODS Workshop 2003

21. The Team’s work … • Covers Marine Met and telecomms • Otherwise drivers similar to here • Their “regional” == US IOOS DMACS • Their local == DMACS/NVODS regional/pilots • Large effort in metadata management • Led by study group on XML • Having many existing prototypes to wrestle with • Data assembly • “Operational” : GTSPP, GODAR… • Pilot: GOSUD, Argo, GODAE, … • Integrity issues (Keeley) • New data types, e.g. biological • … NVODS Workshop 2003

22. Practical considerations • Integrated individual implementations User services NVODS Workshop 2003

23. Practical considerations • Complexities of actual data flow • Local, regional, global links at various stages NVODS Workshop 2003

24. NVODS Workshop 2003

25. NVODS Workshop 2003

26. NVODS Workshop 2003

27. NVODS Workshop 2003

28. NVODS Workshop 2003

29. NVODS Workshop 2003

30. NVODS Workshop 2003

31. Conclusions • Policy: international framework enables and promotes data exchange • Premium on data transport efficiency and effectiveness • Diversity and unevenness of providers and users a challenge • International community shares enthusiasm for innovative data management • Playing field is starting to be ordered • This community needs strong links to DMACS and NVODS – they also have much to offer • The arrangement of Centres will be critical • National Centres is the weakness • We will be strongly linked to the Met community, at least at the operational level • Ocean metadata model highest priority • Data set integrity, versions: assembly NVODS Workshop 2003