Data access to distributed databases A case study from oceanography Taco de Bruin

1. Data access to distributed databases A case study from oceanography Taco de Bruin SCADM/SCAGI meetings Amsterdam, NL, Sep. 7-9, 2009 NIOZ is part of the Netherlands Organisation for Scientific Research (NWO)

2. Goal of all our data management activities is to provide the user access to the data holdings “Collect once, use many times” NIOZ is part of the Netherlands Organisation for Scientific Research (NWO)

3. Situation of data exchange within NL-NODC a couple of years ago: • Online accessibility and availability of data • Waterbase (RIKZ) • CODIS (NIOZ) • KODAC (KNMI) • Watermarkt (RIKZ, many other institutes) • HYDRA (KNMI) • COREBASE (NIOZ) • DINO (TNO- B&O) • Getij.nl (RIKZ) • Waterstat.nl (RIKZ) NIOZ is part of the Netherlands Organisation for Scientific Research (NWO)

4. Situation of data exchange within NL-NODC a couple of years ago: • Online accessibility and availability of data • Waterbase (RIKZ) • CODIS (NIOZ) • KODAC (KNMI) • Watermarkt (RIKZ, many other institutes) • HYDRA (KNMI) • COREBASE (NIOZ) • DINO (TNO- B&O) • Getij.nl (RIKZ) • Waterstat.nl (RIKZ) NIOZ is part of the Netherlands Organisation for Scientific Research (NWO) MEDIN Meeting 20090630

5. Situation of data exchange within NL-NODC a couple of years ago: • Online accessibility and availability of data • Waterbase (RIKZ) • CODIS (NIOZ) • KODAC (KNMI) • Watermarkt (RIKZ + many partner institutes) • many more ….. • - user had no overview of data availability- user had to do all format conversions and syntheses • - user had to spend at least 50% of time on finding data and data conversion, instead of doing science NIOZ is part of the Netherlands Organisation for Scientific Research (NWO)

6. SeaDataNet: A Pan-European distributed data access infrastructure http://www.seadatanet.org/data_access NODC-i: A national distributed data access infrastructure, based on SeaDataNet technology http://www.nodc.nl – Data Access Services NIOZ is part of the Netherlands Organisation for Scientific Research (NWO)

7. SeaDataNet, consortium of national oceanographic data centres: 35 countries IOC/IODE ICES JRC Total 49 partners, links with JCOMM, WDC, DMAC, MMI, EDIOS, EU-MarBEF (NoE), EU-Seased, ESEAS, Esonet, EuroGeoSurveys, POGO, etc. NIOZ is part of the Netherlands Organisation for Scientific Research (NWO)

8. SeaDataNet System Search & shop SeaDataNet Portal Data centre Data centre Data centre Data centre Metadata + transaction data NIOZ is part of the Netherlands Organisation for Scientific Research (NWO)

9. Common Data Index (CDI) • CDI is a fine-grained index to individual data measurements (such as a CTD cast or moored instrument record). • CDI: ‘common’ metadata of individual measurements (location, time, parameter, availability, owner, etc.) CDI records from >29 countries NIOZ is part of the Netherlands Organisation for Scientific Research (NWO)

10. SeaDataNet system User interface CDI Central User Register Shopping basket Request Status Manager Download Manager Download Manager Download Manager data data data NIOZ is part of the Netherlands Organisation for Scientific Research (NWO)

11. CDI Search – home page NIOZ is part of the Netherlands Organisation for Scientific Research (NWO)

12. Search – records selected for shopping basket NIOZ is part of the Netherlands Organisation for Scientific Research (NWO)

13. Contents of shopping basket NIOZ is part of the Netherlands Organisation for Scientific Research (NWO)

14. Submit Shopping list => User Authentication NIOZ is part of the Netherlands Organisation for Scientific Research (NWO)

15. Authentication, Authorization and Administration • - Single Sign On system set up for access to distributed system • - User’s authentication information based on personal login / password • - User must register to get one login, thereby agreeing with the “SeaDataNet User Licence” • - Central User Register, but management is decentralized • - Authorisation based on “User Roles” NIOZ is part of the Netherlands Organisation for Scientific Research (NWO)

16. Data access restrictions in CDI V1 See Vocabulary L081 for definitions Multiple restrictions may apply NIOZ is part of the Netherlands Organisation for Scientific Research (NWO)

17. User Roles in AAA Register, of relevance for data access See Vocabulary C866 for definitions Multiple roles may apply NIOZ is part of the Netherlands Organisation for Scientific Research (NWO)

18. Restriction Role A B C D U OK OK TBD OK V TBD OK OK NO W NO TBD OK OK Business Matrix NIOZ is part of the Netherlands Organisation for Scientific Research (NWO)

19. Request confirmed and link to RSM (also by e-mail) NIOZ is part of the Netherlands Organisation for Scientific Research (NWO)

20. User RSM – Overview of request per Data Centre and Status NIOZ is part of the Netherlands Organisation for Scientific Research (NWO)

21. Downloading data from SeaDataNet infrastructure: • Standard formats (depending on data type): • - ODV ASCII • NetCDF • MedAtlas • Agreed QC flags NIOZ is part of the Netherlands Organisation for Scientific Research (NWO)

22. SeaDataNet is a working, distributed data delivery infrastructure, featuring all elements required by SCADM, SCAR or the Southern Ocean Observing System (SOOS). The big challenge will be to couple the European SeaDataNet infrastructure to the emerging Australian and American systems and prevent these developments from diverging. The International Oceanographic Data and Information Exchange Committee (IODE) is working on achieving this with its Ocean Data Portal project. NIOZ is part of the Netherlands Organisation for Scientific Research (NWO)