1 / 16

Third Plenary Meeting of SeaDataNet

Third Plenary Meeting of SeaDataNet. Bob Keeley Integrated Science Data Management, (ISDM) Canada March, 2009. Overview. General comments Some information about other activities Canada USA Observations on work reported at this meeting General observations Suggestions

kumiko
Download Presentation

Third Plenary Meeting of SeaDataNet

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Third Plenary Meetingof SeaDataNet Bob Keeley Integrated Science Data Management, (ISDM) Canada March, 2009

  2. Overview • General comments • Some information about other activities • Canada • USA • Observations on work reported at this meeting • General observations • Suggestions • Now and after SDN SeaDataNet Mar 2009

  3. General Comments Advantage and disadvantage of a yearly snapshot of the project. I see progress but miss details. Overall, as last year, I am very impressed with the progress and achievements. You have done a lot to further the setting of standards and this is very important. You have the base of a full distributed system that has largely defeated the duplicates problems that plague us. SeaDataNet Mar 2009

  4. What is happening in Canada? DFO is responsible for a much of the oceanographic data collected. Most research data come from the Science Sector of DFO in which ISDM is sited. 4 years ago, Science reorganized so that data management gained a much higher profile and resulted in increased funding We now have a national committee to coordinate activities and roughly 1 million CAD each year for projects (no guarantees into future). Objective is to knit together a whole data system from 7 distinct (largely independent) pieces. SeaDataNet Mar 2009

  5. Scope of projects Fisheries data from trawl surveys, ecosystem work, larval fish, hydroacoustics, … for lots of species. Biological data including plankton measurements, marine mammal and other species tracking. Lots of chemistry and physical oceanography Data rescue projects from both marine and freshwater. Digital images (videos, stills, microarray spectra, …) Web services Data catalogue and metadata (MCP) Gridded bathymetric product served through web services Use of taxonomy (ITIS) Physical sample preservation and tracking. SeaDataNet Mar 2009

  6. Data at ISDM Physical oceanography: delayed mode and lots received in real-time from GTS including from ships, profiling floats, surface drifters, wave buoys, tide gauges. Biological: Invasive species, plankton Chemical: contaminants, nutrients, etc. Bathymetric: all hydrographic charts for Canada plus ENC delivery SeaDataNet Mar 2009

  7. Data Volumes Largest physical oceanography comes from surface drifters. Archive holds global measurements with > 75 million reports and growing at ~ 1 million/month Global real-time profiles (Argo, ships, moorings, …): > 8 million stations and growing about 3 million/year (last year) Wave spectra: ~ 10 million, adding 300,000/year Tide gauges: ~ 4 million variable-days, adding 120,000/year Bathymetry: lots and lots of individual soundings! SeaDataNet Mar 2009

  8. Surface drifter Map Was ~ 33 million in 2004, now is 75 million. SeaDataNet Mar 2009

  9. Summary If nothing else, we have large data volumes to deal with though the variety at ISDM is perhaps less than some of you have. Including the data at regional institutes, we have lots of variety. We have the same issues about placing large archives on-line for download. Some of our requests are very large. We have same issues of adopting/creating standards. We need a robust architecture to deal with additional kinds of data. We need to serve increasingly more varieties of data on-line. SeaDataNet Mar 2009

  10. What is happening in U.S.? DMAC activity in the US has been reinvigorated in last 18 months They have also been pursuing standards some (many?) of which are different from yours They are also are building a distributed, interoperable system without (I believe) an equivalent of CDI, CSR, etc. SeaDataNet Mar 2009

  11. Observations on work reported at this meeting - 1 There seems to have been some intercomparisons of QC at least with WODB. Has there been a check of internal consistency? I would like to see a document. The CDI at present is roughly 1/2 million records. How will the performance scale when this is 10X, 100X larger? (Roy says no problem). Some partners already deal in real-time / operational data and from a variety of sources. This starts to look like the real-time data we process. Does SDN want to open this door further? It will be interesting to propose ODV/DIVA to ICOADS. SeaDataNet Mar 2009

  12. Observations on work reported at this meeting - 2 The download limitation of 500 records / transaction is a (near) future problem. Perhaps a solution is to allow larger single transactions but treat them as you do requests for data from centres without the download manager? (i.e. off-line, off peak processing) Climatology generation needs careful validation. WOA is used extensively and so you should prove your results against that (and OI from CORIOLIS). Do sensitivity analyses of results to CL, S/N. Andrew Wolf provided a favourable review of SDN. Did he comment on IT security issues? Can others use his comments to assure their own IT staff of precautions taken to secure the system? A document outlining these precautions would be nice. SeaDataNet Mar 2009

  13. Observations from this meeting • Up until now, SDN has (rightly) focused inwards on getting your project running. I see that though not complete, you have a base running. • I would suggest it is time to begin to look outward for a couple of reasons: • Your standards developments are important to more than yourselves. • As SDN completes, this is a way to grow what has been produced and sustain the initiative into the future. SeaDataNet Mar 2009

  14. Suggestions to Consider DMAC and SDN are diverging. Send someone to the next possible DMAC meeting to do a presentation. Share your standards work with the world. Use the OceanDataStandards web site and process to do this. (see http://oceandatastandards.org ) JCOMM and IODE are developing the ODP in support of a funded pilot project from WMO. But ODP and SDN seem to be on different development paths. What can be done to remedy this? Do you want 2 systems in Europe? SeaDataNet Mar 2009

  15. Now and after SDN - 1 SDN provides the funding to build a distributed system, but the “glue” that keeps these together is the coordination activities. After SDN, without this coordination, the system risks breaking. Use the international systems, IODE, JCOMM, to help continue the coordination into the future even if you secure more EU funding. When funding ends, what will other partners do if one drops out? OceanObs’09 will take place in September this year. A plenary paper will set out the 10 year objective for ocean data management. You want to have a voice. SeaDataNet Mar 2009

  16. Now and after SDN - 2 The WDC system is undergoing a large change right now. Given that SDN is a distributed system that preserves data in the long term, what do you want from the WDCs? What is the relationship between SDN and the various DACs that now exist? What connections would you want to make to the various data collection panels that operate under JCOMM? What do you want from and what do you want to contribute to IODE and JCOMM? SeaDataNet Mar 2009

More Related