1 / 7

Still serving data with an old DODS server from the early 90's

Still serving data with an old DODS server from the early 90's. Jim Manning NOAA's Northeast Fisheries Science Center NERACOOS/NECOSP Data Management Workshop, Sept. 26, 2012. NEFSC Oceanography Branch. Falls under the “ Ecosystem Processes Division ” based in Sandy Hook, NJ

presta
Download Presentation

Still serving data with an old DODS server from the early 90's

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Still serving data with an old DODS server from the early 90's Jim Manning NOAA's Northeast Fisheries Science Center NERACOOS/NECOSP Data Management Workshop, Sept. 26, 2012

  2. NEFSC Oceanography Branch • Falls under the “Ecosystem Processes Division” based in Sandy Hook, NJ • The branch is based in Narragansett, RI but several of us are located in Woods Hole • Primary functions are to describe & monitor: • basic physical oceanographic processes (T,S, U&V) • status of lower trophic levels • the long-term variability of conditions on the shelf • Primary data consumers: • Fisheries biologist at our lab and elsewhere • Academic researchers • Data Requirements: • ORACLE

  3. About our data • Water Column Profiles (CTD) • Trawl Survey (merged with above) • Zooplankton • Derived Temperature and Salinity Anomalies by Region • Ship Track Surface Properties • Drifter Track Archive • Moored Time series (includes eMOLT & historic current meters) • All of the above are shared (ie accessible) on our website • 1,5,6,7 via OpeNDAP on old DOS server • All of the above (except 3) visualize w/MapServer/gnuplot • All of the above are updated at least a few times per year • All of the above programs are multi-decades (except mine, 6&7) • Most of the above documented on GCMD website

  4. How we manage/serve our data Two options: 1) point & click through Perl/Mapserver Dataserver 2) download example MATLAB or PYTHON code to access it yourselves

  5. Data Management Successes • Original DODS server for relational database • Python & Matlab users can bring the data directly into their working environments • Dozens of data users have registered at least once to let us know they are using the data and what for • Numerical modelers are using the eMOLT data, for example, to assimilation and validation of simulations

  6. Data Management Challenges • Political hurtles: • NEFSC “Oceanography Branch” does not have a person trained in data management so we have to beg the “Data Management Branch” • Overcoming the long-standing perception that there are more pressing issues in fisheries science than monitoring the effects of climate change (sector management, whales, etc. are more important to many) • Technical hurtles: • There is relatively little support in IOOS for serving relational databases? • SOS required MySQL and therefore, for me, a duplicate database? • Only one LINUX system admin guy who deals mostly with ~150 WINDOWS people • Still waiting for a new LINUX box to be setup for updating our server so we can add THREDDS and Python functionality

  7. Take home points • NEFSC has collected a ton of data for decades • eMOLT and Drifter Track archive is just a subset and is a relatively new addition • Most of it is now served either through: • Point & click MapServer interface • OPeNDAP protocols • We are still using an old DODS relational database server on a 10 year old LINUX box • James.manning@noaa.gov

More Related