1 / 8

Near Real Time Ocean Observations Online

Near Real Time Ocean Observations Online. the Detailed Escape of SEACOOS (Southeastern Atlantic Coastal Ocean Observing System) Data Management and Visualization Secrets. Software : engines. Mapserver Open source from University of Minnesota USC runs a mixture of versions from 3.6 to 4.2

albina
Download Presentation

Near Real Time Ocean Observations Online

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Near Real TimeOcean Observations Online the Detailed Escape of SEACOOS(Southeastern Atlantic Coastal Ocean Observing System) Data Management and Visualization Secrets

  2. Software : engines • Mapserver • Open source from University of Minnesota • USC runs a mixture of versions from 3.6 to 4.2 • PostgreSQL • Open source from postgresql.org • USC runs version 7.4.1 • PostGIS • Open source from Refractions.net • USC runs version 0.8.1 Near Real Time Ocean Observations Online : SEACOOS

  3. Software : additional • PHP • Open source from php.org • USC runs version 4.3.2 • Perl • Open source from perl.org • USC runs version 5.8.0 • Miscellaneous • ANiS and gifsicle • Imagemagick Near Real Time Ocean Observations Online : SEACOOS

  4. application server Apache 2.x MapServer Perl, PHP, misc. Hardware • data scout • In-situ • RS • database A • In-situ • RS • database B • In-situ • model output Near Real Time Ocean Observations Online : SEACOOS

  5. Directory structure • Data that sits on USC filesystems • RS images • cached images • All files have strict naming convention that includes timestamp Near Real Time Ocean Observations Online : SEACOOS

  6. System administration • Databases backed up nightly • requires some downtime (~ 2 hours) • Databases cleaned up and optimized nightly • significant overhead ~ 4 hours • Some backend data massaging (mainly for model output aggregation) Near Real Time Ocean Observations Online : SEACOOS

  7. Database structure • One table category per data type • e.g. in-situ winds • wind_prod contains all wind data • wind_map contains wind data appropriate for maps • e.g. QuikSCAT winds • more complicated since requires aggregation and normalization • e.g. OI SST • table containing pointers to data files on disk • Ancillary tables for specific data types • e.g. OI SST • table containing RGB to SST lookup values (for querying purposes) Near Real Time Ocean Observations Online : SEACOOS

  8. Data processing overview • Data scout (netCDF) • Perl code flattens incoming netCDF into arrays which are turned into SQL INSERT statemtents. • Triggers update the new records suitable for normalization as well as Mapserver display elements. • Procedures run to optimize tables for display. • RS images (HDF to PNG) • begin as HDF but are availed to USC as PNG’s • Images have standard naming convention and agreed upon extents as well as predefined RGB to value, e.g. RGB to SST, pairs. • Incoming files cause postgreSQL table to be updated with the new file and timestamp. Near Real Time Ocean Observations Online : SEACOOS

More Related