1 / 25

State-of-the-art tools and practices for marine data and information management – a forward look

State-of-the-art tools and practices for marine data and information management – a forward look. Lesley Rickards British Oceanographic Data Centre (BODC) Chair, International Oceanographic Data and Information Exchange (IODE) Committee. What are we aiming for?

mateja
Download Presentation

State-of-the-art tools and practices for marine data and information management – a forward look

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. State-of-the-art tools and practices for marine data and information management – a forward look Lesley Rickards British Oceanographic Data Centre (BODC) Chair, International Oceanographic Data and Information Exchange (IODE) Committee MAMA MD&IM Workshop - 28 January 2004

  2. What are we aiming for? • Simple access to all types of marine data (and information) on an appropriate time scale • A virtual or distributed data centre • Data available on CD-ROM/DVD • End to end data management Where are we now? • A collection of separate centres with a wide variety of remits, skills and data • Made up of NODCs, RNODCs and WDCs • Sometimes work together in groups for individual projects MAMA MD&IM Workshop - 28 January 2004

  3. MAMA MD&IM Workshop - 28 January 2004

  4. Data Management and Communication System for the coastal module of GOOS MAMA MD&IM Workshop - 28 January 2004

  5. Data management practices: • ‘Best’ rather than ‘state of the art’ • Compliance with IOC Data Exchange Policy • Proper collaborative efforts, building on existing standards and practices – not reinventing the wheel each time a new project comes along • Cooperation – partnership – collaboration MAMA MD&IM Workshop - 28 January 2004

  6. Timely, efficient and open access to the best possible data, metadata & associated products • Metadata Standards • Discovery • Accompanying data (position, date/time, etc) • Data Documentation (qc history) • Quality Control • Automatic tests • ‘Scientific’, delayed-mode • Data dissemination • CD-ROM/DVD • On-line access • Long term stewardship of data MAMA MD&IM Workshop - 28 January 2004

  7. Data management tools - METADATA • Metadata is all the descriptive information necessary to allow users to find (discover), access, manipulate, process (request) and extract (recover) data, information and products • Standards • ISO19115 • Dublin Core • MEDI, GCMD, EDMED, FGDC, ANZLIC, (EDIOS), (CSR), etc, etc. • Mappings between standards available (cross-walks) • ** Compliance with ISO19115 ** • Various controlled vocabularies • Links with XML (schemas/DTDs) • MEDI Authoring Tool • All have search tools • geospatial location, temporal information, keywords, controlled vocabulary items, “free text” MAMA MD&IM Workshop - 28 January 2004

  8. eXtensible Markup Language (XML) • XML is being widely used as a basis for both dynamic web page development and more generally as a data exchange mechanism • Data exchange aspects of XML include the ability to define flexible data structures that utilise the terminology of the subject area • Data to be exchanged is packaged in a form more intuitive to the user. • Extensive availability of free software for manipulation and transformation of the XML data stream • Allows developers to easily develop, populate, exchange and transform data streams. MAMA MD&IM Workshop - 28 January 2004

  9. MarineXML EU MarineXML “… demonstrate how XML technology can be used to develop a framework that improves the interoperability of data for the marine community and specifically in support of marine observing systems. The project will develop a prototype of an XML-based Marine Mark-up Language (MML).” ICES-IOC SGXML “… utilize or establish international standards to promote the seamless exchange of data from distributed data sources, by using a single parameter dictionary, well-defined and explicitly tagged metadata, and a common XML data structure, packaging all content and providing to the client datasets and software tools that are platform independent or web enabled” MAMA MD&IM Workshop - 28 January 2004

  10. Quality Control • (Real-time) automatic tests (but do not rely on them) • Quality flags Data visualisation tools • to include comparison with other data collected in the same place or nearby, climatology; different ways of looking at the same data • Ocean Data View • Ncbrowse (for netCDF files) • EPIC (management, display and analysis of oceanographic and meteorological data) • Sea Level data • POL TASK2000 package + on-line tidal analysis • University of Hawaii JASL software • ESEAS – working towards new package • Document QC (e.g. audit trail, data history) • Use existing standards/guidelines where available • ICES data type guidelines, WOCE standards MAMA MD&IM Workshop - 28 January 2004

  11. Ocean Data View (ODV) • Interactive exploration and graphical display of oceanographic and other geo-referenced profile, sequence or gridded data • Runs on Windows (9x/NT/2000/XP), Linux, UNIX, and Mac OS X • Data collection and configuration files are platform independent • Interactive browse through large sets of station data MAMA MD&IM Workshop - 28 January 2004

  12. Ocean Data View (ODV) • High-quality station-maps, general property-property plots of one or more stations, scatter plots of selected stations, property sections along arbitrary cruise tracks and property distributions on general iso-surfaces • Display of original scalar and vector data by coloured dots, numerical data values or arrows • Fast gridding algorithms allow colour shading and contouring of gridded fields along sections and on iso-surfaces • Derived quantities calculated dynamically, displayed and analysed MAMA MD&IM Workshop - 28 January 2004

  13. MAMA MD&IM Workshop - 28 January 2004

  14. Development of distributed systems OPeNDAP (DODS) • Data servers for making local data accessible at remote locations • Free software for download Live Active Server (LAS) • Best for large, gridded environmental data sets • Dynamically generated graphics • Compare variables from distributed locations (using DODS) Thematic Real-time Environmental Data Distributed Services (THREDDS) • Access to large collection of real-time and archived data sets from a variety of data sources • Analysis and display software MAMA MD&IM Workshop - 28 January 2004

  15. Other examples of distributed systems • US GLOBEC (US JGOFS) • NOAA Coastal Directory • Russian ESIMO • NERC Data Grid • IFREMER/SISMER Data Portal • SeaSearch Common Data Index • Distributed Generic Information Retrieval (DiGIR) – protocol for retrieving structured information from multiple heterogeneous databases • etc. MAMA MD&IM Workshop - 28 January 2004

  16. SISMER Web Portal NOW: SISMER WEB interface WEB interface 1 WEB interface i WEB interface 2 SISMER Server Database« Datasets catalogues » Thematicdatabase 1 Thematic database i Thematicdatabase 2 SISMERWEB portal FUTURE: SISMER Web Portal Automatic querying of existing systems SISMER Server Database« Data sets catalogues » XML / ISO 19115 integrator 1 XML / ISO 19115 integrator 2 XML / ISO 19115 integrator i Thematicdatabase 2 Thematicdatabase 1 Thematic database i MAMA MD&IM Workshop - 28 January 2004

  17. MAMA MD&IM Workshop - 28 January 2004

  18. ArgoData Management • Data transmitted in real-time by satellite • Transferred to data centres • Messages decoded • Automatic real-time quality control tests • Data passed to global centres (GDACs) for dissemination • Delayed-mode quality control and calibration • Replacement version sent to GDACs • Regional centres for further quality control and products MAMA MD&IM Workshop - 28 January 2004

  19. Location of Argo float profiles • 01 – 27 Jan2004 • No. of profiles: • 2139 • Data available for download: • Profiles • Trajectory • Metadata • Technical info • From: • US GODAE • CORIOLIS MAMA MD&IM Workshop - 28 January 2004

  20. Ocean Biogeographic Information System (OBIS) • Web-based provider • Global geo-referenced information • Accurately identified marine species • Expert species level and habitat level databases • Variety of spatial query tools for visualizing relationships among species and their environment. • Strives to assess and integrate biological, physical, and chemical oceanographic data from multiple sources. • Users, including researchers, students, and environmental managers, gain a dynamic view of the multi-dimensional oceanic world Part of the Census of Marine Life (CoML) MAMA MD&IM Workshop - 28 January 2004

  21. Gulf of Maine Biogeographic Information System (GMBIS) • Regional implementation for CoML • Partner with OBIS • Integrated into the Gulf of Maine Ocean Observing System (GoMOOS) • Designed to assimilate and integrate marine ecosystem and fisheries data, as well as natural-history information • Included is an advanced oceanographic geographic information system (GIS) MAMA MD&IM Workshop - 28 January 2004

  22. Gulf of Maine Biogeographic Information System (GMBIS) • Access to biological, physical, chemical and geological data and information • Enhance understanding of biological patterns and their changes through time • “Aggregation server” providing access, rapid visualization and data download capabilities • Server will rely on a combination of archived (local access) data as well as dynamic access to remote data providers • Visualisation and other interactive software will be designed to help the user evaluate what data are available, combine data layers and download data MAMA MD&IM Workshop - 28 January 2004

  23. Gulf of Maine Biogeographic Information System (GMBIS) • Access to: • historical data • taxonomic collections • geological base maps • modern remote sensing data (buoys, CODAR, satellite) • modelling products • broad-scale survey data (e.g. living marine resources) • other monitoring programs (e.g. COASTWATCH, Continuous Plankton Recorder) MAMA MD&IM Workshop - 28 January 2004

  24. GMBIS uses Environmental Analysis System (EASy) • Storage, dissemination, analysis, integration, and dynamic display of spatially referenced series of oceanographic data • PC-based • Aids interfacing of multivariate oceanographic data • Both data and model outputs can be imaged in time through diverse kinds of displays • Including vector, contour, and false-colour imagery • Vertical structures can be depicted along line transects or point-sampling stations • Time series can also be visualised • Patterns in the spatial distribution or organisms to be visualized and compared to other spatial distributions • Including both biological and other oceanographic variables • Even when characterised by different scales of sampling and different degrees of resolution MAMA MD&IM Workshop - 28 January 2004

  25. CONCLUSIONS • Use of modern IT techniques in a transparent manner to improve service to users (internet, web, distributed systems, XML, etc.) • Setting the standards (metadata, data quality control, data stewardship) • Working collaboratively whilst responding to national remit • Increasing data diversity (many different parameters being measured) • Developing systems to deliver (near) real-time data MAMA MD&IM Workshop - 28 January 2004

More Related