1 / 30

Caro-COOPS Data and Information Management in Support of

Caro-COOPS Data and Information Management in Support of Integrated Ocean Observing System Efforts Part I: Where we are. Scientific Advisory Board Meeting Columbia, SC 16-18 December 2003. What is Caro-COOPS?.

addison
Download Presentation

Caro-COOPS Data and Information Management in Support of

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Caro-COOPS Data and Information Management in Support of Integrated Ocean Observing System Efforts Part I: Where we are Scientific Advisory Board Meeting Columbia, SC 16-18 December 2003

  2. What is Caro-COOPS? A wholly-integrated system for coastal observations and their application to user needs • An extensive array of instrumented moorings • A comprehensive data and information management system • An advanced suite of integrated models Scientific Advisory Board Meeting Columbia, SC 16-18 December 2003

  3. IOOS DMAC High-priority Recommendations Data suppliers wishing to participate in IOOS immediately should adopt initial DMAC recommendations including (IOOS DMAC, 2003): - Creating FGDC-compliant metadata - Enabling data discovery - Making data available through the OpeNDAP data access protocol - Making data browsable on-line through the LAS - Ensuring long-term archival of contributed data Scientific Advisory Board Meeting Columbia, SC 16-18 December 2003

  4. A comprehensive data management system

  5. Data and Information Management Subsystem A comprehensive data and information management system consisting of three primary components: 1. Reception, management and transfer of primary and ancillary observation data 2. Delivery of QC’d data, metadata and data products to a Web-based data and information portal 3. Provision of user-friendly tools and information products for model output Scientific Advisory Board Meeting Columbia, SC 16-18 December 2003

  6. Data and Information Management Subsystem Specifically, our system includes: 1. Offshore and shore-based near real-time data transmission via the low-orbit Iridium satellite cellular communications system, the Orbcomm low-orbit satellite communication system, and NOAA’s GOES 2. Data assimilation from observing systems and ancillary data providers (e.g. NDBC) to facilitate data parameterization for modeling and product generation 3. Archival within a relational database system for near real-time display and permanent storage in netCDF format Scientific Advisory Board Meeting Columbia, SC 16-18 December 2003

  7. Data and Information Management Subsystem Specifically, our system includes (cont.): 4. Access to data via DODS/OPenDAP 5. Access to data and derived products available to the public and research community via a web-based portal 6. Enhanced FGDC metadata development, search and query capabilities 7. Management and visualization (e.g. GIS, LAS) tools allowing users to generate products over the web Scientific Advisory Board Meeting Columbia, SC 16-18 December 2003

  8. Data and Information Management Subsystem Current activities focus on: - Maintain and enhance techniques for data assimilation processing and product development - Support models with improved input source data - Maintain and enhance QA/QC procedures - Enhance data documentation tools and techniques - Develop products and web access for user communities Scientific Advisory Board Meeting Columbia, SC 16-18 December 2003

  9. Data and Information Management Subsystem Maintain and enhance techniques for data assimilation processing and product development - IOOS DMAC-compliant but not complacent! - Using relational databases to address diverse dataset types and formats - Working closely with SEA-COOS, CORMP, GoMOOS, SURA, NWLON, NDBC, NCDDC to be cognizant of and accommodate diversity of data types Scientific Advisory Board Meeting Columbia, SC 16-18 December 2003

  10. Data and Information Management Subsystem Maintain and enhance techniques … (cont.) - Assimilating real-time observations via data telemetry, and using a variety of techniques (e.g. screen scraping, ftp) for ancillary data receipt - Using web functionality and Internet-based GIS tools for data ingest, data browsing, product development, and data and product distribution Scientific Advisory Board Meeting Columbia, SC 16-18 December 2003

  11. Data and Information Management Subsystem • Support models with improved input source data • We recognize the importance of quality input source data for modeling and applications • We understand the limits and implications of our source data and products • - Plans to discuss collaboration with interested groups (e.g. USCoE, SCDNR, county governments) for mutually important data development activities Scientific Advisory Board Meeting Columbia, SC 16-18 December 2003

  12. Data issues of spatial and temporal concern

  13. Data and Information Management Subsystem Maintain and enhance QA/QC procedures Relevant QA/QC issues: - On-instrument QA/AC - QA/QC documentation detailing instrument calibration and maintenance procedures - Organizational QA/QC (I.e. the QA/QC link with ancillary data providers) - Regional QA/QC (I.e. data are checked against known or expected environmental range for geographic region) - Feedback QA/QC (I.e. data are checked against immediate past and future readings) Scientific Advisory Board Meeting Columbia, SC 16-18 December 2003

  14. Data and Information Management Subsystem • Maintain and enhance QA/QC procedures (cont.) • - Obviously, the quality of our models and products are dependent upon the quality of our data • - Goal is to minimize turnaround time for release of QA/QC’d data • - Initial QA/QC efforts involve: • 1.) Raw data checked for transmission errors • 2.) Instrument range checks • 3. Time continuity checks Scientific Advisory Board Meeting Columbia, SC 16-18 December 2003

  15. Data and Information Management Subsystem Enhance data documentation tools and techniques - Importance of metadata development within Caro-COOPS cannot be overstated - Metadata standards include: 1. FGDC 2. EML 3. SensorML - Key operational areas supported by metadata standards include observational data, instrumentation, methods and QA/QC Scientific Advisory Board Meeting Columbia, SC 16-18 December 2003

  16. Data and Information Management Subsystem Enhance data documentation tools … (cont.) - Caro-COOPS is developing a metadata editing tool to be incorporated with a data upload wizard - Initially this tool will be a web-enabled metadata editor based on the model developed by the SURA-sponsored Cast-Net initiative. Scientific Advisory Board Meeting Columbia, SC 16-18 December 2003

  17. Data and Information Management Subsystem • Develop products and web access for user communities • - Multiple user communities (e.g. researchers, managers, fisheries) requiring different products • Website provides access not only to data but also information concerning the observational array, instrumentation, products, news and updates • - IOOS-supported data access tools (DODS, LAS) as well as GIS functionality have been implemented Scientific Advisory Board Meeting Columbia, SC 16-18 December 2003

  18. Data and Information Management Subsystem

  19. Data and Information Management Subsystem A HypotheticalCat. 3 hurricane (MCP=960 mb)moving 50 km offshore along the coast (from SW to NE) An advanced suite of integrated models Scientific Advisory Board Meeting Columbia, SC 16-18 December 2003

  20. Data and Information Management Subsystem Products with specific audiences in mind (e.g. emergency planners) Scientific Advisory Board Meeting Columbia, SC 16-18 December 2003

  21. Data and Information Management Subsystem (Future) potential products with specific audiences in mind (e.g. recreational fishermen; courtesy NOAA website) Scientific Advisory Board Meeting Columbia, SC 16-18 December 2003

  22. Data and Information Management Subsystem Visitors and users of the system are on the increase! Scientific Advisory Board Meeting Columbia, SC 16-18 December 2003

  23. Caro-COOPS Data and Information Management in Support of Integrated Ocean Observing System Efforts Part II: Where we are going Scientific Advisory Board Meeting Columbia, SC 16-18 December 2003

  24. Data and Information Management Subsystem Caro-COOPS data and information management efforts provide for … ‘developing in situ and remote sensing techniques for near real-time provision of data on key biological and chemical variables’ (Malone 2003). ‘designing and implementing an enhanced, distributed data and information management system that links all observational and data management systems (across agencies and programs) to all data users’ (Ocean.US 2002). ‘providing an effective and sustained program for enhanced public awareness and education’ (Ocean.US 2003) Scientific Advisory Board Meeting Columbia, SC 16-18 December 2003

  25. Data and Information Management Subsystem Caro-COOPS data and information management efforts provide for … ‘enhancing existing federal networks for in situ measurements … and to improve spatial and temporal resolution … to include physical, chemical and biological variables’ (Ocean.US 2002) ‘adding to the national program of … surveys to establish … multidisciplinary observations (assess the effects of land-based sources of pollution and to improve stock assessments of exploitable fish stocks)’ (Ocean.US 2002). Scientific Advisory Board Meeting Columbia, SC 16-18 December 2003

  26. Data and Information Management Subsystem • Current and proposed Caro-COOPS data and information management efforts are … • - Improving data assimilation, delivery and documentation • Taking information and information products to the user communities • - Putting the “I” in IOOS! Scientific Advisory Board Meeting Columbia, SC 16-18 December 2003

  27. Data and Information Management Subsystem • Improving data assimilation, delivery and documentation • - Program enhancements include: •  establish true 24/7/365 connectivity to moorings with two-hour routine transmissions and on- demand data dumps •  improvements in mechanisms for data delivery from Federal backbone data providers •  enhance GIS capabilities focusing on both Internet-based GIS tools and stand-alone GIS applications •  develop more rigorous QA/QC processes and evaluate appropriate mix of automated versus human-intervention approaches Scientific Advisory Board Meeting Columbia, SC 16-18 December 2003

  28. Courtesy of Data and Information Management Subsystem Taking information and information products to the user communities - Efforts will focus on developing products based on needs identified by specific user communities. Examples include:  working with state and local level emergency preparedness organizations to enhance GIS-based inundation tools  establish data kiosks at public venues including the visitors’ center at Hunting Island, and on the Sunset Pier Scientific Advisory Board Meeting Columbia, SC 16-18 December 2003

  29. Data and Information Management Subsystem Putting the “I” in IOOS! - A major thrust will be the development of Metador, which is:  an enhanced data discovery and documentation tool  focuses on the use of an XML schema (EML) and SensorML while providing for FGDC-compliant metadata  will benefit all IOOS partners, including ocean, terrestrial and estuarine-based systems Scientific Advisory Board Meeting Columbia, SC 16-18 December 2003

  30. Data and Information Management Subsystem Putting the “I” in IOOS! - A second major thrust will be in providing leadership via interoperability projects in designing and demonstrating how local, regional and national systems can successfully integrate Example of an integrated data product developed within the Caro-COOPS, SEA-COOS and SURA/Inter-operability framework Scientific Advisory Board Meeting Columbia, SC 16-18 December 2003

More Related