The Chesapeake Bay Interpretive Buoy System Enterprise Data Management

1. Tucker Pierce Tellus Applied Sciences The Chesapeake Bay Interpretive Buoy System Enterprise Data Management

2. The CBIBS Enterprise Data Management System supports consumers of Chesapeake Bay information and data… • The CBIBS middleware ‘enterprise system’ provides end-users with access to distributed databases containing a wide range of data types. • The system is built on a set of abstracted and loosely coupled components to deliver an open and interoperable architecture with minimum barriers to entry. • The system employs Web Services and XML standards to reduce the system specific knowledge required to access information. • The technology components that currently comprise the CBIBS enterprise system provide the following benefits: • Reduction in data access application development time • Reduced O&M cost through ease of integration of new components • Increased agility in responding to changing system and end-user needs • Standards-based development approach reduces the impact of software obsolescence • A platform and language independent messaging protocol (XML) that facilitates internal and external access to disparate database information and system services. • The on-going CBIBS middleware development process continues to develop capabilities that facilitate end-user access to distributed information: • Incorporation of new data sources into the CBIBS information access framework • Quality assurance options for new data sources (internal and externally sourced) • End-user information discovery, ‘subsetting’, and data access • Metadata definition, implementation and management • Business logic: implementing building blocks for tailored end-user products

3. CBIBS Enterprise Data Management Overview Data Management ‘Middleware’ Retrieving Sensor Systems And Associated Databases Server applications IVR application Web Browser Applications Web Browser Applications Web Browser Applications Web Browser Applications Archiving Web Browser Applications Web Browser Applications Web Browser Applications Web Browser Applications Web Browser Applications Web Browser Applications Data Quality Control Metadata Business Logic/value added processing ‘Front End’ • Provides for data access, and supports third party apps development • Standards-based (XML) data interchange format • Provides for syndicated data feeds such as RSS, Yahoo Pipes • Developer keyed access to allow third party applications to retrieve CBIBS and other data on demand ‘Back End’ Syndicated features, tools, data, and products • Sensor system interface approach decouples framework from sensor hardware specifics • Standards-based (XML) Web Services ingest External Applications And Models

4. Enterprise Data Management Details Public Internet Data Archive Handler Web Browser Applications Web Browser Applications Web Browser Applications Web Browser Applications Web Browser Applications Web Browser Applications Web Browser Applications Web Browser Applications Web Browser Applications Web Browser Applications Web Browser Applications Web Browser Applications QA/QC CBIBS buoys Cache Metadata Mobile Sources CBOS Sensors NERRS CDMO Business Logic Applications Business Logic Business Logic National Estuarine Research Reserve National Weather Service T The system resides on a ‘virtual’ server in a 24X7 offsite facility Web Services Interfaces DBASE Storage Data Retrieval Handler DBASE Client Data Request Handler DBASE . . . DBASE DBASE Web Services Interfaces External Data Sources

5. Applying the Framework to other Environments • The framework is applicable to any environment that accesses multiple data sources and services multiple data users • It is supported by the NOAA Chesapeake Bay Office primarily in support of education and constituent projects (e.g. Nauticus Buoy Kiosk in Norfolk) • Attempts at leveraging the technology into existing IT environments has been (sometimes) problematic • Personnel unfamiliar with current technology • New technologies introduce risk that has not been evaluated • The ‘good news’ is that once these technologies are reviewed and the organization’s IT staff is comfortable, the door is opened to a greatly expanded set of services and capabilities

6. Example of ‘Constituent’ use of the CBIBS Data Management Framework T (Nauticus Information Kiosk)

7. End • Questions

8. Component Details….Sensor Data Acquisition and Storage Data Archive Handler QA/QC Transforms Metadata T The Data Archive Handler is responsible for archiving the data received from the observing system sensors (or other sources) to the appropriate local CBOS/CBIBS data storage facility (Buoys, sensors, meters, etc) The DAH - QA/QC module performs the necessary quality checks on the incoming data, and generates the appropriate system messages in the case of anomalies. TCP/IP XML Data Source 1 System DB Data & Info Storage Data Source 2 System DB Data Source 3 System DB (To DMS Framework) * * System DB Data Source n The DAH - Transformation module performs any data modifications required for the newly acquired ‘raw’ data The Metadata module contains ‘data about the data’’, e.g. sensor type, calibration history, and other information that gives the user confidence in the validity of the data contained in the archive

9. Component Details…..Data Retrieval Handler and Business Logic Business Logic Business Logic Business Logic T NDBC (as required) Data Retrieval Handler Construct, send, receive OpenDap commands and data to and from NDBC OpenDap Connector Data Communications Interface Manages access to the server’s local near-term archive to satisfy user service request Data Storage (from Data Archive Handler) NT Archive Connector Manages access and retrieval of data from external sources, e.g. NWS, NERRS. Services connector XML This function represents ‘Push’ based feed from external data or information source, e.g. NWS, NERRS External Data Sources The Data Retrieval Handler locates and retrieves data and products requested by the internet based ‘front end’ function. It also initiates business logic processing for either internal or external applications supporting value added data product development for target clients. Models, procedures, functions, performing value added services on demand as required to fulfill user needs

10. Server application Web Browser Applications IVR application Web Browser Applications Web Browser Applications Web Browser Applications Web Browser Applications Web Browser Applications Web Browser Applications Web Browser Applications Web Browser Applications Web Browser Applications Component Details….Client Data Request Handler and ‘front-end’ Web Services interface The Client Data Request Handler manages all connections and data requests from client (internet browser) applications. Public Internet Client Data Request Handler Direct Access Mgr XML Data Comm interface Browser applications use API key for access to framework data and/or services XML (to the Data Retrieval Handler) SQS (as required) The system first checks the Near Term Archive for any data requested. If not found, system reformats request as necessary to retrieve the data from the appropriate source The Near Term Archive is a queue which ‘pushes’ older data out in lieu of data which is in more frequent demand Note: Provision has been made to accommodate the Amazon Simple Cue Service if and when the need arises to ‘load balance’ service requests from the connected browser application(s)