Hydrologic Information System for the Nation

1. Hydrologic Information System for the Nation I. Zaslavsky (SDSC) & The CUAHSI HIS Project his.cuahsi.org, hiscentral.cuahsi.org

2. Consortium of Universities for the Advancement of Hydrologic Science, Inc. 122 US Universities as of July 2008 An organization representing more than one hundred United States universities, receives support from the National Science Foundation to develop infrastructure and services for the advancement of hydrologic science and education in the U.S. http://www.cuahsi.org/

3. CUAHSI Hydrologic Information System Models Databases Analysis Goal: Enhance hydrologic science by facilitating user access to more and better data for testing hypotheses and analyzing processes • Advancement of water science is critically dependent on integration of water information • Querying nation’s repository of water data • Linking small integrated research sites (<100 km2) with global and continental models • Integrating data from multiple disciplines to understand controls on hydrologic cycle • It is as important to represent hydrologic environments precisely with data as it is to represent hydrologic processes with equations Water quantity and quality Rainfall & Snow Soil water Meteorology Remote sensing

4. Map for the US Build a common window on water data using web services Observation Stations Ameriflux Towers (NASA & DOE) NOAA Automated Surface Observing System USGS National Water Information System NOAA Climate Reference Network

5. Water Data Web Sites

6. NWISWeb site output # agency_cd Agency Code # site_no USGS station number # dv_dt date of daily mean streamflow # dv_va daily mean streamflow value, in cubic-feet per-second # dv_cd daily mean streamflow value qualification code # # Sites in this file include: # USGS 02087500 NEUSE RIVER NEAR CLAYTON, NC # agency_cd site_no dv_dt dv_va dv_cd USGS 02087500 2003-09-01 1190 USGS 02087500 2003-09-02 649 USGS 02087500 2003-09-03 525 USGS 02087500 2003-09-04 486 USGS 02087500 2003-09-05 733 USGS 02087500 2003-09-06 585 USGS 02087500 2003-09-07 485 USGS 02087500 2003-09-08 463 USGS 02087500 2003-09-09 673 USGS 02087500 2003-09-10 517 USGS 02087500 2003-09-11 454 Time series of streamflow at a gaging station USGS has committed to supporting CUAHSI’s GetValues function

7. Point Observations Information Model USGS Data Source Return network information, and variable information within the network Streamflow gages Network Return site information, including a series catalog of variables measured at a site with their periods of record Neuse River near Clayton, NC Sites ObservationSeries Discharge, stage, start, end (Daily or instantaneous) Return time series of values Values 206 cfs, 13 August 2006 {Value, Time, Qualifier} • A data source operates an observation network • A network is a set of observation sites • A site is a point location where one or more variables are measured • A variable is a property describing the flow or quality of water • An observation series is an array of observations at a given site, for a given variable, with start time and end time • A value is an observation of a variable at a particular time • A qualifier is a symbol that provides additional information about the value

8. CUAHSI Observations Data Model http://his.cuahsi.org/odmdatabases.html

9. WaterML design principles Driven largely by hydrologists; the goal is to capture semantics of hydrologic observations discovery and retrieval Relies to a large extent on the information model as in ODM (Observations Data Model), and terms are aligned as much as possible Several community reviews since 2005 Driven by data served by USGS NWIS, EPA STORET, multiple individual PI-collected observations Is no more than an exchange schema for CUAHSI web services A fairly simple and rigid schema tuned to the current implementation; the least barrier for adoption by hydrologists Conformance with OGC specs not in the initial scope – but working with OGC on this (OGC Discussion Paper 07-041)

10. Set of query functions Returns data in WaterML Water Data Services NWIS Daily Values (discharge), NWIS Ground Water, NWIS Unit Values (real time), NWIS Instantaneous Irregular Data, EPA STORET, NCDC ASOS, DAYMET, MODIS, NAM12K, USDA SNOTEL, ODM (multiple sites)

11. Hydrologic Information System Service Oriented Architecture HIS Lite Servers Test bed HIS Servers Central HIS servers External data providers Global search (Hydroseek) Deployment to test beds Customizable web interface (DASH) Other popular online clients HTML - XML Desktop clients Data publishing HIS CentralRegistry & Harvester WaterOneFlow Web Services, WaterML WSDL - SOAP Ontology ETL services Controlled vocabularies Metadatacatalogs ArcGIS WSDL and ODM registration Matlab IDL, R Ontology tagging (Hydrotagger) MapWindow ODM DataLoader Excel Streaming Data Loading Programming (Fortran, C, VB) ODMTools Modeling (OpenMI) Server config tools

12. 6 5 4 2 3 1 WORKGROUP HIS SERVER ORGANIZATION STEPS FOR REGISTERING OBSERVATION DATA DASH Web Application Web Configuration file Stores information about registered networks MXD Stores information about layers Layer info,symbology, etc. WSDLs, web service URLs Connectionstrings Spatial store WOF services NWIS-IID points NWIS-IID WS USGS SQL Server NWIS-DV points NWIS-DV WS NWIS-IID NCDC ASOS points ASOS WS NWIS-DV STORET points STORET WS ASOS EPA TCEQ points TCEQ WS STORET BearRiver points BearRiver WS TCEQ TCEQ . . . . . . More WS fromODM-WS template More synced layers BearRiver My new points My new WS . . . More databases Background layers(can be in the same or separate spatial store) Geodatabase or collection of shapefilesor both Web services from a common template My new ODM ODMs and catalogs. All instances exposed as ODM (i.e. have standard ODM tables or views: Sites, Variables, SeriesCatalog, etc.) ODMDataLoader

13. HISCentral Services Catalog

15. Against the NIH Syndrome 2006: CUAHSI HIS web services are discussed on the BASINS mailing list as a new way to access hydrologic data. The list is mostly used by hydrologists and developers outside academia; NCDC develops ASOS web services following WaterML 2007: MOU with USGS; USGS is developing WaterML-compliant GetValues service; GLEON uses an early version of ODM to develop their own database schema (VEGA); Phoenix LTER is developing ODM (in MySQL) and WaterML web services (in Java); A Google Earth-based client for CUAHSI web services is developed at CSIRO, Australia; Deployment to 11 hydrologic observatory test beds, + CBEO (CEOP project) 2008: KISTERS develops WaterML-compliant web services over their database, for a client; MapWindow open source GIS develops WaterOneFlow parsers; Florida, Texas and Idaho use ODM and WaterOneFlow web services to provide access to state data repositories; New Jersey is considering the same; Another CEOP project, at UC-Davis, is implementing ODM (in Postgres) and web services (in Java); More, which we don’t know about…

16. US Map of USGS Observations Alaska Puerto Rico Hawaii Antarctica

17. Different types of nutrients by decade: Available Data Total

18. Some physical properties by decade: Available Data Total

19. Hydroseekhttp://www.hydroseek.net Supports search by location and type of data across multiple observation networks including NWIS, Storet, and academic data

20. Semantic Tagging of Harvested Variables

21. HIS Deployment National Hydrologic Information Server San Diego Supercomputer Center • 11 Hydrologic Observatory test bed projects (NSF-funded) Growing number of water data services (LTER, CEO:P, elsewhere) WaterML Adoption (USGS, NCDC)

22. Water Quality in Moreton Bay, Brisbane, Australia (Jane Hunter)

23. Accomplishments • Generic method for managing and publishing observational data • Supports many types of point observational data • Overcomes syntactic and semantic heterogeneity using a standard data model and controlled vocabularies • Supports a national network of observatory test beds but can grow! • WaterML is a common language for water observations data from academic and government sources. A Hydrology Domain Working Group is established at OGC. • Point Observations Data from Agencies and Academic Investigators can be consistently communicated using web services • National Water Metadata Catalog is the most comprehensive index of the nation’s water observations presently existing

24. HIS Overview Report http://his.cuahsi.org/documents/HISOverview.pdf Summarizes the conceptual framework, methodology, and application tools for HIS version 1.1 Shows how to develop and publish a CUAHSI Water Data Service Available at: