A Services Oriented Architecture for Water Resources Data

1. A Services Oriented Architecture for Water Resources Data David R. Maidment Center for Research in Water Resources University of Texas at Austin EPA Storet Conference Austin Tx, Nov 28, 2007

2. Collaborators • University of Texas at Austin • Tim Whiteaker, Stephanie Johnson • San Diego Supercomputer Center • Ilya Zaslavsky, David Valentine, Tom Whitenack • Utah State University • David Tarboton, Jeff Horsburgh, Kim Schreuders • Drexel University • Michael Piasecki, Bora Beran, Yoori Choi • University of South Carolina • Jon Goodall

3. A Services Oriented Architecture for Water Resources Data • CUAHSI and WATERS • CUAHSI water web services • WaterML and WQX • Web Services for TMDL

4. A Services Oriented Architecture for Water Resources Data • CUAHSI and WATERS • CUAHSI water web services • WaterML and WQX • Web Services for TMDL

5. What is CUAHSI? UCAR • CUAHSI –Consortium of Universities for the Advancement of Hydrologic Science, Inc • Formed in 2001 as a legal entity • Program office in Washington (5 staff) • NSF supports CUAHSI to develop infrastructure and services to advance hydrologic science in US universities Unidata Atmospheric Sciences Earth Sciences Ocean Sciences CUAHSI National Science Foundation Geosciences Directorate HIS

6. CUAHSI Member Institutions 115 US Universities as of November 2007

7. HIS Team and its Cyberinfrastructure Partners Government: USGS, EPA, NCDC Industry: ESRI, Kisters, Microsoft CUAHSI HIS HIS Team: Texas, SDSC, Utah, Drexel, Duke Domain Sciences: Unidata, NCAR LTER, CZEN GEON Super Computer Centers: NCSA, TACC

8. Waters Network Testbed Sites

9. WATERS Network Information System HIS Team WATERS Testbed NSF has funded work at 11 testbed sites, each with its own science agenda. HIS supplies the common information system

10. Hydrologic Information Server Deployment Provides access to 1246 sites in 16 observation networks National Hydrologic Information Server San Diego Supercomputer Center metadata for national datasets: NWIS, Storet, Snotel WATERS testbed server

11. GetSites GetSiteInfo GetVariables GetVariableInfo GetValues Hydrologic Information Server WaterOneFlow services DASH – data access system for hydrology ArcGISServer Geospatial Data Observations Data Microsoft SQLServer Relational Database

12. Definition The CUAHSI Hydrologic Information System (HIS) is a geographically distributed network of data sources and functions that are integrated using a web services architecture so that they operate as a connected whole.

13. A Services Oriented Architecture for Water Resources Data • CUAHSI and WATERS • CUAHSI water web services • WaterML and WQX • Web Services for TMDL

14. Point Observations Information Model http://www.cuahsi.org/his/webservices.html Utah State Univ Data Source GetSites Little Bear River Network GetSiteInfo Little Bear River at Mendon Rd Sites GetVariables GetVariableInfo Dissolved Oxygen Variables GetValues 9.78 mg/L, 1 October 2007, 6PM Values {Value, Time, Qualifier, Offset} • A data source operates an observation network • A network is a set of observation sites • A site is a point location where one or more variables are measured • A variable is a property describing the flow or quality of water • A value is an observation of a variable at a particular time • A qualifier is a symbol that provides additional information about the value • An offset allows specification of measurements at various depths in water

15. CUAHSI Observations Data Model http://www.cuahsi.org/his/odm.html

16. Locations Variable Codes Date Ranges WaterML and WaterOneFlow STORET Data GetSiteInfo GetVariableInfo GetValues Data NAM NWIS WaterML Data WaterOneFlow Web Service Data Repositories Client EXTRACT TRANSFORM LOAD WaterML is an XML language for communicating water data WaterOneFlow is a set of web services based on WaterML

17. Set of query functions Returns data in WaterML WaterOneFlow Ilya Zaslavsky and David Valentine, SDSC

18. Syntactic mediation Heterogeneity of format Use WaterML to get data into the same format Semantic mediation Heterogeneity of meaning Each water data source uses its own vocabulary Match these up with a common controlled vocabulary Make standard scientific data queries and have these automatically translated into specific queries on each data source Data Heterogeneity

19. request return return request NAWQA request return return request NAM-12 request return NWIS request return request return return request NARR Objective • Search multiple heterogeneous data sources simultaneously regardless of semantic or structural differences between them What we used to do ….. Michael Piasecki Drexel University

20. NAWQA NWIS NARR HODM Semantic Mediator What we are doing now ….. GetValues GetValues GetValues GetValues generic request GetValues GetValues Michael Piasecki Drexel University GetValues GetValues

21. Hydroseekhttp://www.hydroseek.org Bora Beran, Drexel Supports search by location and type of data across multiple observation networks including NWIS and Storet

22. HydroTagger Ontology: A hierarchy of concepts Each Variable in your data is connected to a corresponding Concept

23. HIS to Google Earthdeveloped by Peter Fitch, CSIRO, Australia http://www.watersnet.org/wtbs/ODMKMLGatway.html A web application housed in Canberra, Australia, that operates over the WATERS Network data services

24. A Services Oriented Architecture for Water Resources Data • CUAHSI and WATERS • CUAHSI water web services • WaterML and WQX • Web Services for TMDL

26. The biggest difference is that the ODM concept of a time series data value (orange line) is derived from two sources: Field activity, and Analysis. This design is to simplify data management

27. SiteInfo Series GetSites GetSiteInfo Site and Series are captured in the WQX model. An activity can produce multiple results. This differs from the ODM model, where a series contains the results.

28. SiteInfo Series DataValues Method LabMethod GetValues GetValues would access all information (by design). Results captures multiple ODM concepts: DataValues, Method, and LabMethod

29. WaterOneFlow WQX Comparison

30. A Services Oriented Architecture for Water Resources Data • CUAHSI and WATERS • CUAHSI water web services • WaterML and WQX • Web Services for TMDL

31. Data Access System for Hydrology (DASH) Web service to TCEQ Mission River (Water Quality Management Segment 2004) DASH map interface to associate gaging station with river segment USGS Flow Gaging Station at Refugio Texas Web service to USGS

32. Web Services Spreadsheet for Load Duration Curves Web service for bacterial data Web service for flow data

33. Flow Duration Curve cfs Mission River at Refugio, Tx derived from USGS web service

34. Load Duration Curve (Fecal Coliform) Coliform Units per day TCEQ staff comment: “This tool produces $20,000 worth of graphs in 5 minutes”

35. Conclusion: Web services work! The CUAHSI Hydrologic Information System (HIS) is a geographically distributed network of hydrologic data sources and functions that are integrated using a web services architecture so that they function as a connected whole. For more information: http://www.cuahsi.org/his.html