Water Quality Exchange WQX

1. Water Quality Exchange (WQX) Kristen Gunthardt Dwane Young US EPA Office of Water

2. What is WQX? WQX defines the framework by which EPA compiles water quality monitoring data (physical, chemical and biological) that are collected by a number of entities via a shared schema. The schema allows for anybody to share data regardless of what the original source of the data was. Framework because WQX is not a distributable system but rather a set of standard protocols through which WQ monitoring data can be shared The standard schema and protocols allow users to do water quality data management in the way they see best for their program while also allowing data sharing at a national level Mostly work with CWA agencies in supporting the monitoring data that is used for CWA decisions ~10 states are either flowing data to WQX or are poised to begin flowing data (VT, GA, WI, TX, RI, DE all have data pushed through, PA, ND, SD, MI, OR are either testing now or overcoming the mapping hurdle) eBeaches monitoring data is coming in through WQX � Public Health agencies that submit microbiological monitoring results (e coli, fecal coliform, interococci) in support of BEACH closure decisions Framework because WQX is not a distributable system but rather a set of standard protocols through which WQ monitoring data can be shared The standard schema and protocols allow users to do water quality data management in the way they see best for their program while also allowing data sharing at a national level Mostly work with CWA agencies in supporting the monitoring data that is used for CWA decisions ~10 states are either flowing data to WQX or are poised to begin flowing data (VT, GA, WI, TX, RI, DE all have data pushed through, PA, ND, SD, MI, OR are either testing now or overcoming the mapping hurdle) eBeaches monitoring data is coming in through WQX � Public Health agencies that submit microbiological monitoring results (e coli, fecal coliform, interococci) in support of BEACH closure decisions

3. The STORET Warehouse The purpose of the compilation of data in the STORET Data Warehouse is to provide a seamless collection of monitoring data that is not restricted by jurisdictional boundaries. Organizations � Who is collecting the data Projects � Why the data are being collected Stations � Where data are collected Activities/Results � How the data are collected/What the data is All data flowing in through WQX ends up in the STORET data warehouse Data warehouse up until WQX has been fed through oracle dmp file submissions Data from the local modernized STORET data systems are also still being supported for coming into the Warehouse until Sept 2009 Data are cross jurisdictional � allow for data retrieval for multiple organizations who have collected data within an area STORET data warehouse termsAll data flowing in through WQX ends up in the STORET data warehouse Data warehouse up until WQX has been fed through oracle dmp file submissions Data from the local modernized STORET data systems are also still being supported for coming into the Warehouse until Sept 2009 Data are cross jurisdictional � allow for data retrieval for multiple organizations who have collected data within an area STORET data warehouse terms

4. How does WQX work? This is the state or tribe node. This also represents the states/tribes/interstate commissions/territories local data management system This is the XML Generation Tool. Say that it is being developed and is meant for small data providers for whom having a �Node� is not an option. This is the Central Data Exchange (CDX). CDX does a schema well formedness check on submitted files. WQX does the remainder of validation checks on any submitted data. It checks to make sure that business rules are not being violated. It will load what it can, and provide processing reports (including errors) back to the data provider via CDX. It�s important to note that we have host of query and solicit request services that come off of the WQX database as a way for users to see what they�ve submitted prior to it being loaded to the STORET warehouse. All domains values are also available through these services. EPA is also developing a whole host of new methods for getting data out of the warehouse. One example is the Watershed Summary reports that we released earlier this year. Dwane will also be talking about the new web services we have from the warehouseThis is the state or tribe node. This also represents the states/tribes/interstate commissions/territories local data management system This is the XML Generation Tool. Say that it is being developed and is meant for small data providers for whom having a �Node� is not an option. This is the Central Data Exchange (CDX). CDX does a schema well formedness check on submitted files. WQX does the remainder of validation checks on any submitted data. It checks to make sure that business rules are not being violated. It will load what it can, and provide processing reports (including errors) back to the data provider via CDX. It�s important to note that we have host of query and solicit request services that come off of the WQX database as a way for users to see what they�ve submitted prior to it being loaded to the STORET warehouse. All domains values are also available through these services. EPA is also developing a whole host of new methods for getting data out of the warehouse. One example is the Watershed Summary reports that we released earlier this year. Dwane will also be talking about the new web services we have from the warehouse

5. We�ve been in production for over a year now with v1.0, the next slide gives more information about what data is flowed with the v1.0 schema For the remainder of my talk, I�ll be focusing on our newer activities, the first being our release of WQX v2.0 and our XML Generation ToolWe�ve been in production for over a year now with v1.0, the next slide gives more information about what data is flowed with the v1.0 schema For the remainder of my talk, I�ll be focusing on our newer activities, the first being our release of WQX v2.0 and our XML Generation Tool

6. WQX v1.0 The physical conditions in the environment at the time of a site visit. The chemical and bacteriological make-up of the water sampled. Chemical analyses of fish tissue collected. The v1.0 schema captures physical, chemical and fish tissue monitoring information The v1.0 schema is heavily based upon the ESAR data standard that grew out of efforts of the NWQMC methods board and the WQDE developed and approved Another area of standardization is our list of reportable chemical parameters or characteristics � the names align and are managed by the SRS (managed in that if a new name is proposed to be added, we use the SRS as a reference for including). Current schema documentation is available on EN websiteThe v1.0 schema captures physical, chemical and fish tissue monitoring information The v1.0 schema is heavily based upon the ESAR data standard that grew out of efforts of the NWQMC methods board and the WQDE developed and approved Another area of standardization is our list of reportable chemical parameters or characteristics � the names align and are managed by the SRS (managed in that if a new name is proposed to be added, we use the SRS as a reference for including). Current schema documentation is available on EN website

7. WQX v2.0 � What�s new? Biological Taxon Abundance data, including population census, frequency class, group summaries, and individual results Toxicity data Habitat Assessment scores and their related metric scores Biological Index scores and their related metric scores 1 additional web method for Query / Solicit No ESAR like data standard has been approved for bio or habitat data. No acceptance from governance structures for the �tox and pop� data elements by the methods board (include toxicity and biological population related data elements), and phab is still a work in progress Relied heavily on user community, other bio db experts, subject matter experts Schema, DET and FCD being reviewed by the NTG for acceptance � still awaiting word � once we have approval will be opening up for full testing The understanding of the physical habitat of an area allows for better assessments of the stream (or lake or coastal or wetland) ecosystem and human caused effects � challenges lie ahead for managing list of characteristics, levels of habitat data (raw measurements, calculations, metrics, idexes) Biological assessments are evaluations of the condition of waterbodies using surveys and other direct measurements of resident biological organisms (macroinvertebrates, fish, and plants) � challenges lie ahead in managing and keeping track of taxa names � biologists can�t even agree on names New web method allows for retrieval of index informationNo ESAR like data standard has been approved for bio or habitat data. No acceptance from governance structures for the �tox and pop� data elements by the methods board (include toxicity and biological population related data elements), and phab is still a work in progress Relied heavily on user community, other bio db experts, subject matter experts Schema, DET and FCD being reviewed by the NTG for acceptance � still awaiting word � once we have approval will be opening up for full testing The understanding of the physical habitat of an area allows for better assessments of the stream (or lake or coastal or wetland) ecosystem and human caused effects � challenges lie ahead for managing list of characteristics, levels of habitat data (raw measurements, calculations, metrics, idexes) Biological assessments are evaluations of the condition of waterbodies using surveys and other direct measurements of resident biological organisms (macroinvertebrates, fish, and plants) � challenges lie ahead in managing and keeping track of taxa names � biologists can�t even agree on names New web method allows for retrieval of index information

8. WQX v1.0 Considerations CDX and WQX backend will be supporting both versions of the schema The namespace of the root element <WQX> must distinguish between v1.0 and v2.0 Both schemas will be supported in CDX and the WQX backend processing software, so users just coming online with WQX v1.0 will be supported, and no reverse engineering of data mappings will need to occur � this mostly matters with fish tissue data and the changes made there between schemas Example of use of namespace to differentiate either submission typesBoth schemas will be supported in CDX and the WQX backend processing software, so users just coming online with WQX v1.0 will be supported, and no reverse engineering of data mappings will need to occur � this mostly matters with fish tissue data and the changes made there between schemas Example of use of namespace to differentiate either submission types

9. What is the XML Generation Tool? A web-based application that assists a data provider with the creation of a valid WQX submission file for the Water Quality Exchange. This tool is meant to allow those who do not have Exchange Network Nodes or who can�t generate the WQX XML yet to submit data using WQX

11. How the XML Generation Tool fits in

16. Web Resources and User Activities www.epa.gov/storet www.exchangenetwork.net/exchanges/water/wqx.htm www.epa.gov/storet/wqx.html Adopt all applicable data standards Encourage capacity with small data providers for participating with other data partners and submitting data through WQX Identify collaborative and innovative approaches for transitioning from local STORET to WQX Develop applications that use STORET Web Services for data analysis Visit the STORET website for additional user documentation. Information about getting started with WQX, additional documentation about the schema, business rules available as well as more information about the project Information about the WQX v2.0 schema is also available here These are some of the user activities that we have listed in Appendix B in the EN Grant solicitation � many of the new pieces to the WQX framework that we�re adding relate to these activitiesVisit the STORET website for additional user documentation. Information about getting started with WQX, additional documentation about the schema, business rules available as well as more information about the project Information about the WQX v2.0 schema is also available here These are some of the user activities that we have listed in Appendix B in the EN Grant solicitation � many of the new pieces to the WQX framework that we�re adding relate to these activities

18. USGS NWIS* & STORET Over Time

22. All services allow access to data without going through STORET webforms. Impact is that organizations can create their own applications to hit directly against the Warehouse.All services allow access to data without going through STORET webforms. Impact is that organizations can create their own applications to hit directly against the Warehouse.

23. An example using Monitoring Data

24. XML Google Earth (KML)