140 likes | 270 Views
Managing Monitoring Data from Many Sources A New Hampshire Experience. Deb Soule Watershed Management Bureau New Hampshire Department of Environmental Services May 9, 2006. Items to be covered. Background on NHDES monitoring data
E N D
Managing Monitoring Data from Many Sources A New Hampshire Experience Deb Soule Watershed Management Bureau New Hampshire Department of Environmental Services May 9, 2006
Items to be covered • Background on NHDES monitoring data • Development of our own database based on EPA’s STOrage and RETrieval (STORET) database • Web access to database • Electronic data submittal process • Next steps - Using the Water Quality Exchange Schema
Prior to 2003, how would you obtain NHDES water monitoring data? • Be on the inside track - know someone who knew where the data was • Sort through paper lab data in file cabinets • Weed through disks and databases of various formats
We looked for a better solution. • Reviewed commercial databases (we’re too cheap) • Asked other states if we could copy their database (cheap option but not fruitful) • Reviewed STORET (didn’t fit all our needs) • Formed a department wide committee to develop a database (couldn’t get consensus) • Began development of our own database in the bureau
Database development process • Bureau met every 2 weeks for several months to determine needs/wants • STORET structure and requirements were used as guidance • Organizations/Programs/Projects/Stations/Activities • Program needs were incorporated • Business plan was developed
Core database is developed • Called the Environmental Monitoring Database (EMD), it was built: • by one developer in-house in 4 months • by a total design/development team of 3 people: business analyst, developer, and database administrator • using Oracle forms 6i with Oracle 8i for the back end • while working on other databases/projects • Imported almost all old bureau data (in electronic format) • Has automatic nightly imports of DES State Lab and bureau’s Limnology Lab data
EMD facts • Went live March 2003 • Accepts both field and lab data for air, water, soil, product etc. • Accepts QA/QC samples and automated data logger files • Handles physical/chemical data – biological later • Has over 110 projects, 15,000 stations, 241,000 activities, and 1,870,000 results • Contains 64 screens and 48 tables (for main part of database only)
EMD features • Contains modules to track complaints, beach inspections and shellfish sanitary surveys • Fulfills beach reporting requirements (both monitoring and advisory) • Can be linked to the Assessment Database (ADB) and is used as the basis for developing automated assessments • Can easily create STORET Import Module (SIM) compatible export files for upload to STORET
The EMD expands within NHDES • Other bureaus/divisions contribute data • Superfund • Site Remediation • Geology Unit • Wastewater • Developed direct links between their databases and the EMD • Developed department wide committee to collectively create changes to the database
The EMD beyond NHDES • Outside agencies/groups contribute data • University of NH • Several volunteer monitoring groups • Working with other agencies to bring in their data • We enhance/standardize their data to meet database requirements • Will upload to STORET for them free of charge • Why do we do this? We want/need data for assessments and the EMD is the basis for assessment data.
EMD data available on the web • Went live June 2004 • Only “final” data is available • Can query by: Organization Project Name Station ID Station Type Town County State Waterbody Name River Name Designated River HUC 12 Name Analyte Medium • Data returned via email in Excel/pipe delimited file complete with your query parameters • Data available via: http://www2.des.state.nh.us/OneStop/Environmental_Monitoring_Query.aspx
Electronic data submittal to the EMD via the web • Needed a way to receive standardized monitoring data electronically from consultants, outside labs, volunteers, cooperating agencies etc. • Developed Excel spreadsheets to import station and activity data separately (with XML planned for the future). • Developed web registration and submittal forms with data validation tool. • Developed interim table to capture what was being submitted, by whom, when etc.
Next steps • Update EMD to Environmental Sampling, Analysis and Results (ESAR) data standards • Report data to EPA via Water Quality Data Exchange (WQX) • Eliminate local copy of STORET • Add biological component to EMD • eventually eliminate local copy of Ecological Data Application System (EDAS) • Update web site to include: • graphing capabilities • geographic queries
Questions? Contact Information: Deb Soule Business Systems Analyst Watershed Management Bureau New Hampshire Department of Environmental Services dsoule@des.state.nh.us (603) 271-8863