1 / 46

Water Information Management Systems Workshop Missoula, MT Sept 7, 2005 Dave Briar

National Water Information System -- Web Interface Operation and Quality Assurance. Water Information Management Systems Workshop Missoula, MT Sept 7, 2005 Dave Briar. Topics:. Architecture reflects history and USGS culture – local control Hardening the system for 24 x 365

maralah
Download Presentation

Water Information Management Systems Workshop Missoula, MT Sept 7, 2005 Dave Briar

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. National Water Information System -- Web Interface Operation and Quality Assurance Water Information Management Systems WorkshopMissoula, MT Sept 7, 2005Dave Briar

  2. Topics: • Architecture reflects history and USGS culture – local control • Hardening the system for 24 x 365 • Quality-assurance tools – existing and needed

  3. -- NWIS database installation

  4. Secure Repository Public “View” 48 NWIS hosts http://waterdata.usgs.gov/

  5. Secure Repository Public “View” 48 NWIS hosts http://waterdata.usgs.gov/ NationalAggregateDataset Total sitesReal-time sitesDaily valuesGround-water levelsQW samplesQW valuesPeaks 1,615,003 9,785225,606,560 7,820,464 4,376,679 69,736,624 662,227

  6. -- NWIS database installation -- NatWeb Node with Firewall

  7. NWIS – NatWeb Delivery NatWeb SD Sioux Falls NatWeb CA Menlo Park Three redundant copies of all real-time data for all sites nationwide NatWeb VA Reston NAD Single copy of all data (real-time and discrete / historical) for all sites nationwide

  8. LRGS WSCDatabase

  9. NWIS – NatWeb Data Transfer LRGS WSC Database NatWeb SD NatWeb VA NatWeb CA HUB1 NAD

  10. NWIS – NatWeb Data Transfer NatWeb SD RT data NatWeb VA RT data RT data NatWeb CA HUB1 NAD All data

  11. NWIS – NatWeb Data Transfer LRGS NatWeb SD HUB2 NatWebVA NatWeb CA HUB1 NAD

  12. NWIS – NatWeb Data Transfer NatWeb SD RT data HUB2 RT data RT data NatWebVA NatWeb CA All data NAD

  13. NWIS – NatWeb Data Transfer NatWeb SD RT data HUB2 RT data RT data NatWeb VA NatWeb CA All data NAD

  14. NWIS – NatWeb Concurrency waterdata.usgs.gov/zz/nwis/rt waterdata.usgs.gov/zz/nwis/rt NAD

  15. NWIS – NatWeb Concurrency waterdata.usgs.gov/zz/nwis/rt NAD

  16. NWIS – NatWeb Concurrency waterdata.usgs.gov/zz/nwis/rt 01646500 POTOMAC RIVER NR WASH, DC LITTLE FALLS Latency = 11 sec naddb01 [2004-06-07 10:15:00 4.82] 2004-06-07 11:36:43 cadb02 [2004-06-07 10:15:00 4.82] 2004-06-07 11:36:36 sdww02 [2004-06-07 10:15:00 4.82] 2004-06-07 11:36:42 vadb02 [2004-06-07 10:15:00 4.82] 2004-06-07 11:36:32 NAD

  17. WSC Database to Web Latency District Database NatWeb SD NatWeb VA NatWeb CA HUB1 NAD

  18. WSC Database to Web Latency Seconds Percent of Sites

  19. NWIS – NatWeb Efficiency May 2005 • 12.5 GB Data transferred from the 48 NWIS Databases to the National system-- includes 4.35 million real-time site updates

  20. NWIS – NatWeb Efficiency May 2005 • 12.5 GB Data transferred from the 48 NWIS Databases to the National system-- includes 4.35 million real-time site updates • 273.9 GB Data delivered to primarily external users by the 4 NatWeb nodes-- 22.9 million requests

  21. WSC Database is single point of failure WSCDatabase NatWeb SD NatWeb VA NatWeb CA HUB1 NAD

  22. LRGS WSCDatabase

  23. NWIS – NWIS-RT Backup Replication WSC Database NatWeb VA HUB1 NWIS-RT in “Hot Standby” NAD

  24. NWIS – NWIS-RT Backup NatWeb VA HUB1 NWIS-RT Serving WSC Data NAD

  25. Louisiana Data -- Hurricane Katrina

  26. Louisiana Data -- Hurricane Katrina

  27. Louisiana Data -- Hurricane Katrina

  28. 2004 Tropical Storm Tracks

  29. Quality Assurance: • Automated • Filters applied during processing • Erroneous values masked from public display and questionable values flagged

  30. Threshold Filters Very High Value ( Impossible Value -- Masked ) High Value ( Warning ) ( Warning ) Low Value Very Low Value ( Impossible Value -- Masked )

  31. Threshold Filters Very Rapid Increase Rapid Increase Very Rapid Decrease Rapid Decrease

  32. Quality Assurance: • Automated • Filters applied during processing • Erroneous values masked from public display and questionable values flagged • Daily “QA report” from National system

  33. Daily QA Report -- Results

  34. Quality Assurance: • Automated • Filters applied during processing • Erroneous values masked from public display and questionable values flagged • Daily “QA report” from National system • Periodic visual inspection • Lists of selected sites • Internal-only reference levels provide context for QA process

  35. Display of “Looked-At” Flag USGS 12345678 Hirsch Creek near Spikeville, VA GAGE HEIGHT – RAW UNREVIEWED GAGE HEIGHT –REVIEWED

  36. Questions? http://waterdata.usgs.gov/ dbriar@usgs.gov

More Related