1 / 21

Where does VIPER get its data from? Tom Pagano tom.pagano@porda 503-414-3010

Where does VIPER get its data from? Tom Pagano tom.pagano@por.usda.gov 503-414-3010. What data do we need? VIPER’s input and storage of data Old and new databases AWDBWYFORCDBSSNOTEL Real-time data schedule Local file import. What data do we need?. Metadata

kort
Download Presentation

Where does VIPER get its data from? Tom Pagano tom.pagano@porda 503-414-3010

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Where does VIPER get its data from? Tom Pagano tom.pagano@por.usda.gov 503-414-3010

  2. What data do we need? VIPER’s input and storage of data Old and new databases AWDBWYFORCDBSSNOTEL Real-time data schedule Local file import

  3. What data do we need? Metadata Station lists, lat/lon/elev Historical time series data Real-time time series data Snowpack, Precipitation, etc Forecast equations, configurations

  4. VIPER Workbook Station Metadata Equations/ Configurations 1 station POR Historical Time-Series Real-time Data All stations 2 years

  5. VIPER Workbook AWDB Station Metadata Equations/ Configurations Metadata Data 1 station POR Historical Time-Series Real-time Data All stations 2 years

  6. VIPER Workbook AWDB Station Metadata Equations/ Configurations Metadata Data 1 station POR Historical Time-Series Real-time Data All stations 2 years USGS webpage

  7. Old databases and new databases Old time series databases • CDBS • “Final” data • Period of record • ~1 year out of date • Daily, monthly • Snow, Precip, Temp • Resv, Strm

  8. Old databases and new databases Old time series databases • CDBS • “Final” data • Period of record • ~1 year out of date • Daily, monthly • Snow, Precip, Temp • Resv, Strm • SNOTEL • Provisional • Recent 2 years • Hourly (and daily) • All snotel elements • Archive kept (since ?)

  9. Old databases and new databases Old time series databases • CDBS • “Final” data • Period of record • ~1 year out of date • Daily, monthly • Snow, Precip, Temp • Resv, Strm • SNOTEL • Provisional • Recent 2 years • Hourly (and daily) • All snotel elements • Archive kept (since ?) • WYFOR • Ephemeral • Recent 2 years • Monthly • Snow, Prec, Strm • Resv, “Misc” • Archive kept (since ‘93)

  10. Old databases and new databases Old time series databases • CDBS • “Final” data • Period of record • ~1 year out of date • Daily, monthly • Snow, Precip, Temp • Resv, Strm • SNOTEL • Provisional • Recent 2 years • Hourly (and daily) • All snotel elements • Archive kept (since ?) • WYFOR • Ephemeral • Recent 2 years • Monthly • Snow, Prec, Strm • Resv, “Misc” • Archive kept (since ‘93) New time series database • AWDB • Provisional and final mixed • All years • All durations • All elements* *Planned NOTE: AWDB load strategy being reworked so rest of slides may not apply in near future!

  11. Example- where do historical monthly snow come from? 1960 Brumley Snotel Back estimated snowcourse 1970 Site automated (aug 1980) 1980 1990 2000

  12. Example- where do historical monthly snow come from? 1960 1x load of cdbs… updated if Dana notified of changes 1x load daily data in off season into monthly slots (snotel.. Or daily cdbs?) CDBS 1970 1980 SNOTEL 1990 2000

  13. Historical streamflow data 1960 1970 1980 1990 2000

  14. CDBS Wyfor “under” CDBS Historical streamflow data 1960 1x load of old wyfor files since mid 1990’s Then on top of that… 1x load of cdbs… updated if Dana notified of changes 1970 1980 1990 2000

  15. The realtime data schedule and synchronization AWDB database Wyfor SNOTEL 4x a day, complete load (1:20am, 5:20am, 11:20am, 10:20pm)

  16. The realtime data schedule and synchronization AWDB database Wyfor SNOTEL DMP Edits Any DMP edits to snotel loaded every 5 minutes to wyfor/awdb

  17. The realtime data schedule and synchronization AWDB database Wyfor SNOTEL CSV processor CSV spreadsheet files (non-snotel data) Spreadsheet files loaded every 10 minutes all day to wyfor/awdb

  18. The realtime data schedule and synchronization AWDB database Wyfor SNOTEL Manual edits Direct edits of wyfor reach wyfor right away But reach AWDB 50 minutes after the hour 5-9am, 11am-4pm and 11pm. Worstmethod of entering data for viper.

  19. If you have custom datasets, Viper can also access local CSV format files. Same layout as individual station data sheets (e.g. data1, data2) Realtime data gets manually entered in “ExternalData” sheet.

  20. OMG Viper is so slow!!!! Viper is a network-hungry application. Within the USDA backbone, your connection should be very fast. Outside the backbone, speed varies from tolerable to disgruntling.

  21. OMG Viper is so slow!!!! Viper is a network-hungry application. Within the USDA backbone, your connection should be very fast. Outside the backbone, speed varies from tolerable to disgruntling. The way viper connects to the database is the same way that users contact the new update report…. which can mean:

More Related