130 likes | 298 Views
CHPS Era Archive/Data Backup. John Halquist Development and Operations Officer National Operational Hydrologic Remote Sensing Center (NOHRSC) HIC Meeting – Feb 2010. Major categories for Archive use. General Archive requirements Model development Unitgraph Routing Calibration
E N D
CHPS Era Archive/Data Backup John Halquist Development and Operations Officer National Operational Hydrologic Remote Sensing Center (NOHRSC) HIC Meeting – Feb 2010
Major categories for Archive use • General Archive requirements • Model development • Unitgraph • Routing • Calibration • Verification • Analogs • Studies • Case studies • Applied Research • Modeling/Product improvement • Hindcasting/Reanalysis
Current situation NWS (field) data flat files (grids) acquisition ICP EVS SHEF SHEF RAX archive RDB SHEF-dec. raw SHEF-dec. IHFS(ops DB) proc.SHEF-dec. FEWS fc DB ofs-de SHEF SHEF NWSRFS MPE XSETS … NAVX products
Current situation (FEWS archive) FEWS • FEWS archive: • all data related to forecast bundled in one zip file • forecast and associated inputs can be retrieved in stand alone FEWS • data not directly accessible • Recommendation: intermediatesolution only(as short as possible) fc DB meta data RDB zip files archive-server
GG Verif Calb Enhanced or Additional FEWS Export functionality Could use SOAP or PI FEWS DB ? export External Sources Possible NEW import JAVA class Outbox (dir) IWRSS Archive Relational DB Archive Files
Key differences from existing Archive System • Data flows “through” the operational system to feed the Archives • eliminates Quality Control nightmare • minimizes multiple “decode” pathways • Provide for a more “Integrated” Archive • 2 way data flow to support usage in operations and analysis • Scalable, Flexible & Extensible architecture • hardware & database schemas
Gaps in current requirements: • Meta-data • What do we really need? • What do we want? • What about current archived data with little meta-data? • Modifiers in CHPS • Implemented significantly different than NWSRFS • Some decisions need to be made on how we want to handle modified time series
CHPS time series, example: original data timeseries modifiers time series used
Hardware modification: * Indicates New hardware
Schedule objectives: • Finalize conceptual design - Done • Determine Hardware requirements – Mar 2010 • Data requirements finalization – May/Jun 2010 • Build Prototype system – Sept 2010 • Test system built at NOHRSC • Operational field test at NWRFC • Provide initial DB Schema for review – Nov 2010 • Field Deployment - ~2012