1 / 12

Operational Deployment of Datasets

Operational Deployment of Datasets. ATS 690 Jason Burks Holly Allen. Introduction. The objective is to show the steps taken to migrate a dataset from a research facility to a weather service operational environment. Knowledge gained will be beneficial to both the NWS and NASA SPoRT group.

clune
Download Presentation

Operational Deployment of Datasets

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Operational Deployment of Datasets ATS 690 Jason Burks Holly Allen

  2. Introduction • The objective is to show the steps taken to migrate a dataset from a research facility to a weather service operational environment. • Knowledge gained will be beneficial to both the NWS and NASA SPoRT group.

  3. Steps Needed • Identifying Dataset for Operational Use • Display and Data Type Implementation Considerations • Setup of Data Ingest • Data Reliability • Testing of Data • Training for Use of Dataset • Use and Evaluation of the Dataset • Application of the Process to Multiple Offices

  4. Identify Dataset for Operational Use • Finding Datasets • Dropped in lap • Have a Problem? Find a solution with a new dataset • File Size is Considered • How much bandwidth will it require? • Bandwidth creates ingest limitations • EX: ADAS datasets were pared down because of their size. • How much data can AWIPS hold?

  5. Display and Data Type Implementation Considerations • How to Display dataset in AWIPS? • What kind of data is it? • Categorized in terms of how the data will be displayed • Text, Point (obs, profiler), Satellite, RADAR, Gridded (model, LMA) • Does the data make sense? If so,… • Create a color curve, contouring intervals, temporal resolution, projection information, etc. • Add data to menus and volume browser. • Becomes an experimental dataset. • If proven needed, then determine how to get the data.

  6. Setup of Data Ingest • How do we get the data? • FTP - push the data to a FTP site and go get it. • LDM - places data in queue. Goes to whoever requests it. • From the LDAD: AWIPS LDAD Regional HQ /Data/Incoming Directory Moved into a local file, where it is stored. Purged off AWIPS. Firewall A script runs from AWIPS to get files from the incoming directory on LDAD.

  7. Data Reliability • Once the data is ingested and displayable, is it reliable? • Questions to ask in order to determine if its ready for the next step: • Is it timely? • Does the data look realistic? • High quantity of erroneous data? • Is data consistently available?

  8. Testing of Data • It is hard to overcome a ‘tainted mindset’ with forecasters when they are given a bad dataset. • Datasets should be tested and quality controlled before they are viewed by forecasters. • Normally, for a dataset to be implemented, an advocate for that dataset is needed. • The advocate is not a defined person, but usually someone who takes an interest in the new data, and encourages others to look at and use it.

  9. Training for Use of Dataset • Normally is done by the Advocate or the Researcher that supplied the dataset. • Researchers might not know how the dataset will be applied operationally, so contact with the forecaster(s) is necessary. • Contact with the Researcher is maintained in order to provide information about data outages, server problems, etc.

  10. Use and Evaluation of the Dataset • Qualitative • Immediate feedback (i.e., surveys) • Useful in identifying case studies • Quantitative • Case study analysis. • Shows impact of data: • Was lead time or false alarm ratio improved? • In order to bring a new dataset into full operational use, there must be hard facts and numbers to back up the utility of the dataset. • Not: “Oh, they thought it was great.”

  11. Application of the Process to Multiple Offices • Color Curves • Better way to transfer between offices. • Recommend for future AWIPS builds an export/import option. • Compatibility issues between different servers. • Sites with upgrades or builds vs. those not. • Allocate data key for each product • Details for each product is kept under that key. • A way to apply data without getting messed up during each build process. • The extent of the dataset; if it is too large, can make it difficult to send. • After dataset is provided to new offices, information on how to use/modify/troubleshoot data must also be provided. • Written documentation. Ex: If this occurs, then check here, etc. • The process for implementation at the new office begins with training, whether it is done by someone in-house, someone from the previous office, or the initial research team.

  12. Summary • Identifying Dataset for Operational Use • Display and Data Type Implementation Considerations • Setup of Data Ingest • Data Reliability • Testing of Data • Training for Use of Dataset • Use and Evaluation of the Dataset • Application of the Process to Multiple Offices • These steps should be done in order to provide a successful transfer from the research facility to operations.

More Related