1 / 14

Datzilla Usage NCDC Perspective

Datzilla Usage NCDC Perspective. Karsten Shein NOAA National Climatic Data Center Karsten.Shein@noaa.gov. NWS Sub-regional data stewardship workshop – Fargo, ND 12-13 May 2007. What is Datzilla?.

cleary
Download Presentation

Datzilla Usage NCDC Perspective

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Datzilla UsageNCDC Perspective Karsten Shein NOAA National Climatic Data Center Karsten.Shein@noaa.gov NWS Sub-regional data stewardship workshop – Fargo, ND 12-13 May 2007 http://www.ncdc.noaa.gov

  2. What is Datzilla? Datzilla is a Web-based tool to report and track errors in NOAA-held data, metadata, or data delivery systems. It is maintained by the Southern Regional Climate Center and NCDC is a participant to the extent of addressing data it holds and systems it maintains.

  3. Errors vs. Corrections • Datzilla intended to correct errors • Corrections to observations are not errors. • For changes to recent observations (common) • William Angel (COOP / ASOS) • Blake Lasher (LCD ASOS) • Send amended form to NCDC via normal channels. • Changes to historical observations (uncommon) • Amended forms • Datzilla (limited to changing estimates of invalid obs) • Must have a compelling reason to change a historical obs. • Corrections due to equipment problems may also require a metadata rendition.

  4. When NCDC receives a Datzilla ticket … • Datzilla gatekeeper • Initial determinations • Reassignment • Investigation • Course of action • Resolution • Closure Your friendly Datzilla Gatekeeper

  5. Some possible reasons for a Datzilla ticket to NCDC • Data issue (usually) • That TMAX of 74 should be a 47! • Your #$&!@* QC clobbered my data! • System issue • CDO inventory doesn’t match the data I got! • Metadata issue • This station’s COOP number is wrong!

  6. Good tickets and Bad • Pre-report investigation (critical) • Is the error really an error? • Is it the right station? • Is it reproducible (browsers, systems) • Do you have the right system? • ACIS, ThreadEx, NCDC data may be different • Have you proposed a solution?

  7. A good ticket • Clear title • Good description of problem • ID of station • Attached form • Proposed solution.

  8. A bad ticket • Vague title • Vague description of problem • No ID of specific station, element, date • No supporting material • No suggested fix • Hunches • Data “corrections”

  9. Investigating the error • xmACIS is a display tool for NOAA data • Interprets data flags from NCDC data sets • Prioritizes simultaneous values from different data sets • Displays “M” for invalid as well as missing data • May thread data from several stations • Primary, Secondary, Tertiary forms: • 1001, B-91, 10-A, … • 1014, LCD, CD, CRB • Media articles, eye-witness accounts, proxy data • Paper, WSSRD, IPS …

  10. My ticket’s been open forever! • Some are easy to fix • Small-batch keying errors • A few missing values • Some are not • May be part of a larger issue • May exceed NCDC resources • May not have a straightforward fix • May involve parties/systems outside NCDC

  11. Your #$&!@* QC clobbered my data! • You are right to alert us. • However, before you do … • Think about why our QC would flag that data. • Most often, the problem is not our QC • Observer date shifting • Observing at time different than expected. • Known climate anomalies? • Unreported temporary anomalies?

  12. Fuel for the Fire Should that 850° F TMAX really be invalidated?

  13. One NOAA • We’re all interested in having the best NOAA data possible. • Better data means more accurate forecasts and research. • NCDC has limits on what it can do to data. • We will address and work to fix all legitimate errors.

  14. Any questions? Karsten.Shein@noaa.gov 828.271.4223 Once again, your friendly (no, not that friendly) NCDC Datzilla Gatekeeper! NCDC Halloween, 2006

More Related