National Weather Service River Flood Warning Verification

1. National Weather Service River Flood Warning Verification Mary Mullusky, Ernest Wells, Timothy Helble, and Thomas Graziano Hydrologic Services Division Office of Climate, Water, and Weather Services National Weather Service

2. Floods versus Flash Floods • River Flood • River rises to flood stage over many hours or days (> 6 hrs) • Longer lasting events • Based on forecasts for specific gaged locations, prepared by NWS hydrologists • Flash Flood • Quick responding streams (reach flood stage in < 6 hrs after the heavy rain started) • Warnings issued for defined areas based on hydrometeorological criteria and observer reports

3. NWS River Flood Warning VerificationBackground The NWS issues river forecasts, watches and warnings for over 4000 locations nationwide.

4. Flood Watch versus River Flood Warning WGUS42 KMHX 061600FLWMHX BULLETIN - IMMEDIATE BROADCAST REQUESTED FLOOD WARNINGNATIONAL WEATHER SERVICE NEWPORT/MOREHEAD CITY NC1200 PM EDT FRI APR 6 2012 ...THE NATIONAL WEATHER SERVICE IN NEWPORT HAS ISSUED A FLOOD WARNING FOR THE FOLLOWING RIVER IN NORTH CAROLINA... NEUSE RIVER AT KINSTON AFFECTING CRAVEN AND LENOIR COUNTIES. .HEAVY RAINFALL TOMORROW WILL CAUSE THE NEUSE RIVER TO RISE NEAR OR ABOVE FLOOD STAGE. SAFETY MESSAGE...NEVER DRIVE CARS...TRUCKS OR SPORT UTILITY VEHICLES THROUGH FLOODED AREAS. NCC049-107-070200-/O.NEW.KMHX.FL.W.0001.120407T1300Z-000000T0000Z//KINN7.1.ER.120407T1300Z.120412T0100Z.000000T0000Z.NO/1200 PM EDT FRI APR 6 2012 THE NATIONAL WEATHER SERVICE IN NEWPORT HAS ISSUED A * FLOOD WARNING FOR THE NEUSE RIVER AT KINSTON.* FROM SATURDAY MORNING UNTIL FURTHER NOTICE.* AT 900 AM EDT FRIDAY THE STAGE WAS 13.5 FEET.* MINOR FLOODING IS FORECAST.* FLOOD STAGE IS 14.0 FEET.* FORECAST...FLOOD STAGE WILL BE REACHED AT 900 AM SATURDAY. MAXIMUM STAGE WILL BE 15.0 FEET AT 900 PM EDT WEDNESDAY. THE RIVER MAY REMAIN ABOVE FLOOD STAGE FOR SEVERAL WEEKS. * IMPACT...AT 14 FEET...WATER WILL BEGIN TO OVERFLOW INTO LOWLANDS ADJACENT TO THE NEUSE RIVER. $$ • Flood Watch • Conditions are favorable for flooding but flooding is not imminent or occurring • Not verified • River Flood Warning • Issued for any high flow, overflow, or inundation event which is threatening lives and property and can be quantified or indexed at specific river locations • Verified

5. NWS River Flood Warning VerificationObjective • Objective: To provide national metrics of river flood warning lead time and accuracy similar to flash flood metrics • Hydrology Program committed to providing national metrics of river flood warning lead time and accuracy in both PPBES and OMB PART • The implementation of VTEC/HVTEC provided the data necessary to create a single source river flood warning verification system • OCWWS HSD and Regions coordinated on the requirements for the river flood warning verification system in Q2 FY08 • The OCWWS Performance and Awareness Branch began archiving all FLW’s (and all other products with the same WMO header) since October 1, 2007. • The OCWWS Performance and Awareness Branch delivered a prototype system September 2008

6. NWS River Flood Warning VerificationMetrics and System Requirements • Metrics • Probability of Detection • False Alarm Ratio • Critical Success Index • Lead Time • Absolute Timing Error • Metric Stratification • by Time (day, multiple days, month, multiple months, year) • by location (point, groupings of points, WFO, RFC, Region, National) • by Typical River Response (slow, medium, fast)

7. Product Information Used Hydrologic VTEC used: • 1st Product Issuance • NWS location identifier, VTEC action code (NEW), product issuance time, rise above time (forecast) • Last Product Issuance • NWS location identifier, VTEC action code (CAN,EXP), product issuance time, rise above time (observation), fall below time (observation) NCC049-107-070200-/O.NEW.KMHX.FL.W.0001.120407T1300Z-000000T0000Z//KINN7.1.ER.120407T1300Z.120412T0100Z.000000T0000Z.NO/1200 PM EDT FRI APR 6 2012 Issuances Used:

8. NWS River Flood Warning VerificationLogic Tree Event Not Over Do NOT USE in official stats Event Not Over Do NOT USE in official stats Is the last product action code equal to ‘CAN’ or ‘EXP’ or Is the last product PVTEC End time +120 hours EARLIER than the current time? NO Is the Last product Rise Above Time And Crest Time and Fall Below Time Equal to a string of zeros and Is the action code equal to ‘CAN’? YES YES NO 1 Incorrectly Warned Event (e.g., false alarm) LT and ATE not calculated 1 Incorrectly Warned Event (e.g., false alarm) LT and ATE not calculated Is the Last product Rise Above Time EARLIER than the last product issuance time? NO YES Is the 1st product Issuance Time EARLIER than or Equal to the last product Rise Above Time? 1 Unknown Event Do NOT use in verification statistics 1 Unknown Event Do NOT use in verification statistics NO YES 1 Warned Event (e.g., hit) LT = last product Rise Above time – 1st Product Issuance Time ATE = |last product Rise Above time-1st Product Rise Above Time| 1 Warned Event (e.g., hit) LT = last product Rise Above time – 1st Product Issuance Time ATE = |last product Rise Above time-1st Product Rise Above Time| 1 Unwarned Event (e.g., missed event) LT=0, ATE not calculated 1 Unwarned Event (e.g., missed event) LT=0, ATE not calculated

9. NWS River Flood Warning VerificationLimitations • Observations Issues • Quality Control of Observations - same level of WHFS QC applied prior to product issuance. Some QC automated with data ingest. • No Observation – event not included in metric • Reporting Frequency - system captures verification statistics for all locations where observations are available. However, we will only report statistics for those sites where the observational frequency is sufficient to assure the integrity of the verification data. • Underestimates missed events. • Prototype is still being tested – errors are likely • Database incomplete – Currently only includes points that have river response classification and are in the HADS database

10. NWS River Flood Warning VerificationWY2008 Preliminary Results

11. NWS River Flood Warning VerificationFY2008 Preliminary Results

12. NWS River Flood Warning VerificationNext Steps • Expand Interface to get detailed reports of events • Expand database - Each flood warning location must have a classified river response and an observation reporting frequency • Rigorous Testing • Develop River Flood Warning Lead Time and Accuracy performance goals

13. Any Questions?