1 / 38

Evaluation of Selected Winter ’04/’05 Performance Results

Evaluation of Selected Winter ’04/’05 Performance Results. Seth Linden and Jamie Wolff NCAR/RAL. Weather Forecast Verification. Consensus (RWFS) forecast is compared to individual model components Air-temperature, dewpoint, wind-speed and cloud-cover forecasts

tad-butler
Download Presentation

Evaluation of Selected Winter ’04/’05 Performance Results

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Evaluation of Selected Winter ’04/’05 Performance Results Seth Linden and Jamie Wolff NCAR/RAL

  2. Weather Forecast Verification • Consensus (RWFS) forecast is compared to individual model components • Air-temperature, dewpoint, wind-speed and cloud-cover forecasts • 18 UTC runs for the entire season (1 November 2004 to 15 April 2005) • Error (RMSE) calculated for: • Colorado Plains: 176 sites • Mountains: 119 sites Blizzard of March 2003

  3. Air temperature RMSE Colorado Mountains RWFS Colorado Plains

  4. Colorado Plains Colorado Mountains Forward Error Correction Due to 3-hour MOS data Dewpoint RMSE

  5. Colorado Plains Colorado Mountains Wind Speed RMSE

  6. Colorado Mountains Colorado Plains Cloud Cover RMSE

  7. Summary/Recommendations • The ensemble approach utilized by the RWFS does improve the predictions on average for all verifiable parameters • No single model performs better for all parameters • A blend of weather models will provide better results

  8. Forecast Model Weights Used by the RWFS • System automatically weights forecasts based on skill • Distribution of weight values per lead time for air-temperature, dewpoint, and wind-speed • 18 UTC run on 3 May 2005 • Weights looked at for two sites: • Denver International Airport • I-70 at Genesse Which models have the most skill?

  9. Denver Int. Airport Air Temperature Model Weights ETA I-70 at Genesee MOS GFS RUC MOS

  10. Denver Int. Airport Dewpoint Model Weights I-70 at Genesee

  11. Denver Int. Airport Wind Speed Model Weights I-70 at Genesee MM5 WRF

  12. Clear Conditions Insolation Weights • For MDSS static weights were applied: • - 50/50 split between MM5 and WRF for the • 0-23 hour forecast • - All Eta for the 24-48 hour forecast • No one model consistently outperforms the others • MM5 and WRF forecast hourly instantaneous values, • ETA forecasts 3-hour instantaneous values and • GFS forecasts 3-hour averages

  13. QPF Weights • Due to a lack of quality precipitation observations static weights were applied • Weights fixed based on expert opinion • MM5 and WRF were given 80% of the total weight

  14. Summary/Recommendations • Weight distribution reflects that the corrected (dynamic MOS) NWS models (ETA, GFS, and RUC) had the most overall skill • WRF and MM5 were given the highest static weights for Insolation and QPF • No one model dominates for all parameters • The limitation of the NWS models is their 3-hr temporal resolution

  15. Road Temp Observation Variance • Tr variance across E-470 corridor • Shading by permanent structures or passing clouds • Make/model/installation/age of temperature sensors

  16. E-470 Road/Bridge Sites Platte Valley (road and bridge) Colorado Blvd 6th Ave Pkwy Smokey Hill Rd (road and bridge) Plaza A

  17. SCT BKN OVC 27 Nov 2004 28 Nov 2004 LOCAL TIME (19 = noon, 07 = midnight)

  18. OVC BKN SCT CLR 29 Nov 2004 30 Nov 2004 LOCAL TIME (19 = noon, 07 = midnight)

  19. Summary/Recommendations • Large variations in observed road and bridge temperatures • Over relatively small area (10s of miles) • Makes prediction and verification of pavement temperatures very challenging • Difficult to establish ground truth

  20. Road/Bridge Forecast Verification • Road and bridge temperature forecasts • Using recommended treatments from MDSS • Error (MAE) and bias calculated for: • For each lead time (0-48hrs) 18 UTC runs • E-470: 6 roads/2 bridge (1 Nov 2004 – 15 Apr 2005) • Mountains: 5 roads (1 Feb 2004 – 15 Apr 2005) East bound lane of I-70 at the summit of Vail Pass

  21. Peak insolation Morning hours Consistent low bias Perfect forecast E-470 road sites Lead Time (0 = 18 UTC = noon, 18 = 12 UTC = 6am)

  22. evening Shadowing? morning E-470 bridge sites Lead Time (0 = 18 UTC ~ noon, 18 = 12 UTC ~ 6am)

  23. evening morning CDOT mountain road sites Lead Time (0 = 18 UTC = noon, 18 = 12 UTC = 6am)

  24. Summary/Recommendations • Larger Tr differences during times of high solar insolation likely due to several factors: • Errors in measuring pavement skin temp • Mountain shading during low sun angle • Limitations in insolation prediction in models • Limitations in pavement heat balance model • Simplified assumptions about pavement characteristics • Tb analysis compromised by: • Sensors shadowed by bridge rail • Bias results suggest tuning may be beneficial • Overall Issue: • Actual/Recommended treatments not the same

  25. Case Study Analysis • 183 day demonstration • 16 winter weather days • 10 light snow • 5 moderate snow • 1 heavy snow

  26. November 27-29, 2004 • First significant snow storm of the season • 5-8” in the Denver area • Large variations in parameter predictions • Forecast vs. observations • Denver International Airport • Ta, Td, Wspd, Cloud Cover and Precipitation • 12 UTC 28th examined • Captured the start time of event

  27. 8C/14F diff 2C/4F diff Air Temperature Snow 28 Nov 2005 LOCAL TIME (19 = noon, 06 = midnight)

  28. 6C/11F diff Dewpoint Temperature Snow 28 Nov 2005 LOCAL TIME (19 = noon, 06 = midnight)

  29. Wind Speed Snow 28 Nov 2005 LOCAL TIME (19 = noon, 06 = midnight)

  30. FEC Cloud Cover Snow 28 Nov 2005 LOCAL TIME (19 = noon, 06 = midnight)

  31. Quantitative Precipitation Forecast Snow 28 Nov 2005 LOCAL TIME (19 = noon, 06 = midnight)

  32. March 13, 2005 • Moderate Snow Event • 4-6” along the E-470 corridor • Warm air temps before start of snow • Dropped from 11C (52F) to -2C (29F) in 5 hours • Large variations in parameter predictions • Forecast vs. observations • Denver International Airport • Ta, Wspd, Cloud Cover and Precipitation • 00 UTC 13 March 2005 run examined • Captured both start and end times

  33. Air Temperature Snow 13 March 2005 LOCAL TIME (18 = noon, 07 = midnight)

  34. Wind Speed Snow 13 March 2005 LOCAL TIME (18 = noon, 07 = midnight)

  35. SCT - OVC Cloud Cover Snow 13 March 2005 LOCAL TIME (18 = noon, 07 = midnight)

  36. actual actual forecast forecast Start time End time Quantitative Precipitation Forecast 13 March 2005 LOCAL TIME (18 = noon, 07 = midnight)

  37. Summary/Recommendations • Large discrepancies between weather models in predicting state weather parameters • All too dry for Td and cloud cover • Low wind speed bias during windy conditions • Overall, no ONE model outperforms => Ensemble approach key • Supports probabilistic forecast presentation • Atmosphere is unpredictable • Best approach to present uncertainty to end users?

  38. Thank You!Questions?

More Related