1 / 15

Motivation

Recent Reduction in the Area of “EC” in CPC Outlooks and its Relationship to Value Ed O’Lenic, Kenneth Pelman, and David Unger NOAA-NWS-Climate Prediction Center 34th Climate Diagnostics and Prediction Workshop, October 26-30, 2009, Monterey, California. Motivation.

rumor
Download Presentation

Motivation

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Recent Reduction in the Area of “EC” in CPC Outlooksand its Relationship to ValueEd O’Lenic, Kenneth Pelman, and David UngerNOAA-NWS-Climate Prediction Center34th Climate Diagnostics and Prediction Workshop, October 26-30, 2009, Monterey, California

  2. Motivation This paper updates our efforts to improve the skill and utility of CPC official (OFF) 3-Month Temperature and Precipitation Outlooks. • We demonstrate that the skill and value of CPC’s .5-month-lead 3-Month Temperature and Precipitation Outlooks has improved since we began using an objective method to consolidate the forecast tools (CON) in our operations starting in 2006. • To do this we first compare the performance of Official (OFF) outlooks, made in real-time from1995-2004, with that of outlooks made by objective consolidation (CON) retrospectively over the same time period.

  3. Skill Measures We examine measures of performance including: • the fractional improvement by the forecast over random forecasts, sn, 2) the frequency (average area), n/T, of non-EC (equal chances) probabilities, and • sa , the area-weighted sn. • There is evidence of a sustained improvement in these metrics. • We also make a case for sa as a measure of value.

  4. s =fractional improvement over random forecasts Scores only non-EC areas Scores non-EC and EC areas n/T = frequency non-EC = % coverage T = # all forecasts n= # non-EC forecasts C = # correct forecasts E = # forecasts, out of T, expected correct by chance e= # forecasts, out of n, expected correct by chance EC= # EC forecasts

  5. The Case for Using sa as a Metric • The official metric, sn, rewards larger EC regions, this is not in the interest of users. • sa = (n/T) * sn, rewarding larger non-EC areas. • Users have made clear that forecasts with fewer EC forecasts have higher overall value. • Problem: sa is smaller, on average, than sn. Premise for this paper: sa is a measure of value

  6. Consolidation versus Official ForecastsOver 1995-2004 • We applied the objective consolidation technique, previously used for our official SST forecasts, to CONUS temperature and precipitation. • We compared the skill of consolidated forecasts (CON) with official forecasts (OFF) for 30 forecasts in each of 4 seasons from 1995-2004. • OFF forecasts were those made in real-time. • CON forecasts used the same 4 tools as OFF. • Results are reported in O’Lenic et al, 2008, and Unger et al, 2009.

  7. NEW OTLK

  8. sa Decomposed into sn (contours), and n/T (colors) for .5 mo-lead 3-Month Precipitation Forecasts Change in n/T (CON-minus-OFF) is given in % at bottom. 1995-2004 mean sn*100is at lower right in each figure. Skill,Official (OFF) .5 mo-lead 3-Month Precipitation Outlook, 1995-2004 FMA, MAM, AMJ (30 fcsts) ASO, SON, OND (30 fcsts) 19 2 MJJ, JJA, JAS (30 fcsts) NDJ, DJF, JFM (30 fcsts) 4 FMA, MAM, AMJ (30 fcsts) 4 Skill, Consolidation (CON) .5 mo-lead 3-Month Precipitation Outlook, 1995-2004 +8% +18% +20% +16% 13 ASO, SON, OND (30 fcsts) 17 NDJ, DJF, JFM (30 fcsts) MJJ, JJA, JAS (30 fcsts) 12 NDJ, DJF, JFM (30 fcsts) MJJ, JJA, JAS (30 fcsts) FMA, MAM, AMJ (30 fcsts) ASO, SON, OND (30 fcsts) 9 FMA, MAM, AMJ (30 fcsts) 0.5 Month Lead 3-Mo Precipitation Outlooks, 1995-2004: CON Raises U.S. Annual Mean sn from 9 to 12, and Increases non-EC Forecasts in All Seasons

  9. sa Decomposed into sn (contours), and n/T (colors) for 0.5 mo-lead 3-Month Temperature Outlooks Change in n/T (CON-minus-OFF) is given in % at bottom, 1995-2004 mean sn*100 is at lower right in each figure. Official (OFF) .5 mo-lead 3-Month Temperature Outlook, 1995-2004 FMA, MAM, AMJ (30 fcsts) ASO, SON, OND (30 fcsts) MJJ, JJA, JAS (30 fcsts) NDJ, DJF, JFM (30 fcsts) FMA, MAM, AMJ (30 fcsts) 23 22 16 37 Consolidation (CON) .5 mo-lead 3-Month Temperature Outlook, 1995-2004 +40% +55% +11% +31% ASO, SON, OND (30 fcsts) NDJ, DJF, JFM (30 fcsts) MJJ, JJA, JAS (30 fcsts) 25 33 23 33 FMA, MAM, AMJ (30 fcsts) 0.5 Month Lead 3-Mo Temperature Outlooks, 1995-2004: CON Raises U.S. Annual Mean sn from 18 to 24, and Increases non-EC Forecasts in All Seasons

  10. GPRA Score Official Skill Metric:48-Mo. Running Mean of sn Fraction x 100

  11. CONUS Spatial Average, 48-Month Running Mean of area covered ,sn, sa, sn-sa 1995-2005 compared with post-2005, Temperature CONSOLIDATION IMPLEMENTED n/T sd = 34 sd = 38 sn 30 27 Fraction x 100 sa 12 11 sa sn sn-sa n/T

  12. CONUS Spatial Average, 48-Month Running Mean of area covered ,sn, sa, sn-sa 1995-2005 compared with post-2005, Precipitation CONSOLIDATION IMPLEMENTED n/T Fraction x 100 sn sd = 31 sd = 34 sa 2.5 sa sn sn-sa n/T

  13. SUMMARY • Retrospective verification shows increase in the skill of objective Consolidation (CON) forecasts compared with Official (OFF), over 1995-2004. CONUS mean sn, n/T rise in all seasons, except winter for both T, P. For temperature, the sa OFF vs CON difference is significant at the 95% level, the sn rise is not. For precipitation, both sn and sa OFF vs CON differences are significant at the 95% level. • OFF forecasts made after 2005: 1. sa (Area-weighted sn) is a user-centric skill and value metric. 2. Mean sn ~ same/higher than for 1995-2005, without strong ENSO. 3. Sustained increase in skill, frequency (sa), and therefore, value. 4. The variability of sn drops, another measure of value. 5. Skill of L3MTO has risen as a direct result of use of CON. 6. Better forecasts can be made by focusing on value.

  14. CONUS Spatial Average, 48-Month Running Mean of area covered ,sn, sa, sn-sa 1995-2005 compared with post-2005, Temperature sa sn sn-sa n/T CONSOLIDATION IMPLEMENTED • T, post 2005: • N/T + • sn rises, stays at levels above those during period including a • strong ENSO • sa rises from 8 to 12 • Coverage rises to above strong ENSO levels • sd falls 38 to 34 • Mean sn ~ same as that for 1995-2005, but without strong ENSO n/T Fraction x 100 sn sd = 34 sd = 38 31 27 12 11 sa

  15. CONUS Spatial Average, 48-Month Running Mean of area covered ,sn, sa, sn-sa1995-2005 compared with post-2005, Precipitation sa sn sn-sa n/T CONSOLIDATION IMPLEMENTED • P, post 2005: • Coverage rises • sn rises to levels • comparable to those during strong ENSO • sa rises from 0.5 to 2.5 • Mean sn ~ same as that for 1995-2005, but without strong ENSO • sd falls 34 to 31 Fraction x 100 n/T sn sd = 34 sd = 31 2.5 sa

More Related