1 / 32

Skilled Predictability Assessment in Climate Forecast System: Challenges and Insights

Explore definitions & challenges in evaluating real-time forecast skill vs. hindcasts in the Climate Forecast System. Analyze predictability of different kinds & implications for CFS forecasts. Discover the importance of degrees of freedom in prediction skill.

lstiles
Download Presentation

Skilled Predictability Assessment in Climate Forecast System: Challenges and Insights

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Huug van den Dool and Suranjana Saha Prediction Skill and Predictability in CFS

  2. Definitions Prediction Skill and PredictabilityOpinion: Literature fuzzies up ‘predictability’ vs ‘prediction skill’

  3. Definition 1: Evaluation of skill of real time prediction; the old-fashioned way. Problems: a) Sample size! , b) Wait a long time(and funding agents are impatient)

  4. Definition 1: Evaluation of skill of real time prediction; the old-fashioned way.Definition 2: Evaluation of skill of hindcasts; hard, not impossible.Problems: a) Sample size, b) ‘honesty’ of hindcasts

  5. Definition 1: Evaluation of skill of real time prediction; the old-fashioned way. Definition 2: Evaluation of skill of hindcasts; hard, not impossible.Definition 3: Predictability of the 1st kind (~ sensitivity due to uncertainty in initial conditions)

  6. Definition 1: Evaluation of skill of real time prediction; the old-fashioned way. Sample size!Definition 2: Evaluation of skill of hindcasts; hard, not impossibleDefinition 3: Predictability of the 1st kind (~ sensitivity due to uncertainty in initial conditions)Definition 4: Predictability of the 2nd kind due to variations in external boundary conditions (AMIP; Potential Predictability; Reproducibility; Madden’s approach)

  7. Predictability (theoretical/intrinsic) is a ceiling for actual prediction skill.Any other ‘kinds’ of predictability?

  8. CFS forecast:X (space, lead, member ,year) • Space is 2.5oX2.5o (Z500) or 1oX2o (SST/mask), or 1.875 by Gaussian (Soilw, T2m, Precip) • Basic data used is monthly mean • Lead = 0, 8 in units of months; member = 1, 15 • Year = 1981 – 2003 (increases annually) • Example: ‘Initial’ Month is August (= lead 0); • Note IC is Jul 11/21/Aug 1 for SST, and Jul 09-13/ 19-23 / Jul 30-Aug3 for atmosphere and soil. • ‘Member’ 16 is ensemble average • ‘Member’ 17 is matching observed field • X = ( Z500, SST, Soilw, T2m, Precip)

  9. ASPECTS • Prediction skill (member i vs member 17) • Predictability (member i vs member j) • Monthly mean • Seasonal mean • Ensemble average • Predictability of 1st kind only.

  10. Two types of climatology plus complications • Xclim_mdl (space, lead) is average over years and (14 or 15) members, depending. • Xclim_verif (space, lead) is ave over (same) years for either member 17, or member i, i=15. • Anomaly = X minus Xclim, whichever is relevant • Systematic error (SE) is automatically corrected by the above • CV of the SE correction (exclude from Xclim the member and the year to be verified). Not trivial.

  11. Prediction Skill Monthly

  12. Conclusions (monthly data) • CFS data is a goldmine. • CFS has enough (?) data for forecast evaluation (and diagnostics) • Member i vs member j unifies predictability of 1st and 2nd kind in CFS output • CFS has some prediction skill. In order of skill: SST, {tropical variables}, soilw,T2m, Precip • CFS has some more predictability (as defined), but ceiling is ‘low’ in mid-latitudes. • Seasonality (no surprise)

  13. To do: • Identify interdecadal skill source (if any) • Identify soil moisture skill source (are models still too strong on local effects? How about non-local effects) • Daily data for the finer temporal scales in skill/predictability. • Why do models like CFS have predictability in so few d.o.f. (and is that really all there is) • Further ideas about ‘new’ predictability notions

  14. A case for the importance of knowing the effective number of degrees of freedom (edof) in which we have forecast skill.Considerations:-) physical models have one clear strength: they can execute the non-linear terms-) a model needs at least 3 degrees of freedom to be non-linear (Lorenz, 1960)-) a non-linear model with nominally a zillion degrees of freedom, but skill in only <= 3 dof is functionally linear in terms of the skill of its forecasts - and, to its detriment, the non-linear terms add random numbers to the tendencies of the modes with predictability. ==> Therefore: Physical models need to have skill in, effectively, > 3 dof before they can be expected to take advantage of non-linearity. (In a forecast setting). ( Note: not any 3 degrees of freedom will do.)

  15. ‘Lingering memory’ Cai+Van den Dool(2005); Schemm et al calibration data set, (CFS daily data set will be used also).

More Related