1 / 66

Probabilistic Forecast

This forecast by Kiyoharu Takano from JMA's Climate Prediction Division discusses the predictability, examples, and quality of probabilistic forecasts, and the economical benefits of forecasting. It also explores the limitations of deterministic forecasts and the predictability of seasonal forecasts.

fosterwilliam
Download Presentation

Probabilistic Forecast

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Probabilistic Forecast Kiyoharu Takano Climate Prediction Division JMA

  2. 3-month(Dec.-Feb.) forecast issued by JMA at 25,Nov.

  3. Contents (1) Predictability of Seasonal Forecast (2) Example of Probability Forecast (3) Quality of Probability Forecast Reliability Resolution (4) Economical Benefit of Forecast Deterministic forecast Probabilistic forecast

  4. (1)Predictability of Seasonal Forecast cf. Dr. Sugi’s presentation < Predictability of 1st kind >Originates from Initial conditionDeterministic forecast fails beyond a few weeks due to the growth of errors contained in the initial states (Lorenz, 1963; 1965). < Predictability of 2nd kind > Lorenz (1975) Originates from boundary condition Effective for longer time scale; Month to season There remains internal variability which is not controlled by boundary conditions

  5. Growth of forecast error(Predictability of 1st kind) 1. Error growth due to imperfectness of numerical prediction ⇒Improvement of numerical model 2. Growth of initial condition error 　⇒ Improvement of objective analysis However………... (1) There remains finite (non zero) error in initial condition, (although it will be reduced as improvement of objective analysis). We cannot know ‘true’ initial condition (2)Small initial error(difference) grows fast(exponentially) as time progresses and the magnitude of error becomes the same order as natural variability after a certain time. ⇒Chaos Deterministic forecast becomes meaningless after sufficiently long time

  6. Chaos Lorenz,E. N. ,1963, J.A.S. 20, 130- Equations for simplified role type convection.　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　‘The Lorenz system’ X: Fourier component of stream function Y,Z：Fourier component of temperature Nonlinear terms :XZ,XY r：Stability parameter

  7. X Time Z Y X • Findings of Lorenz • ‘The solution (x(t),y(t),z(t)) • behaves strangely’ • The solution (x(t),y(t),z(t)) • is bound. • Non-periodic • Slight initial difference causes • large difference. Chaos Observation Numerical prediction Time steps (x0.01))

  8. What can we do on the predictability limit? Predictability limit varies depending on flow pattern.

  9. 0:100 slow initial errors growth fast initial errors growth 70:30 50:50 • What can we do on the predictability limit? • Predictability limit varies depending on atmospheric flow pattern and initial condition • Prediction of uncertaintyof forecast are important warm cold

  10. The Ensemble Prediction 7/24 a forecast +prediction of uncertainty of forecast Temperature anomalies at 850hPa over the Northern Japan (7day running mean) 6/ 5 Initial：5 June 2003 Initial：24 July 2003

  11. Longer than one month forecast Seasonal forecast One-month forecast One-week forecast Longer than one month forecast based on initial condition is impossible at least generally!

  12. Predictability of second kind In seasonal time scale, forecast based on initial condition is impossible at least generally The forecast is based on the influences of boundary conditions such as SSTs or soil wetness. Prediction of second kind However ……….. Regression OLR map with Nino3 SSTs (DJF)

  13. Atmospheric variation is not fully controlled by variation of boundary condition such as SSTs but there are internal variation.. Examples of internal variations are baroclinic instability , typhoon, Madden-Julian oscillation e.t.c. and these can be predicted as initial value problem in short time scale but they are unpredictable in seasonal time scale. Then a variation X is writtenas X=Xext+Xin Xext : variation controlled by boundary conditions(Signal) Xin : internal variation(Noise) (cf. Mr. Sugi’s presentation)

  14. 30day mean 850hPa Temperature prediction at the Nansei Islands Noise Signal Individual numerical prediction Average of individual predictions (ensemble mean) Observation

  15. Reduction of noise Since the internal variation can be reduced by time mean but the signal is not reduced, Time-mean is taken in seasonal forecast. Unpredictable reduced by time mean predictable This time mean is effective especially in tropics. In addition, main SST signals such as ENSO are in tropics. Seasonal forecast in tropics is easier than in mid-latitude.

  16. The noise reduction effect of time mean 5day mean Individual numerical prediction Average of individual predictions (ensemble mean) Observation 30day mean 90day mean

  17. Though time mean is effective to reduce noise, the noise is not removed completely. Therefore there remains uncertainty from internal variation and again probabilistic forecast is necessary. Individual numerical prediction Average of individual predictions (ensemble mean) Observation 90day mean

  18. (2) Example of probabilistic forecast (a) One-month forecast NJ WJ Temperature at 850hPa EJ Nansei Surface Temperature Forecast 1 month 1st week 2nd week 3rd-4th week Forecast Period11.22-12.21 11.22-11.28 11.29-12.5 12.6-12.19 category - 0 + - 0 + - 0 + - 0 + Northern Japan 20 40 40 20 40 40 20 50 30 20 40 40 Eastern Japan 20 30 50 20 30 50 20 40 40 20 40 40 Western Japan 10 40 50 10 40 50 20 30 50 20 40 40 Nansei Islands 10 40 50 10 30 60 20 30 50 20 40 40 ( category : below normal, 0 : near normal, + : above normal, Unit : %)

  19. (2) 3 month forecast Temperature Forecast Period 3 months 1st month 2nd month 3rd month Nov.-Jan. Nov. Dec. Jan. Category - 0 + - 0 + - 0 + - 0 + Northern Japan 30 50 20 30 50 20 30 50 20 20 50 30 Eastern Japan 20 5030 20 50 30 20 50 30 20 40 40 Western Japan 20 40 40 20 50 30 20 40 40 20 40 40 Nansei Islands 20 30 50 20 40 40 20 30 50 20 30 50 ( category ? : below normal, 0 : near normal, + : above normal, Unit : %)

  20. Precipitation Forecast (3 month forecast) Period 3 months 1st month 2nd month 3rd month Nov.-Jan. Nov. Dec. Jan. category - 0 + - 0 + - 0 + - 0 + Northern Japan Japan Sea side 20 50 30 20 50 30 20 50 30 30 40 30 Pacific side 30 50 20 30 50 20 30 50 20 30 40 30 Eastern Japan Japan Sea side 20 50 30 20 50 30 20 50 30 30 50 20 Pacific side 20 50 30 20 50 30 20 50 30 20 40 40 Western Japan Japan Sea side 20 50 30 20 50 30 20 50 30 30 40 30 Pacific side 20 50 30 20 50 30 20 50 30 20 40 40 Nansei Islands 20 50 30 40 40 20 20 40 40 30 40 30 ( category -: below normal, 0 : near normal, + : above normal, Unit : %)

  21. Seasonal forecast of U.S.

  22. Seasonal forecast by I.R.I.

  23. Examples of probabilistic forecast excluding seasonal forecast at JMA JMA uses probabilistic expression not only in seasonal forecast but in many forecasts where there is uncertainty of forecast. (1) Short-range forecast Tokyo District Today North-easterly wind, fine, occasionally cloudy, Wave 0.5m Probability of Precipitation 12-18 10% 18-00 0% Temperature forecast today’s maximum in Tokyo 14 degrees centigrade

  24. Date 24Mon 25Tue26Wed 27Thu 28Fri 29Sat 30Sun Max.,Min. T(℃） Probability of Precipitation (2) One-week forecast 　　　　　　　　　１0 / 1611/15 7/15 9/16 9/17 11/18 70 50 40 30 30 40 Issuedat23,NOV

  25. (3) Typhoon Forecast The probability that a district will be in storm warming area is also being issued.

  26. Quality of Probabilistic Forecast What is good probabilistic forecast? ‘A good probabilistic forecast must express the uncertainty of forecast exactly and have large dispersion from ‘climatic proportion of frequency’ (a) Reliability (b) Resolution

  27. (a) Reliability Probability forecast P was issued ‘M(P)’ times for a event ‘E’. In M(P) times, event ‘E’ occurred N(P) times. If probability forecast is reliable for large number of M(P). Ex. Probability 30% was issued 50times. Event ‘E’ is expected to occur about 15timesin them.

  28. The reliability diagram Event E Ideal reliability diagram Relative frequency Forecast probability

  29. Reliability Diagram Monthly mean surface temperature forecast in Japan Relative frequency Forecast probability

  30. monthly mean Psea anomalies >0 Reliability diagram Example of probability by ensemble one month forecast

  31. (b) Resolution Climatorogical relative frequency for a event is perfectly reliable so far as there is no climatic Change. ex. It is known that relative frequency of rainy day is about 30% at Tokyo from historical data set. Can we issue probability of 30% as a tomorrow's probability of precipitation at Tokyo every day? When the reliability is perfect, dispersion of probability from climatorogical relative frequency is another measure of probabilistic forecast quality ‘resolution’. The best resolution probabilistic forecasts are those of 100% or 0% provided that reliability is perfect. =perfect forecast.

  32. Resolution Monthly mean surface temperature forecast in Japan Relative frequency Forecast probability Frequency

  33. Resolution of probability forecast 7day mean Z500 anomalies>0 1st week Reliability diagram Relative frequency(resolution) 1st week probabilistic forecast is better than 2nd week in ‘resolution’ measure 2nd week From one-month ensemble forecast

  34. Quantitative evaluation of probabilistic forecast -------The Brier score------- The Brier score is defined as pi : forecast probability vi=1: if event E occurred =0 : if not occurred N: total number of forecast bis mean square error of probability

  35. After some algebraic transformations, b can be rewritten as brel bres bunc where Nt :frequency of forecast probability pt Mt: frequency of occurrence of event E within Nt

  36. represents ‘Reliability’ represents ‘Resolution’ represents ‘Uncertainty’ (not depends on the forecast. shows difficulty of forecast) Murphy’s decomposition(1973)

  37. Brier skill score The absolute value of brier score is difficult to understand except perfect reliability and resolution case (=0). Then usually following Brier skill score is used where bc is the brier score of climatorogical relative frequency( probabilistic climate forecast).B=1 when the forecast is perfect. Additionally, following skill score is also used. Brel=Bres=1 when the forecast is perfect.

  38. Example of Brier skill score Ensemble one-month forecast 7day mean Z500anomalies>0 2nd week 1st week All scores are expressed in %

  39. (4) Economical value of forecast ----a consideration with Cost/Loss model----- The Cost/Loss model an event E ①If E occurs, the loss is Ldue to a damage ②To protect form the damage, a action costs C (<L) ex. If the temperature exceeds a threshold, a crop is damaged by a pest. The damage is L. To protect the crop from a pest , spraying agrochemical is necessary and it costsC.

  40. The cost/benefit model a Event E if the event E occurs , a benefit is B. If not occurs, we lose C. Example、 If it is fine, a benefit B is earned by selling lunch boxes, but if it rains, no benefit is earned. The cost to make lunch boxes is C. The discussion of Cost/Loss is almost the same as that of Cost/Benefit model.

  41. The case without forecast (Cost/Loss model) Considering D times operations. The climatorogical proportion of occurrence of event E is R. When always taking action to protect from damage, the expense is , If no action is done, the expense is Then, when R>C/L If the event E occurs frequently, it is better to take action always.

  42. The case with perfect forecast (Cost/Loss model) If forecast is perfect, we take action only when event E is forecasted. Then the expense is of course, Perfect forecast always reduces the expense

  43. The case with actual deterministic forecast Actual forecast sometimes fails. Then we make following contingency matrix Occurs Forecast ( ) shows in case of perfect forecast If we use these forecasts, the expenses corresponding individual boxes above are, Occurs Take Action

  44. The relationships with pre-defined variables are Occurs Forecast We newly define following variables which express forecast skill Hit rate False-alarm rate The lager H is and the smaller F is , the better forecast is. H=1 and F=0, for perfect forecast

  45. The expense with these forecasts is If forecast is worse, Mp sometimes become lager than Mclim2 or Mclim1 . IF H=0 and F=1(the worst forecast!) Mp=DLR+D(1-R)C>Mclim1,Mclim2

  46. It should be also noted that Mp depends not only on H and F but C/Land R . An imperfect deterministic forecast is not always useful for all users.

  47. The case with probabilistic forecast (Cost/Loss model) How do we use probabilistic forecast? Consider to take action always when the probabilistic forecast is P0 for the occurrence of event E. The expense is, Mp0=DpC where Dp is total frequency that the event E was forecasted with probability P0 If probabilistic forecast is perfectly reliable, the frequency that event E occurred is DpP0 and then the expense without taking action is Mc=DpP0L

  48. Probabilistic forecast is useful when, Mp0=DpC < Mc=DpP0L Therefore, we should take action when P0>C/L This is the simplest way to use probabilistic forecast for decision making. Note the threshold probabilities are different for individual users with different C/L and all users can get some gain with their own threshold. In case of Cost-benefit model, with similar calculations, the criterion is, P0 >C/(B+C)

  49. Ex. 1(Cost-Loss model) If the temperature exceeds a threshold =33℃, a crop is damaged by a pest. The damage is L=10,000$ To protect the crop from a pest , spraying agrochemical is necessary and it costsC＝3,000$ C/L=3000/10000=0.3 Consider 10 times forecast of above 33℃ with probability 20%(30%,40%) . When we take action the expense is 3000x10=30,000$ When we do not take action, the expense is 10000x(10x0.2)=20,000 $ for probability 20% 10000x(10x0.3)=30,000 $ for probability 30% 10000x(10x0.4)=40,000 $ for probability 40% We had better take action when Probability>30%=C/L

More Related