Separation of True and Spurious Breaks in Climate Records Analysis

1. On the multiple breakpoint problem and the number of significant breaks in homogenisation of climate recordsSeparation of true from spurious breaks Ralf Lindau & Victor VenemaUniversity of Bonn

2. Internal and External Variance Consider the differences of one station compared to a neighbour or a reference. Breaks are defined by abrupt changes in the station-reference time series. Internal variance within the subperiods External variance between the means of different subperiods Criterion: Maximum external variance attained by a minimum number of breaks 12th EMS Annual Meeting, Lodz, Poland – 13. September 2012

3. Decomposition of Variance n total number of years N subperiods ni years within a subperiod The sum of external and internal variance is constant. 12th EMS Annual Meeting, Lodz, Poland – 13. September 2012

4. First Question How do random data behave? Needed as stop criterion for the number of significant breaks. 12th EMS Annual Meeting, Lodz, Poland – 13. September 2012

5. with stddev = 1 Segment averages xi scatter randomly mean : 0 stddev: 1/ Because any deviation from zero can be seen as inaccuracy due to the limited number of members. Random Time Series 12th EMS Annual Meeting, Lodz, Poland – 13. September 2012

6. c2-distribution Weighted measure for the variability of the subperiods‘ means The external variance is equal to the mean square sum of a random standard normal distributed variable. 12th EMS Annual Meeting, Lodz, Poland – 13. September 2012

7. From c2 to b distribution As the total variance is normalized to 1, a kind of normalized chi2-distribution is expected: This is the b-distribution. 7 breaks in 21 years n = 21 years k = 7 breaks data b The exceeding probability P gives the best (maximum) solution for v c2 Incomplete Beta Function 12th EMS Annual Meeting, Lodz, Poland – 13. September 2012

8. Added variance per break Incomplete b-function: 95% 90% Transformation to dv/dk: mean 12th EMS Annual Meeting, Lodz, Poland – 13. September 2012

9. The extisting algorithm Prodige Original formulation of Caussinus and Mestre for the penalty term in Prodige Translation into terms used by us. Normalisation by k* = k / (n -1) Derivation to get the minimum In Prodige it is postulated that the relative gain of external variance is a constant for given n. 12th EMS Annual Meeting, Lodz, Poland – 13. September 2012

10. Shorter length, less certainty n = 101 years n = 21 years Exceeding probability 1/128 1/64 1/32 1/16 1/8 1/4 12th EMS Annual Meeting, Lodz, Poland – 13. September 2012

11. Second Question How do true breaks behave? 12th EMS Annual Meeting, Lodz, Poland – 13. September 2012

12. True Breaks 12th EMS Annual Meeting, Lodz, Poland – 13. September 2012

13. Identical Behaviour True breaks behave identical to random data. But the abscissa-scale is now: k / nk instead of k / n. Compared to random time series the external variance grows faster by the factor n / nk nk = 19 true breaks within n = 100 years time series data theory Assumed / True Break Number k / nk 12th EMS Annual Meeting, Lodz, Poland – 13. September 2012

14. Break vs Scatter Regime h Simulated data with 19 breaksinterfered by scatter The internal variance decrease as a function of break number. In the break regime the variance decrease faster by the factor: 15 breaks are detectable, depending on signal to noise ratio. Time series length Number of true breaks 12th EMS Annual Meeting, Lodz, Poland – 13. September 2012

15. Conclusions • The analysis of random data shows that the external variance is b-distributed, which leads to a new formulation for the penalty term. • True breaks are also b-distributed. Their external variance increases faster by a factor of n/nkcompared to random scatter. 12th EMS Annual Meeting, Lodz, Poland – 13. September 2012