Internal and External Variance

1. On the multiple breakpoint problem and the number of significant breaks in homogenisation of climate recordsSeparation of true from spurious breaks Ralf Lindau & Victor VenemaUniversity of Bonn

2. Internal and External Variance Consider the differences of one station compared to a neighbour or a reference. Breaks are defined by abrupt changes in the station-reference time series. Internal variance within the subperiods External variance between the means of different subperiods Criterion: Maximum external variance attained by a minimum number of breaks 12th EMS Annual Meeting, Lodz, Poland – 13. September 2012

3. Decomposition of Variance n total number of years N subperiods ni years within a subperiod The sum of external and internal variance is constant. 12th EMS Annual Meeting, Lodz, Poland – 13. September 2012

4. First Question How do random data behave? Needed as stop criterion for the number of significant breaks. 12th EMS Annual Meeting, Lodz, Poland – 13. September 2012

5. with stddev = 1 Segment averages xi scatter randomly mean : 0 stddev: 1/ Because any deviation from zero can be seen as inaccuracy due to the limited number of members. Random Time Series 12th EMS Annual Meeting, Lodz, Poland – 13. September 2012

6. c2-distribution Weighted measure for the variability of the subperiods‘ means The external variance is equal to the mean square sum of a random standard normal distributed variable. 12th EMS Annual Meeting, Lodz, Poland – 13. September 2012

7. From c2 to b distribution As the total variance is normalized to 1, a kind of normalized chi2-distribution is expected: This is the b-distribution. 7 breaks in 21 years n = 21 years k = 7 breaks data b The exceeding probability P gives the best (maximum) solution for v c2 Incomplete Beta Function 12th EMS Annual Meeting, Lodz, Poland – 13. September 2012

8. Added variance per break Incomplete b-function: 95% 90% Transformation to dv/dk: mean 12th EMS Annual Meeting, Lodz, Poland – 13. September 2012

9. The extisting algorithm Prodige Original formulation of Caussinus and Mestre for the penalty term in Prodige Translation into terms used by us. Normalisation by k* = k / (n -1) Derivation to get the minimum In Prodige it is postulated that the relative gain of external variance is a constant for given n. 12th EMS Annual Meeting, Lodz, Poland – 13. September 2012

10. Shorter length, less certainty n = 101 years n = 21 years Exceeding probability 1/128 1/64 1/32 1/16 1/8 1/4 12th EMS Annual Meeting, Lodz, Poland – 13. September 2012

11. Second Question How do true breaks behave? 12th EMS Annual Meeting, Lodz, Poland – 13. September 2012

12. True Breaks 12th EMS Annual Meeting, Lodz, Poland – 13. September 2012

13. Identical Behaviour True breaks behave identical to random data. But the abscissa-scale is now: k / nk instead of k / n. Compared to random time series the external variance grows faster by the factor n / nk nk = 19 true breaks within n = 100 years time series data theory Assumed / True Break Number k / nk 12th EMS Annual Meeting, Lodz, Poland – 13. September 2012

14. Break vs Scatter Regime h Simulated data with 19 breaksinterfered by scatter The internal variance decrease as a function of break number. In the break regime the variance decrease faster by the factor: 15 breaks are detectable, depending on signal to noise ratio. Time series length Number of true breaks 12th EMS Annual Meeting, Lodz, Poland – 13. September 2012

15. Conclusions • The analysis of random data shows that the external variance is b-distributed, which leads to a new formulation for the penalty term. • True breaks are also b-distributed. Their external variance increases faster by a factor of n/nkcompared to random scatter. 12th EMS Annual Meeting, Lodz, Poland – 13. September 2012