Hypothesis testing and confidence intervals by resampling

1. Hypothesis testing and confidence intervals by resampling by J. Kárász

2. Contents • The bootstrap method • The bootstrap analysis of the Kolmogorov-Smirnov test • The bootstrap analysis of the GEV parameter investing • The bootstrap analysis of the movig window method

3. Testing of homogenity • Kolmogorov-Smirnov test • Investigating the GEV parameter of dependence on time

4. Bootstrap methodconditions of use is a random sample from the unknown probability distribution function (F) with finite variance. is the unknown parameter, the function of F. is the non-parametric estimate of the parameter, the function of the random sample.

5. Bootstrap methodbootstrap estimate of the standard error is the standard error of the estimate. Then the bootstrap estimate is Unfortunatly in most cases it’s impossible to express it as a simple function of or the random sample, so we have to use numeric approximation.

6. Bootstrap methodbootstrap sample and bootstrap replicate To approximate the empirical distribution function, the bootstrap algorithm takes random samples from the empirical distribution function: where This is the bootstrap sample. It is nothing else but a random sample from with replacement. By evaluating the statistic of interest we get a bootstrap replicate:

7. Bootstrap methodapproximation with Monte Carlo method • Independently draw a large number of bootstrap samples: • Evaluate the statistic of interest, so we get B bootstrap replicates: • Calculate the sample mean and sample standard deviation of the replicates:

8. Bootstrap methodconfidence intervals The histogram of the bootstrap replicates is an empirical density function for , so the  and  histogram percentiles are suitable limit estimates for the  percent confidence interval.

9. Bootstrap methodhypotesis testing H0: are iid random variables. H1: are not iid random variables. The bootstrap samples are drawn from the same distribution ( ) independently, so if H0 holds, then and the replicates are from quite similar distribution, because F and are similar for large sample size. If is out of the empirical confidence interval,we accept H1, in other case, we accept H0.

10. H0: are iid random variables. H1: are not iid random variables. Our suppose was that the annual maximum water levels are independent, so if we refuse H0 we have to accept that are not from the same distribution. This can even mean trend. Kolmogorov-Smirnov test

11. Results 1 means H0 was refused. t=1,2,3 : parameter for cutpoint in K-S test.

12. Results - example Annual maximum water level at Szolnok, t=1

13. Results - example Annual maximum water level at Szolnok, t=2

14. H0: are iid random variables. H1: are not iid random variables. Our suppose is that the annual maximum water levels are independent, so if we refuse H0 we have to accept that are not from the same distribution. This can even mean trend. Examinating the GEV parameter of shape

15. Results 1 means H0 was refused. No dataset was found to refuse H0 both in K-S test and GEV parameter testing.

16. Results – data table

17. Results - example

18. Further questions – two-peeked bootstrap empirical distributions

19. Permutation testingsimilar to bootstrap method Same as the bootstrap algorithm except that permutation sample is drawn without replacement. The hypotesis testing is similar too, we examine the estimate and the empirical confidence interval.

20. Moving window method forecastanalysis by permutation method • Our aims were: • Simulating the original dataset by permutation. • Supervise the quality of the forecast.

21. Results -example

22. Results - example

23. Results - example

24. Results - example