Correction of daily values for inhomogeneities

1. Correction of daily valuesfor inhomogeneities P. Štěpánek Czech Hydrometeorological Institute, Regional Office Brno, Czech Republic E-mail: petr.stepanek@chmi.cz COST-ESO601 meeting, Tarragona, 9-11 March 2009

2. Using daily data for inhomogeniety detection, is it meaningful?

3. Homogenization of daily values –precipitation series • working with individual monthly values (to get rid of annual cycle) • It is still needed to adapt data to approximate to normal distribution • One of the possibilities: consider values above 0.1 mm only • Additional transformation of series of ratios (e.g. with square root)

4. Homogenization of precipitation – daily values Original values - far from normal distribution (ratios tested/reference series) Frequencies

5. Homogenization of precipitation – daily values • Limit value 0.1 mm (ratios tested/reference series) Frequencies

6. Homogenization of precipitation – daily values • Limit value 0.1 mm, square root transformation (of ratios) (ratios tested/reference series) Frequencies

7. Problem of independence, Precipitation above 1 mm • August, Autocorrelations

8. Problem of independece,Temperature • August, Autocorrelations

9. Problem of independece,Temperature differences (reference – candidate) • August, Autocorrelations

10. Homogenization • Detection (preferably on monthly, seasonal and annual values) • Correction – for daily values

11. WP1 SURVEY (Enric Aguilar)Daily data -Correction (WP4) • Very few approaches actually calculate special corrections for daily data. • Most approaches either • Do nothing (discard data) • Apply monthly factors • Interpolate monthly factors • The survey points out several other alternatives that WG5 needs to investigate

12. Daily data correction methods • „Delta“ methods • Variable correction methods – one element • Variable correction methods – several elements

13. Daily data correction methods • Interpolation of monthly factors • MASH • Vincent et al (2002) - cublic spline interpolation • Nearest neighbour resampling models, by Brandsma and Können (2006) • Higher Order Moments (HOM), by Della Marta and Wanner (2006) • Two phase non-linear regression (O. Mestre) • Modified percentiles approach, by Stepanek • Using weather types classifications (HOWCLASS), by I. Garcia-Borés, E. Aguilar, • ...

14. Adjusting daily valuesfor inhomogeneities, from monthlyversus dailyadjustments(„delta“ method)

15. Adjustingfrom monthly data • monthly adjustments smoothed with Gaussian low pass filter (weights approximately 1:2:1) • smoothed monthly adjustments are then evenly distributed among individual days

16. Adjusting straight from daily data • Adjustment estimated for each individual day (series of 1st Jan, 2nd Jan etc.) • Daily adjustments smoothed with Gaussian low pass filter for 90 days (annual cycle 3 times to solve margin values)

17. Adjustments(Delta method) • The same final adjustments may be obtained from either monthly averages or through direct use of daily data (for the daily-values-based approach, it seems reasonable to smooth with a low-pass filter for 60 days. The same results may be derived using a low-pass filter for two months (weights approximately 1:2:1) and subsequently distributing the smoothed monthly adjustments into daily values) (1 – raw adjustments, 2 – smoothed adjustments, 3 – smoothed adjustments distributed into individual days), b) daily-based approach (4 – individual calendar day adjustments, 5 – daily adjustments smoothed by low-pass filter for 30 days, 6 – for 60 days, 7 – for 90 days)

18. Spline through monthly temperature adjustments(„delta“ method) • Easy to implement • No assumptions about changes in variance • Integrated daily adjustments = monthly adjustments • But, is it natural?

19. Variable correction • f(C(d)|R), function build with the reference dataset R, d – daily data • cdf, and thus the pdf of the adjusted candidate series C*(d) is exactly the same as the cdf or pdf of the original candidate series C(d)

20. Variable correction • Trewin & Trevitt (1996) method: Use simultaneous observations of old and new conditions

21. Variable correction 1996

22. HSP2 HSP1 The HOM method concept: Fitting a model • Locally weighted regression (LOESS) (Cleveland & Devlin,1998)

23. The HOM method concept: Calculating the binned difference series Decile 10, k=10 Decile 1, k=1

24. The HOM method concept: The binned differences DELLA-MARTA AND WANNER, JOURNAL OF CLIMATE 19 (2006)4179-4197

25. SPLIDHOM (SPLIne Daily HOMogenization), Olivier Mestre • direct non-linearspline regression approach (x rather than a correction based on quantiles), cubic smoothing splines for estimating regression functions

26. Variable correction, q-q function Michel Déqué, Global and Planetary Change 57 (2007) 16–26

27. Our modified percentiles based approach

28. Our percentiles based approach

29. Variable correction methods – complex approach (several elements) • not yet available …

30. Comparison of the methods, ProClimDB software

33. Correction methods comparison

34. PERC EMPIR 0.000 0 0.000 5.000 10.000 15.000 20.000 25.000 30.000 0 5 10 15 20 25 30 -0.100 -0.1 -0.200 -0.2 -0.300 -0.3 -0.400 -0.4 -0.500 -0.5 -0.600 -0.6 -0.700 -0.7 -0.800 -0.8 -0.900 -0.9 HOM SPLIDHOM 0 0 5 10 15 20 25 30 -0.1 -0.1 -0.2 -0.2 -0.3 -0.3 -0.4 -0.4 -0.5 -0.5 -0.6 -0.6 -0.7 -0.7 -0.8 -0.8 -0.9 -0.9 Correction methods comparison, different parameters settings 0 0 5 10 15 20 25 30

35. Correction methods comparison, different parameters settings

36. Correction of daily values • We have some methods … • - but we have to validate them -> benchmark dataset on daily data • Do we know how inhomogeneites in dailydata behave? • we should analyse real data • who and when?, what method for data comparison?