Accuracy of ESBD in the Alps: Analysis and Evaluation

1. Alpine Soil Information System Analysis of the accuracy of ESBD in the Alps region Luis Rodríguez Lado E-mail : luis.rodriguez-lado@jrc.it JRC – Ispra – 23 July 2004

2. Introduction There is a increasing demand of soil maps and of their properties in the frame of the EU. This information is needed to develop policies linked to sustainable land management practices, and to avoid the damage risk to ecosystems. At present, the 1:1M digital soil map and of some of their properties are available at European Scale. JRC – Ispra – 23 July 2004

3. Objective In this exercise, we evaluate the accuracy of the ESDB maps by comparison with some reference maps derived from detailed survey (ECALP). JRC – Ispra – 23 July 2004

4. Methodologydata from ECALP areas Accurate digital soil maps were computed for 5 pilot areas in the Alps region (ECALP Project). JRC – Ispra – 23 July 2004

5. Methodologydata from ECALP areas Maps in the ECALP areas are available as to raster based soil maps. The pilot areas were divided in 1Km2 cells. In this analysis, the soil properties used for each cell are those of the main Soil Map Unit in the cell (% area). JRC – Ispra – 23 July 2004

6. Methodologydata from ESDB areas The 1:1M ESDB was rasterized into a 1Km2 cell raster grid. The soil properties for each grid cell were also those of the main Soil Map Unit in the cell (% area). We compare the results of both maps. JRC – Ispra – 23 July 2004

7. Methodologyproperties analyzed Texture. Depth of presence of an obstacle to roots. Depth of presence of an impermeable layer. JRC – Ispra – 23 July 2004

8. Methodology The accuracy of the 1:1 M map is expressed by “naïve” measures of accuracy using “confusion matrices”. JRC – Ispra – 23 July 2004

9. Objective JRC – Ispra – 23 July 2004

10. Methodology The User’s accuracy expresses the probability that one class (in ESDB) is well mapped in relation to the reference dataset (ECALP). The Producer’s accuracy indicates the proportion of cells that were correctly classified. JRC – Ispra – 23 July 2004

11. Objective JRC – Ispra – 23 July 2004

12. Methodology The Overall accuracy is the sum of the correctly classified cells (diagonal values) divided by the total number of cells analyzed. It indicates the proportion in which those maps agree. The KAPPA coefficient of agreement is a measure of the chance in the agreement. It indicates whether the agreements found in the overall accuracy are due to the map accuracy of due to chance. JRC – Ispra – 23 July 2004

13. Objective JRC – Ispra – 23 July 2004

14. Methodology For example: An Overall Accuracy of 0.655 indicate that both maps agree in 65% of the cases. A Kappa statistic of 0,557 indicates that 55,7% of this agreement is due to the mapper competency, and 9,3% of the agreements were due to chance. JRC – Ispra – 23 July 2004

15. Methodology Low values of Kappa indicate : a) Bad Map. Errors due the mapper or to the mapping technique. We can do another map with the same accuracy simply by random assignation using the same classes. b) An highly homogeneous area (1 class in whole area).For these areas, high values of agreement can be achieved also randomly. JRC – Ispra – 23 July 2004

16. Results JRC – Ispra – 23 July 2004

17. Resultsfrequency distribution (n = 1818 cells) Texture JRC – Ispra – 23 July 2004

18. confussion and probabilities matrices; Accuracy index Texture JRC – Ispra – 23 July 2004

19. Texture class Lombardia-Switzerland (ECALP) (ESDB) JRC – Ispra – 23 July 2004

20. confussion and probabilities matrices; Accuracy index Texture Lombardia JRC – Ispra – 23 July 2004

21. Texture class Friuli-Slovenia (ECALP) (ESDB) JRC – Ispra – 23 July 2004

22. confussion and probabilities matrices; Accuracy index Texture Friuli JRC – Ispra – 23 July 2004

23. Conclusions Texture JRC – Ispra – 23 July 2004

24. Conclusions Texture JRC – Ispra – 23 July 2004

25. Conclusions Texture JRC – Ispra – 23 July 2004

26. Resultsfrequency distribution (n = 1818 cells) Depth of an obstacle for roots JRC – Ispra – 23 July 2004

27. confussion and probabilities matrices; Accuracy indexes Depth of an obstacle for roots JRC – Ispra – 23 July 2004

28. Depth to obstacle to roots Lombardia-Switzerland (ECALP) (ESDB) JRC – Ispra – 23 July 2004

29. Conclusions Obstacle to roots JRC – Ispra – 23 July 2004

30. Conclusions Obstacles to roots JRC – Ispra – 23 July 2004

31. Conclusions Obstacles to roots JRC – Ispra – 23 July 2004

32. Resultsfrequency distribution (n = 1818 cells) Depth of an impermeable layer JRC – Ispra – 23 July 2004

33. Resultsconfussion and probabilities matrices; Accuracy index (n = 1818 cells) Depth of an impermeable layer JRC – Ispra – 23 July 2004

34. Conclusions Depth of an impermeable layer JRC – Ispra – 23 July 2004

35. Conclusions We found that the present 1:1M ESDB maps means a great generalization of soils and their properties, being inappropriate to derive effective policies in the EU at medium and large scales due to the uncertainty of its information. The overall accuracy of these maps is generally lower than 50%. It varies between 0,33 (obstacle to roots) to 0,8 (depth of impermeable layer) but low values of Kappa were found, indicating high influence of chance in the success of classification. This low values of Kappa are greatly due to the low discrimination in classes in ESDB (general map). JRC – Ispra – 23 July 2004

36. Conclusions Friuli-Slovenia was the region that showed a better agreement with the ECALP database, particularly for the depth of an obstacle to roots, where it also exhibits a high value of Kappa. JRC – Ispra – 23 July 2004

37. Conclusions • Need of more accurate soil maps than ESDB • Provide soil sample description as metadata • Consensus in the description of properties • Implementation of accuracy tests for maps JRC – Ispra – 23 July 2004