KIM Validation for EO Archived Data Exploitation Support (KIMV)

1. KIM Validation for EO Archived DataExploitation Support (KIMV) Mihai Datcu DLR Oberpfaffenhofen

2. 1997 first tests (ETH Zurich) what’s that?¼ scene • 1999 MMDEMO (ETH Zurich) I2M exists and works!10 scenes • 2002 KIM (ESA, DLR, ETHZ, NREC, EUSC) system, appl. Use20 GB • 2003/4 use KIM in projects: SMART, PRESENCE • 2003/4 Information Theory for evaluation. • 2005 KIMV: operational system, bugs, enhancements accuracy, application scenarios…1 TB, 1m – 1Km, optical and SAR…. • ….more TB…more users…more sensors TerraSAR

3. KIM/KES system concept • Knowledge-driven Information Mining (KIM) • Knowledge enabled services (KES) • KIM and KES are based on Human Centred Concepts • Implements improved feature extraction • search on a semantic level • availability of collected knowledge • interactive knowledge discovery • share knowledge • new visual user interfaces

4. KIM and KES systems • A library of algorithms which is used to extract the primitive features • A machine learning (Bayesian network) algorithm to generate interactively image classifications • A data base management system for the image content information catalogue and semantics and knowledge • The systems are helping the user in his analytical task to extract the information; the system records the knowledge and can reuse or communicate it. In addition, KIM and KES adapt to the user conjecture and are designed to operate very fast on large image volumes.

5. System complexity • KIM/KES integrate • natural language • text • numerical records • GIS • spatial data representation • database and visual capabilities • analysis of multidimensional pictorial structures • computer vision • pattern recognition • relational data models • knowledge representation and bases

6. System complexity Important complexity factors is the unbalanced ratio between the huge informationvolume of EO data (i.e. enormous image archives) and the sequential, mainly linguistic, and limited capacity of people to access information perception of information as ``signals-signs-symbols'' is generally not dependent on the form in which the information is presented but rather on the context in which it is perceived, i.e. upon the intentions and expectations of the perceiver.

7. Validation procedure • objective evaluation of system performance • relevance in real applications with users in the loop, i.e. validation from the “subjective” perspective of the users interested in specific data and applications.

8. The expert evaluators KIM/KES systems respond to existing and new requirements of a very broad range of applications aerospace agencies (ESA, CNES, NASA, DLR) satellite centres (EUSC, ARCS) universities and research units industry data providers

9. The tasks access to information in large EO archives image interpretation understanding phenomena target or objects detection information mining and knowledge discovery

10. Sensor Collection Tiles ERS ers_GEC 571 ERS + LANDSAT Mozambique 448 HYPERSPECTRAL Presence 4 HYPERSPECTRAL/R Smart 24 IKONOS Ikonos 207 IKONOS ikonos_geo 651 LANDSAT Switzerland 184 LANDSAT 5/7 land_IT 468 LANDSAT 5/7 ls_urbex 468 LANDSAT 7 landsat7 168 MERIS meris_120 1099 MERIS MerSelectFrame 1039 MIXED Nepal 188 SPOT 5 cnes_eval 423 SPOT 5 cnes_spot50 45 SPOT 5 cnes_spotM 45 SPOT 5 geo_spot 9 SPOT 5 mihai_spot 9 The data sets

11. The operation modes Content Based Image Retrieval (CBIR) CBIR is based on utilization of semantic queries CBIR enables an operator to “see” into a large volume Data/Information mining explore the information content of the images probabilistic image retrieval integrated with interactive learning and image classification Scene understanding derive knowledge, interpret or understand the structures and objects

12. The questionnaire • evaluation for information retrieval systems, man • machine communication and image classification • rank the user satisfaction on scale with 4 qualitative • values (very good, good, acceptable, uncertain) • semantic differentials for questionnaire-based system • validation (for characterization of the task, search • process, retrieved result and system behaviour) • evaluation of the man-machine communication ( extent • of system functionalities, effects on the user, specific • system like items, and general score) • guideline for a general assessment of the validation • results and suggestions.

13. CBIR results analysis

14. I2M results analysis

15. SU/Classification results analysis

16. Man-Machine Communication

17. MERIS: the classification

18. The method

19. l2_flags classification cloud water land cloud 96% 0,6% 3,4% water 0,1% 99,9% 0% land 20,3% 0,5% 79,2% Results

20. cloud_type cloud water cloud 98% 2% water 0% 100% Results

21. Meris Level 2 Product Cloud Water Land Cloud 97,3% 1,2% 4,3% Water 3,8% 96,2% 0% Land 19,1% 0,3% 80,7% Results

22. Conclusions ·cloud and water the classification given by training the system is more than 90% similar to level 2 product in most of the cases. ·land classification is not as similar as for cloud and water, this is due to two facts: ·land could be covered by cloud land is a very general concept ·ice classification a big difference is detected. level 2 product classification is considering ice over the water and for as it is a classification of snow. snow classification: level 2 product is including the snow in cloud class, meanwhile KIM can separate snow and cloud as two different classes

23. Feature constancy (data models) • Gemetry (HR vs. LR) • SPOT (CNES) data quality • # semantic labels grows with higherresolution • MERIS Landsat ERSSPOTIKONOS • 10 10010 k1001000