Outlier Detection in Geodetic Applications: Analysis of Observation Imprecision

1. Title Outlier Detection in Geodetic Applicationswith respect to Observation Imprecision Ingo Neumann and Hansjörg Kutterer Geodetic Institute University Hannover Germany Steffen Schön Engineering Geodesy and Measurement Systems Graz University of Technology Austria REC 2006

2. Contents • Motivation • Uncertainty modeling in geodetic data analysis • Statistical hypothesis tests in case of imprecise data • -One-dimensional case • -Multi-dimensional case • Geodetic applications • -Global testin least squares adjustment • - Outlier detection (e.g., GPS-baseline) • Conclusions and future work

3. Motivation • Tasks and methods • Determination of relevant quantities / parameters • Calculation of observationimprecision • Propagation of observationimprecision to the est. parameters • Assessment of accuracy (imprecise case) • Regression and least squares adjustments • Statistical hypothesis tests • Optimization of configuration

4. Systematic effects Motivation • Measurement process: • Stochasticity • Observation imprecision • (Outliers) • Model uncertainty, object fuzziness, etc. Focus in this presentation: Stochastics (Bayesian approach) Interval mathematics Fuzzy theory

5. Solution: Describing the influence factors for the preprocessing step of the originary observation with fuzzy sets a Uncertainty modeling in geodetic data analysis Requirements: • Adequate description of Stochastics • Adequate description of Imprecision e. g., LR-fuzzy-number

6. - Instrumental error sources - Uncertainties in reduction and corrections Influence factors (p) - Rounding errors Linearization Partial derivatives for all influence factors Imprecision of the influence factors Uncertainty modeling in geodetic data analysis Sensitivity analysis for the calculation of observation imprecision:

7. with The fuzzy sets of the observation are splitted in a centre ( ) and radius ( ) part for a sufficient number of a-cuts. Uncertainty modeling in geodetic data analysis Sensitivity analysis for the calculation of observation imprecision: The sensitivity of the observations as a result of the preprocessing steps

8. 1 Stochastics (Bayesian approach) Observation imprecision • Propagation of observationimprecision to the estimated parameters Uncertainty modeling in geodetic data analysis • Tasks and methods (Special case of Random-Fuzzy) • Determination of relevant quantities / parameters • Calculation of observationimprecision

9. Statistical hypothesis tests in case of imprecise data Precise case (1D) 1 Example: Two-sided comparison of a mean value with a given value Clear and unique decisions ! x Null hypothesis H0, alternative hypothesis HA, error probability g → Definition of regions of acceptance A and rejection R

10. 1 x Imprecision of test statistics due to the imprecision of the observations Statistical hypothesis tests in case of imprecise data Consideration of imprecision Imprecise case Precise case 1 x

11. Statistical hypothesis tests in case of imprecise data Consideration of imprecision Precise case Imprecise case 1 1 x x Imprecision of the region of acceptance due to the linguistic fuzziness or modeled regions of transition Fuzzy-interval

12. Statistical hypothesis tests in case of imprecise data Consideration of imprecision Precise case Imprecise case 1 1 x x Imprecision of the region of rejection as complement of the region of acceptance

13. Statistical hypothesis tests in case of imprecise data Consideration of imprecision Precise case Imprecise case 1 1 x x Conclusion: Transition regions prevent a clear and unique test decision !

14. Statistical hypothesis tests in case of imprecise data Conditions for an adequate test strategy • Quantitative comparison of the imprecise test statistics and the regions of acceptance and rejection • Precise criterion pro or con acceptance • Probabilistic interpretation of the results

15. Statistical hypothesis tests in case of imprecise data Basic idea Degree of disagreement Degree of agreement Considered alternatives height criterion card criterion

16. Degree of agreement Degree of disagreement Degree of rejectability Statistical hypothesis tests in case of imprecise data Test decision:

17. with: Statistical hypothesis tests in case of imprecise data The height criterion: ~ ~ ~

18. Overlap region Statistical hypothesis tests in case of imprecise data The card criterion: ~ ~ ~ with:

19. degree of rejectability for the card criterion degree of rejectability for the height criterion Statistical hypothesis tests in case of imprecise data Test situation with tight bounds (weak imprecision): ~ ~ ~ ~

20. degree of rejectability for the card criterion degree of rejectability for the height criterion Statistical hypothesis tests in case of imprecise data Test situation with wide bounds (strong imprecision): ~ ~ ~ ~

21. Multidimensional hypothesis tests in Geodesy Test situation and test value without imprecision: precise case n:= number of observations u:= number of parameters d:= rank deficiency of the normal equations matrix

22. n:= number of observations u:= number of parameters d:= rank deficiency of the normal equations matrix f = n -u+d:= degrees of freedom Multidimensional hypothesis tests in Geodesy with precise case and

23. Hypotheses: 1 precise case x Multidimensional hypothesis tests in Geodesy Test decision: (g:=error probability)

24. Search the smallest and largest element s for for a sufficient number of a-cuts Optimization algorithm Multidimensional hypothesis tests in case of imprecise data Strict realization of Zadeh‘s extension principle!

25. Multidimensional hypothesis tests in case of imprecise data a-cut optimization for a 2-dimensional point test:

26. Multidimensional hypothesis tests in case of imprecise data a-cut optimization for a 2-dimensional point test:

27. Multidimensional hypothesis tests in case of imprecise data Resulting test scenario  1D comparison Final decision based on height or card criterion

28. monitoring the actual movements of the lock: Applications A geodetic monitoring network of a lock: The lock Uelzen I Monitoring network

29. OUTLIERS in the collected measurements! Remove the OUTLIERS from the collected measurements, because they may falsify point coordinates! Statistical hypothesis tests in case of imprecise data Applications A geodetic monitoring network of a lock: n = 313 observations u = 45 parameters d = 3 datum defects

30. Applications Global test in least squares adjustment

31. Applications GPS-baseline test (907-908)

32. Applications GPS-baseline test (907-908)

33. Conclusions and future work • Statistical hypothesis tests can be extended for imprecise data • Degrees of agreement and disagreement • Degree of rejectability  comparison of fuzzy sets • 1D case is straightforward, mD case needs a-cut optimization • card criterion more adequate than (easier-to-apply) height crit. • Not shown but computable: Type I and Type II error probs. • Not shown but available: Extended regression and optimization • In progress: Assessment and validation using real data

34. Acknowledgements The presented results are developed withinthe research project KU 1250/4-1 ”Geodätische Deformationsanalysen unter Verwendung von Beobachtungs-impräzision und Objektunschärfe”, which is funded by the German Research Foundation (DFG). This is gratefully acknowledged. The third author stays as a Feodor-Lynen-Fellow with F. K. Brunner at TU Graz, Austria. He thanks his host for giving the possibility to contribute to this study and the Alexander von Humboldt Foundation for the financial support. Thank you for your attention!