1 / 24

Measurements & Validation

Learn about the validation methodology for the European Air Traffic Management System (EATMS), which demonstrates the system's ability to operate effectively in real-life scenarios. Discover the different validation techniques, measurements of merit, and the importance of consistent application of the methodology across all sites.

piccirillo
Download Presentation

Measurements & Validation

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Measurements & Validation R G Stamp Head of Department of ATM Research National Air Traffic Services Ltd, UK

  2. What is Validation? • EATMS Validation Strategy (EVAS) “The process through which a desired level of confidence in the ability of a deliverable to operate in a real-life environment may be demonstrated against a pre-defined level of functionality, operability and performance”

  3. What is Validation? • EATMS Validation Strategy (EVAS) “The process through which a desired level of confidence in the ability of a deliverable to operate in a real-life environment may be demonstrated against a pre-defined level of functionality, operability and performance”

  4. What is Validation? • EATMS Validation Strategy (EVAS) “The process through which a desired level of confidence in the ability of a deliverable to operate in a real-life environment may be demonstrated against a pre-defined level of functionality, operability and performance”

  5. What is Validation? • EATMS Validation Strategy (EVAS) “The process through which a desired level of confidence in the ability of a deliverable to operate in a real-life environment may be demonstrated against a pre-defined level of functionality, operability and performance”

  6. What is Validation? • EATMS Validation Strategy (EVAS) “The process through which a desired level of confidence in the ability of a deliverable to operate in a real-life environment may be demonstrated against a pre-defined level of functionality, operability and performance” • Validation builds confidence that the system is fit for purpose

  7. How to Validate

  8. How to Validate Analytic modelling Fast-time Field test simulation Shadow-mode Small-scale trial real-time simulation Operational trial Operations Large-scale real-time simulation

  9. Where to Validate Small-scale real-time simulation

  10. Where to Validate NATS DLR CENA NLR EEC real-life environment Small-scale real-time simulation

  11. Validation Methodology • Consistent Application of Validation Methodology Across All Sites

  12. Validation Methodology • Consistent Application of Validation Methodology Across All Sites

  13. Validation Methodology • Consistent Application of Validation Methodology Across All Sites

  14. Validation Methodology • Consistent Application of Validation Methodology Across All Sites

  15. Validation Methodology • Consistent Application of Validation Methodology Across All Sites

  16. Validation Methodology • Consistent Application of Validation Methodology Across All Sites pre-defined levelof functionality, operability and performance

  17. Measures of Merit • MoM’s Chosen to Test Hypotheses • Controller Workload • Airspace Capacity • Quality of Service • Usability • Safety • Measurements Taken: • e.g. Workload: ISA, TLX, R/T, ….. • Objective; Subjective; Observed

  18. Analysis • Descriptive Statistics

  19. Analysis • Descriptive Statistics • Inferential Statistics • significant at 5% confidence level

  20. Analysis • Descriptive Statistics • Inferential Statistics • significant at 5% confidence level • Backed up by Controller Subjective Comments • “I can still handle the aircraft while doing the crossword!”

  21. Analysis • Descriptive Statistics • Inferential Statistics • significant at 5% confidence level • Backed up by Controller Subjective Comments • “I can still handle the aircraft while doing the crossword!” desired level of confidence

  22. Effectiveness of PHARE Validation • Core Validation Group with Representation from all Participating Sites • Common Methodology Crucial • Methodology Improved with Experience • Validation Conducted by Individual Sites • Lessons Re-learnt &/or Lost • Effectiveness Diminished • Validation Should be Applied Throughout Project - Not Just at the End

  23. Summary • Validation Methodology Major Contributor to Credibility of PHARE • Forms the Backbone for much of the Validation Activities within European ATM R&D

  24. Summary • Validation Methodology Major Contributor to Credibility of PHARE • Forms the Backbone for much of the Validation Activities within European ATM R&D

More Related