250 likes | 360 Views
Measurements & Validation. R G Stamp Head of Department of ATM Research National Air Traffic Services Ltd, UK. What is Validation?. EATMS Validation Strategy (EVAS)
E N D
Measurements & Validation R G Stamp Head of Department of ATM Research National Air Traffic Services Ltd, UK
What is Validation? • EATMS Validation Strategy (EVAS) “The process through which a desired level of confidence in the ability of a deliverable to operate in a real-life environment may be demonstrated against a pre-defined level of functionality, operability and performance”
What is Validation? • EATMS Validation Strategy (EVAS) “The process through which a desired level of confidence in the ability of a deliverable to operate in a real-life environment may be demonstrated against a pre-defined level of functionality, operability and performance”
What is Validation? • EATMS Validation Strategy (EVAS) “The process through which a desired level of confidence in the ability of a deliverable to operate in a real-life environment may be demonstrated against a pre-defined level of functionality, operability and performance”
What is Validation? • EATMS Validation Strategy (EVAS) “The process through which a desired level of confidence in the ability of a deliverable to operate in a real-life environment may be demonstrated against a pre-defined level of functionality, operability and performance”
What is Validation? • EATMS Validation Strategy (EVAS) “The process through which a desired level of confidence in the ability of a deliverable to operate in a real-life environment may be demonstrated against a pre-defined level of functionality, operability and performance” • Validation builds confidence that the system is fit for purpose
How to Validate Analytic modelling Fast-time Field test simulation Shadow-mode Small-scale trial real-time simulation Operational trial Operations Large-scale real-time simulation
Where to Validate Small-scale real-time simulation
Where to Validate NATS DLR CENA NLR EEC real-life environment Small-scale real-time simulation
Validation Methodology • Consistent Application of Validation Methodology Across All Sites
Validation Methodology • Consistent Application of Validation Methodology Across All Sites
Validation Methodology • Consistent Application of Validation Methodology Across All Sites
Validation Methodology • Consistent Application of Validation Methodology Across All Sites
Validation Methodology • Consistent Application of Validation Methodology Across All Sites
Validation Methodology • Consistent Application of Validation Methodology Across All Sites pre-defined levelof functionality, operability and performance
Measures of Merit • MoM’s Chosen to Test Hypotheses • Controller Workload • Airspace Capacity • Quality of Service • Usability • Safety • Measurements Taken: • e.g. Workload: ISA, TLX, R/T, ….. • Objective; Subjective; Observed
Analysis • Descriptive Statistics
Analysis • Descriptive Statistics • Inferential Statistics • significant at 5% confidence level
Analysis • Descriptive Statistics • Inferential Statistics • significant at 5% confidence level • Backed up by Controller Subjective Comments • “I can still handle the aircraft while doing the crossword!”
Analysis • Descriptive Statistics • Inferential Statistics • significant at 5% confidence level • Backed up by Controller Subjective Comments • “I can still handle the aircraft while doing the crossword!” desired level of confidence
Effectiveness of PHARE Validation • Core Validation Group with Representation from all Participating Sites • Common Methodology Crucial • Methodology Improved with Experience • Validation Conducted by Individual Sites • Lessons Re-learnt &/or Lost • Effectiveness Diminished • Validation Should be Applied Throughout Project - Not Just at the End
Summary • Validation Methodology Major Contributor to Credibility of PHARE • Forms the Backbone for much of the Validation Activities within European ATM R&D
Summary • Validation Methodology Major Contributor to Credibility of PHARE • Forms the Backbone for much of the Validation Activities within European ATM R&D