Safety case development in ATM R&D

1. Safety case development in ATM R&D Safety feedback for decision-makers and concept developers Episode 3 - CAATS II Final Dissemination Event Jelmer J. Scholte NLR-ATSI CAATS II Brussels, 13 & 14 Oct 2009

2. Contents • Motivation • Safety case contents • Practical development of safety case • Concluding remarks Episode 3 - CAATS II Final Dissemination Event

3. History (1/2) Accident statistics of Large Aeroplane flights in commercial aviation Accidents Fatal Accidents Fatalities 1980-1999 period 2340 613 15,554 Average per year 117 30.7 777.7 Average per flight 5.57 E-6 1.46 E-6 37.0 E-6 Separation related 7.9% 3.75% 5.0% Source: NLR-ATSI’s Air Safety Data Base Episode 3 - CAATS II Final Dissemination Event

4. History (2/2) Episode 3 - CAATS II Final Dissemination Event

5. Current picture • It is good practice for an ANSP to develop a safety case for implementation of changes to its ATM system • to fulfill its own objectives and responsibilities • to satisfy safety regulations • Several safety regulations and methods are in use that were developed for use by an ANSP for changes to its ATM system • ESARR 4 • EC regulation 2096/ 2005 • EATMP ANS Safety Assessment Methodology (SAM) • Eurocontrol Safety Case Development Manual (SCDM) Episode 3 - CAATS II Final Dissemination Event

6. A practical example (1/2) Independent paralleldepartures on SIDs Episode 3 - CAATS II Final Dissemination Event

7. A practical example (2/2) • Key hazards in cockpit and at ATC • Crew makes error in entering the SID in FMS • ATC fails to communicate a late SID change to aircraft • ATC-published SID design entered wrongly in database • Resolution of conflicts involves ATCo and pilots • ATCo cannot solve the conflict without pilot • Pilot may correct SID errors independently • Timing of pilot’s R/T frequency change from TWR to APP • Challenge: • The role of the airline and the pilots is crucial • Focusing on ANSP is not desired! Episode 3 - CAATS II Final Dissemination Event

8. Future challenges (1/2) Episode 3 - CAATS II Final Dissemination Event

9. Future challenges (2/2) Episode 3 - CAATS II Final Dissemination Event

10. Example solutions proposed • Reference business trajectories • Functional airspace blocks • Flexible use of airspace • ASAS applications • Reduced separation criteria • ... Episode 3 - CAATS II Final Dissemination Event R&D required to tackle the major design hurdle faced!

11. Implemented Idea Concept V 0 V 1 V 2 V 3 V 4 V 5 ATM Needs Scope Feasibility Integration Pre-operation Operation Integrate concept in wider context And confirm performance Iteratively develop and evaluate concept Identify ATM performance needs & constraints Industrialisation and procedure approval Scope operational concepts and create validation strategy Implementation E-OCVM (1/2) • E-OCVM to support effective R&D: “... the process whereby the many stakeholders eventually should come to a decision to either: • Continue development to ... or • stop or substantially modify developments...” Episode 3 - CAATS II Final Dissemination Event

12. E-OCVM (2/2) • E-OCVM poses specific, new requirements to safety case development • Feedback to stakeholders! Episode 3 - CAATS II Final Dissemination Event

13. Summary of motivation • Most safety regulations & methods were developed for use by ANSP for changes to its ATM system • Major changes to air traffic operations are needed to maintain an acceptable level of safety • ambitious targets in multiple KPAs • large number of stakeholders involved • Major changes require R&D supported by safety analysis • E-OCVM is the framework for validation of these major changes • E-OCVM poses specific, new requirements to safety case development Episode 3 - CAATS II Final Dissemination Event

14. Contents • Motivation • Safety case contents • Practical development of safety case • Concluding remarks Episode 3 - CAATS II Final Dissemination Event

15. Safety analysis feedback to design Design Analysis Episode 3 - CAATS II Final Dissemination Event

16. Safety analysis tailoredto maturity • The aim of safety analysis changes from V1 to V5 Safety feedback to design Safetyassurance Episode 3 - CAATS II Final Dissemination Event V5 V1

17. Safety analysis objectives per phase Episode 3 - CAATS II Final Dissemination Event

18. Safety analysis methods Safety case development in R&D has been subject of a lot of recent research • Experiences with developing a safety case in E-OCVM are just building up • Large design challenges pose several new needs to safety case development in R&D • Several complementary approaches are emerging that aim to address the SESAR-identified emerging needs • Integration so far limited Episode 3 - CAATS II Final Dissemination Event

19. SESAR-identified emerging needs • The need for a ‘macro’ safety case • The need to address safety regulations • The need to address the multi-stakeholder nature of advancing air traffic operations • The need to address the success side of a change also • The need to cover human operators in the ATM system • The need to identify unknown ‘emergent’ risks • The need to address E-OCVM requirements • The need to assess concept maturity • The need for managing relations between cases Episode 3 - CAATS II Final Dissemination Event

20. A: The need for a ‘macro’ safety case • Motivation: • SESAR consists of multiple local changes by various stakeholders. • Example: Functional Airspace Blocks • Includes many smaller changes • Identified approaches: • Connect to an overall incident-accident model • Apportioned safety criteria based on statistics • ‘Joint safety analysis’ Episode 3 - CAATS II Final Dissemination Event

21. B: The need to address safety regulation • Motivation: “Developing the ATM safety regulatory framework is essential to the success of SESAR” • Example: ASAS applications • Responsibilities transfer from ground to cockpit • ESARR 4 applied to airline? • Identified approaches: • Early scanning of concepts on fundamental safety issues including existing safety regulations • Address impact of changed regulations in early safety analysis • Safety assessment assuming current regulations, while keeping track of needs for changes Episode 3 - CAATS II Final Dissemination Event

22. C: The need to address the multi-stakeholder nature • Motivation • SESAR will fundamentally change stakeholder roles • Example: FABs • Who manages traffic? • Who is responsible? • Who decides on acceptability of risk? • Identified approach: • Safety validation framework with active roles to be played by all stakeholders • joint goal oriented approach • joint safety validation Episode 3 - CAATS II Final Dissemination Event

23. D: The need to address the success side of a change also • Motivation: • Safety assessments have often focused on failure • ICAO has always asked to address the success side also • Example: TCAS RA downlink • Focus on failure of downlink? • What if downlink successful? • Identified approaches: • Integrated safety analysis covering both failures and successes • Complement traditional ‘failure approach’ with dedicated ‘success approach’ Episode 3 - CAATS II Final Dissemination Event

24. H: The need to assess concept maturity • Motivation: How to decidewhether a concept is readyfor next E-OCVM phase? • Example: individual SESAR development projects • Identified approaches: • Generic SARD criteria (Strategic Assessment of ATM R&D) • Safety case specific set by CAATS II in SARD update • Safety case specific set by EEC (for ‘SAME’) Episode 3 - CAATS II Final Dissemination Event

25. I: The need for managing relations between cases • Motivation: effectiveness and efficiency • Example: use of real-time simulations • Can multiple cases benefit? • Identified approaches: • Safety & HF: share info where useful, disjoint where needed • Safety & environment: disjoint analyses • Safety providing input to business • Framework for managing relations between cases Episode 3 - CAATS II Final Dissemination Event

26. Contents • Motivation • Safety case contents • Practical development of safety case • Concluding remarks Episode 3 - CAATS II Final Dissemination Event

27. Basic steps • Select the phase of E-OCVM’s Concept Lifecycle Model to be tackled • Determine objective and scope of safety analysis in line with the selected phase • Determine methods and techniques to be used • Document the results Episode 3 - CAATS II Final Dissemination Event

28. Selection of methods/ techniques • Develop expertise and practical experience with emerging methods • Work on integration of emerging methods to combine their strong points • There are complementary needs of • advanced safety courses and • hands-on safety learning • Get an expert aware of these emerging needs, and with experience with emerging approaches! Episode 3 - CAATS II Final Dissemination Event

29. Documentation • ‘Negative’ analysis results have great value as feedback to design • In R&D, the value is in the explanation why a concept is not yet valid or safe Episode 3 - CAATS II Final Dissemination Event Validation is most of the time invalidation Only the last cycle is validation!

30. Contents • Motivation • Safety case contents • Practical development of safety case • Concluding remarks Episode 3 - CAATS II Final Dissemination Event

31. Concluding remarks • Experiences with developing a safety case in E-OCVM are just building up • Several needs are emerging for safety case development for large design challenges, as traditional approaches fall short • Several complementary approaches have been identified that aim to address the SESAR-identified emerging needs • Key focus points: • Gain experience with emerging complementary approaches • Integration of emerging complementary approaches Episode 3 - CAATS II Final Dissemination Event

32. Questions? Episode 3 - CAATS II Final Dissemination Event Brussels, 13 & 14 Oct 2009