IET International System Safety Conference 2010

1. Unclassified IET International System Safety Conference 2010 Incremental Safety Assurance of an Autonomous Decision-making System for UAS C.A. Lucas*, A. Lyons†, W. Glover*, J.S. Cross* *Agent Oriented Software Limited, †Aero Synergy

2. Outline • Role of safety-critical software in autonomous UAS • The challenge of clearing incremental changes to this software • Evolving regulatory environment • Modular safety case • Icing case study • Conclusions

3. Background • Modern aircraft and engines rely on software for primary air vehicle control, no manual reversion • Consequently FCS or FADEC software is safety critical • Substantial cost and timescale required to introduce software changes, once safety assured • Modifications incur re-certification costs that approach cost of original clearance • Desirable to have a modular software safety assurance process • relating software re-assurance effort to scale of change introduced

4. Implications for Autonomous UAS • Have decision-making software that replaces many functions performed by human crew of a manned or remotely piloted vehicle • Scope extends beyond simply flying the vehicle to a wide range of other critical roles, such as: • detect and avoid • mission management and re-routing • power management • failure detection, analysis and resolution, to Prognostic Health Management • centre of gravity and fuel management • ATC and communications • ground handling

5. Challenge • Many successful manned A/C recognised for their versatility • e.g., C130, dH Mosquito, Tornado • Breadth of capability results from inherent human versatility: • take on new missions if correctly briefed • train as necessary for the new mission • Autonomous UAS must be endowed with versatile behaviours • If adapted to new or different missions then on-board Autonomous Decision-making System (ADMS) has to be upgraded with necessary additional or revised behaviours • Challenge – how can these be safely introduced without requiring the re-generation of evidence for parts that remain unchanged?

6. Approach used and relevance • Strategy to achieve incremental approval of autonomous software systems • Start with clearance of the decision-making software engine (ADMS) • Then incrementally adding behaviours and data to reason with • Critical aspects: • Reasoning engine • Behaviours • Data • Relevant programmes: • SUAV(E) & Taranis • ASTRAEA • Ministry of Defence SSEI

7. Evolving Regulatory Environment • Civil airworthiness regulators starting to address UAS certification • Most UAS are currently operated in combat zones under war-like conditions • Not cleared to any formal set of regulations • Australia’s CASA one of first regulators to introduce civil UAS regulation • CASR, 1998, Part 101, Subpart F specific to UAS • NATO nations recently adopted STANAG 4671 • However 4671 has a number of critical limitations when applied to autonomous vehicles: • Introduction: “These requirements may not be sufficient for the certification of UAV Systems with unconventional, novel or extremely complex features”, and further: “….the following areas are not covered by this airworthiness code: • Airspace integration and segregation of aircraft (including “detect and avoid”), • Non-deterministic flight, in the sense that UAV flight profiles are not pre-determined or UAV actions are not predictable to the UAV crew,…”

8. UK Regulatory Initiatives • 2008 CAA updated CAP722 “Unmanned Aircraft System Operations in UK Airspace – Guidance” • Chapter on autonomy introduced for the first time in a regulatory document • 2009 CAA first released a draft of AMC UAS.1309: “Guidance material for UAS system safety requirements”: • (a) “This Acceptable Means of Compliance (AMC) is designed to set minimum acceptable UAS systems integrity levels in order to protect persons from collisions with UAS. • (b) This AMC is applicable to the Special Conditions defining the system requirements for the collection of systems that perform the functions usually assigned to an aircraft located pilot, in other words the ‘Synthetic Pilot’ (SP)….” • (c) This AMC, a UAS certification document, has been specifically tailored for use with UAS of all sizes …. to ensure the protection of persons in the air and on the ground from the effects of UAS…. • (d) This AMC generally follows the ethos of AMC CS/FAR-25.1309. • (e) The certification basis for UAS will be similar to those for manned aircraft however there will be differences related to the absence of an aircraft-located pilot.”

9. ADMS Basis of Certification –In-flight Icing • Evaluation of requirements for a UAS to fly into known icing conditions chosen as the basis of the study • This case study then used as the vehicle for the study of incrementally certifiable software system

10. In-Flight Icing • Comprised of two scenarios – baseline and an incremented case • Baseline is UAS not cleared for flight into known icing conditions, ADMS will include reasoning to: • Detect and identify possible icing conditions • Take necessary actions to avoid these conditions • Ensure effect of the icing conditions is minimised • Continue mission if possible

11. In-Flight Icing – Increment • Incremented case describes the changes made so that UAS cleared for flight into known icing conditions • Autonomous software will require modification to: • Manage and maintain anti-icing and de-icing mechanisms • Assess the severity of icing conditions and determine if UAS can fly through them or if it should attempt to exit (cf Rex SAAB 340) • Continually monitor ice sensors to determine level of ice accretion • Determine preferred routes both through and out of icing conditions • Process will be developed to ensure that evidence for core autonomous system does not need to be reproduced • Only modified or new modules affected by the changed functionality

12. Software Approach • Approach for incremental certification is:    • Modularisation of safety argumentation, allowing re-use of base modules (e.g. compiler and Run-time Infrastructure (RTI)) for each new system  • Classification of incremental changes leading to different, and potentially limited, needs for re-certification of applications whose base version is itself certified • A language is developed to: • simplify modularisation of multi-agent systems and individual agents,  • provide support for formal analysis of application • Additional tools help evidence collection suitable for incremental certification (non-regression of unchanged functionality)

13. Example reasoning plan

14. Proposed Safety Assurance Process • Establish a basis of certification, agreed with the CAA • Draft a Certification Plan, review with CAA • Functional Hazard Assessment • FMECA • Reliability assessment • Safety targets per AMC (UAS).1309

15. ADMS Basis of Certification – How Established? • Existing code clauses applicable without alteration • Clauses wholly inapplicable to UAS • Clauses with a valid safety objective but which need to be re-written • Wholly new clauses unique to UAS

16. Establish Basis of Safety Assurance • Source Documents: • CS-23 • STANAG 4671 • EU-OPS • Part Ops • Part FCL • AMC (UAS)1309

17. Establish Basis of Safety Assurance- Airframe Codes • CS23 – • EASA code for manned aeroplanes • Normal, Utility, Aerobatic, Commuter

18. Establish Basis of Safety Assurance- Airframe Codes • STANAG 4671 • NATO code for Unmanned Air Systems • Based upon CS23, but 2 issues: • Failure mode severities, no compatibility to manned codes • Ignores any issue that affects a pilot in the manned code

19. UAS Airframe Codes Issues with the STANAG – Example 1 • CS23.773 Pilot’s View: • Clear and undistorted view • Free from Glare • Protected against fog or frost • STANAG says: Not Applicable

20. UAS Airframe Codes Issues with the STANAG – Example 1 • CS(UAS)23.773 Sensor View: • Clear and undistorted view of intended field • Free from Glare • Protected against fog or frost

21. UAS Airframe Codes Issues with the STANAG – Example 2 • CS23.775 (h)(i) Windshields and Windows • (1) Windshield panes directly in front of the pilot(s) in the normal conduct of their duties, and the supporting structures for these panes must withstand, without penetration, the impact of a 0∙91 kg (2 lb) bird when the velocity of the aeroplane relative to the bird along the aeroplane’s flight path is equal to the aeroplane’s maximum approach flap speed.

22. UAS Airframe Codes Issues with the STANAG – Example 2 • USAR 775 says Not applicable • CS (UAS) 23.775 Clause devised by Aerosynergy: • (a) Forward-facing structure, covers and lenses, which directly protect systems and equipment essential for detect and avoid and flight control functions must withstand, without penetration, the impact of a 0∙91 kg (2 lb) bird when the velocity of the aeroplane relative to the bird along the aeroplane’s flight path is equal to the aeroplane’s maximum approach flap speed.

23. What is the Problem with the Existing Codes? - Example • AMC 25.1309 7 (a) 5 Classification of Severity of failure mode: • Catastrophic: Failure Conditions, which would result in multiple fatalities, usually with the loss of the aeroplane. (Note: A “Catastrophic” Failure Condition was defined in previous versions of the rule and the advisory material as a Failure Condition which would prevent continued safe flight and landing.)

24. Hence • Discussions with CAA • Regulatory discussions at the JARUS forum, NAAs plus FAA • Led to: • CAA document AMC UAS 1309

25. AMC UAS 1309 Failure Mode Severities Example: Catastrophic Failure Conditions that could result in multiple fatalities

26. AMC UAS 1309 Main PointsFailure Mode Severities vs. Probability:

27. AMC UAS 1309 Main Points:With D & A; Integrity of Airframe and Complex Flight Mgt System: • AMC UAS 1309 Main Points: • With D & A; Integrity of Airframe and Complex Flight Mgt System:

28. Other Codes: • EU-OPS Part Ops • Operational requirements • Part FCL • Looking at requirements for Flight Crew Licensing to understand what the human pilot is expected to do

29. UAS Airframe Codes Issues with the STANAG - Example 3 • USAR1772 Part time data • USAR1725 Powerplant data etc.. • Puts a lot of emphasis on the UAS Pilot, and this is a problem because: • Estimated Reliability of the data link is 10-3 • Not sufficient for critical functions

30. Firescout Incident According to Capt. Tim Dunigan, Fire Scout program manager, the helicopter was at 1,700 feet AGL 75 minutes into a test flight from its base, Naval Air Station Pax River in Southern Maryland, when operators temporarily lost communications with the unmanned rotary aircraft.

31. Firescout Incident • He added that the MQ-8B Fire Scout has flown more than 1,000 flight hours since December 2006. i.e. 10-3 • Conclusions: • The integrity of the UAS must be on board the UAV • The relevance of the pilot is limited • Autonomous logic is necessary... • ...including ‘interrupts’ to prevent pilot error

32. Concluding • Firescout incident highlights the risks • This paper proposes a means for dealing with autonomous UAS regulation and ADMS safety assurance • Cross fertilisation of results from SSEI Programme to ASTRAEA, with opportunity to flight test ADMS behaviours on Cranfield A/C • Flow through to civil and military UAS programmes worldwide • Civil regulators, CAA and FAA, developing regulations with industry cooperation • What will MoD and the MAA do?