Operational Considerations in the Certification of Technically Advanced Aircraft (TAA)

1. Operational Considerations in the Certification of Technically Advanced Aircraft (TAA) Robert Barnes, Robert B Barnes Associates, Inc. (USA) Charles Adam, Federal Aviation Administration (USA)

2. Today’s topics • The need to consider the user • What we’ve seen during flight test • Current policy • Current design practice • Importance of real user trials • Our suggestions

3. “General aviation today is experiencing the most rapid pace of innovation since the late 1940s … … These advances offer general aviation pilots the promise of increased levels of safety and performance … … Unfortunately, these increased levels of safety have not materialized.” -- Littlefield, Glass Cockpit Flying, 2010

4. “Why has the promise of greater levels of safety for glass cockpit airplanes not been realized? Because, in order to realize these benefits, general aviation pilots must learn a new way of flying. Unfortunately, general aviation pilots and training providers have not yet evolved the way they train and fly to catch up with the advances in glass cockpit technology.” -- LinkedIn Aviation Training Group Discussion

5. As seen during flight test • Design may be functionally acceptable to engineers • Is the design acceptable to properly qualified pilots? • Can a crew safely operate new design?

6. Some personal findings • Engineers tend to design systems which may seem logical and intuitive to them (an engineering skill) • Sequences of steps • Multifunction control logic • These same systems may be far from logical or intuitive to the typical pilot (a pilot skill) • Recognizing operation of dedicated controls

7. Effectively demonstrating design acceptability requires: Involvement of qualified pilots … Representing a cross-section of the target pilot population … Who are operating the new design under realistic conditions.

8. Human Factors in Part 23 Aircraft Certification • FAA Certification Policy on Human Factors (PS-ACE100-2001-004, Aug 2002) • Response to industry requests for guidance • Advocates early FAA involvement during development • Goal: To aid in the timely identification and resolution of human factors-related issues during the aircraft certification process • Guidance only … not a regulatory requirement • “General Aviation fatalities (part 91 operations) occur at higher rates than part 121 and part 135 operations.”

9. Operational Considerations • Pilot characteristics • Operational considerations • System operation procedures • Training requirements

10. FAA Policy on Human Factors • Not a regulatory requirement but FAA ACO “should attempt” to follow • Increased effort being made to address design-related human error • FAA rulemaking effort (25.1302) already being applied by EASA • Human factors issues are open to broad interpretation • Component level • Operational level

11. Human Factors Design Practice • Design and Operational factors affect workload • There are many ways to evaluate these affects • 500 pages of methods described • User trials are on page 475

12. Some user trial advantages • Simplistic and flexible approach to usability evaluation • Evaluates the system based upon the potential end users’ performance • Gives the designers a powerful insight into how the system will be used

13. Some user trial disadvantages • Time consuming to conduct • Large amounts of data are collected, ensuring a lengthy analysis process • It may be difficult to gain access to the required personnel or end users • End users may be biased towards the old system or procedure

14. Applying User Trials in U.S. • Limited form now used periodically by FAA for avionic component evaluations • Multiple Pilot System Usability Evaluation (MPSUE) • FAA pilots not typical end users Does this adequately evaluate operation by the typical user?

15. TAA certification user trial • Removing restrictions required a human factors workload evaluation • Applicant chose to use Modified Pilot Subjective Evaluation (MPSE) • Two typical user pilots were qualified • Certification was successful but many lessons were learned by manufacturer, training provider, and regulator

16. Conclusions • Human factors evaluation is subjective • Level of pilot competence required to operate design is critical • Using experienced test pilots is not enough “Is the aircraft capable of being operated safely by the manufacturer’s target customer with the type of training proposed?”

17. Questions/Comments? For a copy of our papers, please leave me your card