Training Situation Assessment and Decision Making A rule making approach

1. Training Situation Assessment and Decision Making A rule making approach Rene Amalberti, MD, PhD Professor of Medicine, Physiology and ergonomics, Head Cognitive Science Department, IMASSA, Test-flight center, Brétigny-sur-Orge Ramalberti@imassa.fr Amalberti

2. Outline of the presentation • Basic Assumptions on Training and Decision Making in Aviation • The Reality gap • Tactical and strategic options for rule making • The JAA approach Amalberti

3. Basic Assumptions • Situation assessment and decision making are priorities for flight safety (CAST, JSSI, ALAR) • Specific training is needed Amalberti

4. Situation Assessment and Decision Making are priorities for Safety (CAST, JSSI, ALAR Fatalities by Accident Categories Fatal Accidents, Worldwide Commercial Jet Fleet (1990-1999) Approach & Landing Accident Reduction Amalberti

5. Approach & Landing Accident Reduction • Challenge or refuse ATC instructions when they are not clearly understood, are questionable or conflict with your assessment of aircraft position relative to the terrain. • Exercise good radio communication discipline. • Know the height of the highest terrain or obstacle in the operating area. • Know your aircraft’s position in relation to the surrounding high terrain. ATC

6. Approach & Landing Accident Reduction • Monitor the autoflight system for desired operation. • Use the best available mode for current flight conditions. • Follow procedures. • Monitor navigation performance. Autoflight System

7. CRM, SOPs and Training Factors Involved in ALAs Approach & Landing Accident Reduction • 74% - Inadequate crew decision making • 72% - Inadvertent non adherence to procedures • 63% - Failure in CRM (cross-check/coordination) • 46% - Failures in company management • 40% - Deliberate non adherence to procedures • 37% - Inadequate training CRM = Crew resource management SOPs = Standard operating procedures Amalberti

8. The reality gap • Training time costs money, and tends to remain as short as possible • Consistent plan for integrated human factors training, conceptually and practically hard to design • Persistent false believes on cognition, • human behaviour, and human error • Training evaluation for human factors related aspects hard to accept Amalberti

9. Reduction time in training courses Feedback from the FAA/JAA report on The Interfaces Between Flightcrews and Modern Flight Deck Systems, June 1996 • Reduction time • Strong belief (evidence?) that learning cognitive skills needs a shorter time compared to learning motor skills • Back to the minimum • Ambiguities for decision making Amalberti

10. Persistent false belief on Human behaviour and human error • Training priorities not necessary the ones identified • False belief regarding « back to manual » • Remaining language problems aggravating situation awareness • Effects of complexity on pilots’ expertise Amalberti

11. Speed corruption Mach corruption Altitude corruption Altitude preset ineffective Automatic reversion of descent mode between Heading-Vertical Speed (Hdg-Vs) and Track-Flight-path angle (TK-FPA) and versus. Poor situation awareness and false expectation on crew behaviour • 10 crews experiencing rare software failures • Modification of Flight Simulator Capacities • LOFT scenario: LYON - MADRID Amalberti

12. Experimental crew training to deal with automation surprises (Results 1) • All crews landed safely • Only four crews went back to manual • All crews showed a tendency to cycle the failing system several times to test the failure before trying to reset the system in order to recover it (resetting fuse, proceeding the global function reset, e.g. resetting FMGS) • A thorough analysis of crews’ mental representation and situation awareness via verbal protocols analysis show a series of unexpected ‘magic’ thinking on sub-systems organisation and inter-relations. • Crews gave priority to the situation management, not to optimal error or failure management. HE rate increases to 50%, error detection rate going down to 30%, only 10% of HE properly recovered. • No causal reasoning, search for comprehension collectively refrained • First priorities for safe situation management : control the very short term, and search for solutions compliant with the goal. Amalberti

13. Complexity, and Expertise Virtual space of total system knowledge Knowledge at the end of A/C type-rated Initial phase Routines Exploration phase Cognition in the wild Retraction phase Core of routines Glass-cockpit a/c, Amalberti, 1996, 2000 Amalberti

14. STATUS INOP SYS WINDSHEAR DETATT LIMIT OVSPD LIMIT ADR2+  RA1+2 SPLR 1+2+5 ELAC 2 SEC 2+3 ILS 2 FAC 1+2A/CALL OUTVHF 2ACP 3CAPT STATCAPT TATWSHLD HEAT APPR PROC: -L/G……………ON -APPR NAVAID….ON RMP1 APPR SPD: VREF +10KT LDG DIST…………X 1.55 ENG 1 APPR IDLE ONLY ENG 2 APPR IDLE ONLY ALTN LAW: PROST LOST WHENL/G ON:DIRECT LAW CTR TK FUEL UNUSABLE PACKS AT FIXED TEMP INCREASED FUEL CONSUPPSLATS/FLAPS SLOWCAT 1 ONLY TAT +28°CSAT +16°C GW XX KG 11H05 New Technologies and Associated Cultural Problems Amalberti

15. More solutions? Negative feedbacks SMART Nearloss NEEDED Negative feedbacks Performance and technology induced problems Performance Comprehension Management Back to manual? LOSS OF CONTROL Workload Management Amalberti

16. oint A viation A uthorities Tactical and strategic options for rule making • Cultural approach to rule making • All training should be evaluated, including Human factors training • ICAO annexes • A good training should combine basic knowledge (classroom), and practice. Amalberti

17. oint A viation A uthorities Cultural differences for rule making • The southern Mediterranean way of thinking • Rules are objectives, goals, stimuli to change behaviours • Rules are promoted when no –or very few- actors can comply with • Recommendations are useless • The US and Northern Europe way of thinking • Rules are references • They are set when all of the field actors can comply with • Recommendations prepare field actors, well before the rule, to comply with the future rule Amalberti

18. Competing philosophies for training design Amalberti

19. oint A viation A uthorities JARs / HF TRAINING • JAR FCL Cockpit crew training • 1.160, 1470 Ab initio : theoretical knowledge in HF / Theoretical HF examination to get the ATPL - Compliant with ICAO annex 1 • 1.240 Flight check licensing : demonstrating ability in crew co-ordination and airman ship, requirements extended into MCC regulations • JAR OPS Recurrent training Cockpit & Cabin Crew • 1.940, 45, 55, 65 CRM for cockpit crews. Recent change of requirements following the NPA 16, compliant with ICAO annex 6 • CRM for Cabin crew : TGL5 in progress Amalberti

20. Recurrent CRM 3Yrs cycle Modular/LOFT JAR OPS 1st year Conversion course CRM training when changing operator HPL MCC Initial CRM Conversion course CRM training when changing aeroplane type Theoretical coursein Human Factorsaccording JAR FCL Command course CRM training Zoom on HF training flight crew & on the NPA 16 CRM Cockpit crew ATPL JAR FCL Amalberti

21. NPA 16. Appendix 1.965New need for CRM assessment Amalberti

22. THE NOTECHS PROJECT 1997-1998 Sponsored by 4 CAAs [NL,D,UK,&F], and DGVII Basic design and development NLR (J.van Avermaete, E. Kuriijsen, H. Nijhuis + KLM Pilots), project leader DLR ( K-M Goeters, H-J. Hormann) Aberdeen University (R. Flin, L. Martin) IMASSA (R. Amalberti & C. Valot) The NOTECHS project “Non-technical skills refer to a pilot’s attitudes and behaviours in the cockpit not directly related to aircraft control, system management, technical consequences and standard operating procedures.” THE JARTEL PROJECT 1999-2001 • Sponsored by DGTREN • Validation & benchmarking • Same teams • NLR (J.van Avermaete, E. Kuriijsen, H. Nijhuis + KLM Pilots) • DLR ( K-M Goeters, H-J. Hormann) • Aberdeen University (R. Flin, L. Martin) • IMASSA (R. Amalberti & C. Valot) • Plus • British Airways • Alitalia • Airbus • DERA • Sofreavia (project leader) Amalberti

23. NOTECHSPrinciples for system design and operation FOUR PRINCIPLES FOR DESIGN • Maximum mutual exclusivity for categories and elements • Rule of parsimony - minimalist approach • Everyday terminology • Skills directly observable or inferred from other behaviours (communication) FOUR PRINCIPLES FOR OPERATIONS, designed to ensure that each crewmember receives as fair and as objective an assessment as possible. • Only observable behaviour is to be assessed The evaluation must exclude reference to a crewmember’s personality or emotional attitude and should be based only on observable behaviour. Behavioural markers were designed to support an objective judgement. • Need for technical consequence For NTS to be rated as unacceptable, flight safety must be actually (or potentially|) compromised. Thus demanding a related objective technical consequence. • Repetition required Repetition of unacceptable behaviour during the check must be observed to conclude that there is a significant problem. If, according to the JAR-paragraph concerned, the nature of a technical failure allows for a second attempt, this should be granted, regardless of the NTS rating. • Explanation required For each Category rated as unacceptable the examiner must: Indicate the Element(s) in that Category where the unacceptable behaviour was observed. b.  Explain where the observed NTS (potentially) led to safety consequences. Give a free-text explanation on each of the Categories rated unacceptable, using standard phraseology. Amalberti

24. The NOTECHS framework Non-technical skills Leadership & management skills Situation awareness Decision making Co-operation Team building & maintaining Category Element Considering others Behaviour Supporting others Helps other crew members in demanding situations Offers assistance Conflict solving Amalberti

25. Framework: Elements & Behaviours for Category - Situation Awareness Category: Situation Awareness Elements Environmental Awareness Anticipation System Awareness Behaviours -Monitors and reports changes in systems states -Acknowledges entries and changes to systems -Discusses contingency strategies -Identifies possible/future problems -Collects information about the environment -Contacts outside resources when necessary -Shares information about the environment with others Amalberti

26. Conclusion • Significant changes in the past decade • Extension to Cabin crew, Maintenance, etc, in progress • Remaining problems for success • Ends : Adhere to procedure Vs teach adaptation • Cost and fragility : what about HF training courses after the initial burst of enthusiasm ? • Means : who should be the instructors ? (good recent UK contribution) • Evaluation : Can we design HF training without evaluation? Amalberti