Situation Awareness & Levels of Automation

1. Situation Awareness & Levels of Automation David Kaber and Kheng-Wooi Tan Mississippi State University

2. Objective • To assess the effect of different LOAs on pilot performance, SA, and cognitive workload in the context of a simulation of MD-11 flight deck • To identify LOAs that will: • Achieve optimal human-automation integration Alleviate problems of pilot OOTL performance Support pilot achievement of SA

3. Approach • Scope of study: • Apply theoretical hierarchy of LOAs to MD-11 (Categorize modes of MD-11 automation in terms of theoretical taxonomy of LOAs) • Model and examine modes of automation currently available in MD-11 • Model and study higher levels of cockpit automation by assuming utilization of expert system in FMS

4. Approach (Cont.) • Purpose of approach: • Explore effect of a broad spectrum of LOAs on human-machine performance and pilot SA in realistic simulation of aircraft (MD-11) flight control • Investigation of higher LOAs serves as applicable guide for future improvement of automated systems in cockpits

5. Applying Theoretical LOA Taxonomy to MD-11 • Categorizing existing modes of autoflight in terms of theoretical LOAs: - In depth analysis of cockpit systems and flight tasks.  Review of pilot’s reference manuals.  Familiarization with MD-11 simulator.  Consultation with experienced pilot.  Development of autoflight mode trees (establish what is possible and impossible in use of autoflight system) • List of available functions • List functional settings

6. Applying Theoretical LOA Taxonomy to MD-11 (Cont.) • Formal description of who is doing what under each mode of autoflight. Description of communication channels. • Categorize formal descriptions in terms of theoretical levels in taxonomy. Categorization of flight control functions in terms of four- stages of information processing (represented in taxonomy). Consideration of function ownership and matching of actual mode to theoretical level.

7. Modeling Theoretical LOAs in Context of MD-11 • Identified theoretical LOAs currently represented in autoflight system. • Conceptualized new autoflight capability to model higher LOAs of taxonomy in context of cockpit.  Assumed intergration of expert system in FMC associated with MCDU. • System capable of auto generation of flight plans. • System capable of evaluation of flight plans in terms of efficiency (including pilot plans). • System capable of selecting “optimal” flight plan

8. Modeling Theoretical LOAs in Context of MD-11(Cont.) • Identify autoflight subsystems involved in each theoretical mode of automation. • Identify who’s in charge of specific flight tasks based on generic function allocation presented in taxonomy. • Identify specific subsystems to be used for flight tasks.

9. Description of LOAs in Theoretical Taxonomy in Context of MD-11 Flight Deck • Level 1: Manual Control • Complete human control of flight activities • Pilots monitor on-board operations & system status to account for changes in flight events & system failures • Pilots formulate, evaluate, and select an appropriate flight plan • Pilots manually control aircraft to implement selected plan

10. Level 2: Action Support Pilots & auto jointly monitor on-board operations & system status Pilots formulate, evaluate, and select an appropriate flight plan Pilots use FCP to engage selected plan Pilots & auto jointly implement flight plan  Auto provides limited assistance to implement selected plan  Pilots can choose to use manual control Corrective actions performed by either pilots or auto Auto - warning provided to pilot Pilots - can request computer implementation of action or use manual control Level 3: Batch Processing Pilots & auto jointly monitor on-board operations & system status Pilots formulate, evaluate, and select an appropriate flight plan Pilots use MCDU to program selected plan (explicit act of selection) Fully automated plan implementation Corrective actions performed by either pilots or auto Auto - warning provided to pilot Pilots - must edit/enter new plan via MCDU for computer implementation Description of LOAs Taxonomy (Cont.)

11. Level 4: Shared Control Pilots & auto jointly monitor on-board operations & system status Both generate flight plans Pre-determined flight plans generated by expert system and presented via MCDU Pilots evaluate alternative flight plans & select an “optimal” plan Pilots use MCDU to program selected plan (explicit act of selection) Both jointly implement flight plan Auto works through FD to implement plan in MCDU and display control commands on FMA when protection envelope is active Pilots can use manual control of speed, heading, etc Corrective actions performed by either pilots or auto Auto - warning provided to pilots Pilots - can edit/enter new plan via MCDU for computer implementation or use manual control Level 5: Decision Support Pilots & auto jointly monitor on-board operations & system status Both generate flight plans Pre-determined flight plans generated by computer and presented via MCDU Pilots evaluate alternative flight plans & select an “optimal” plan Pilots use MCDU to program selected plan (explicit act of selection) Fully automated plan implementation Corrective actions performed by either pilots or auto Auto - warning provided to pilot Pilots - must edit/enter a new plan via MCDU for computer implementation Description of LOAs Taxonomy (Cont.)

12. Level 6: Blended Decision Making Pilots & auto jointly monitor on-board operations & system status Both generate flight plans Pre-determined flight plans generated by computer are presented via MCDU Pilot-generated flight plan is entered via MCDU Auto evaluates flight plan alternatives and selects an “optimal” plan; Auto consults pilots prior to flight plan implementation Pilots can override computer plan selection Fully automated implementation of plan Corrective actions performed by pilots and auto Auto - warning provided to pilots Pilots - can enter new plan into MCDU for auto to select “best” corrective plan; pilots can override selection Level 7: Rigid System Pilots & auto jointly monitor on-board operations & system status Auto generates flight plans & presents them via MCDU Pilots evaluate & select “optimal” flight plan from list Fully automated implementation of plan If corrective actions are needed, auto generate new plans for pilots to select from Description of LOA Taxonomy (Cont.)

13. Description of LOA Taxonomy (Cont.) • Level 8: Automated Decision Making • Pilots & auto jointly monitor on-board operations & system status • Both generate flight plans Pre-determined flight plans generated by expert system are presented via MCDU Pilot-generated flight plan is entered into MCDU • Auto evaluates flight plan alternatives & selects an “optimal” plan; no pilot intervention permitted in selection • Fully automated implementation of plan • If corrective actions are needed, pilots can program new plan for auto to consider for selection from among computer-generated plans

14. Level 9: Supervisory Control Pilots & auto jointly monitor on-board operation & system status Auto generates and selects an appropriate flight plan Fully automated implementation of plan Corrective actions performed by either pilots or auto Auto - warning provided to pilots Pilots - can intervene and request a shift to Decision Support in order to re-program new plan Level 10: Full Automation Pilots observe system Auto monitors on-board operation & system status Auto generates and selects an appropriate flight plan Fully automated implementation of plan Corrective actions performed by auto and no pilot intervention permitted Description of LOA Taxonomy (Cont.)

15. Selected LOAs for Experimental Assessment • Manual Control (Level 1) • Batch Processing (Level 3) • Shared Control (Level 4) • Supervisory Control (Level 9) (Decision Support (Level 5) to be studied during intervention in Supervisory Control (Level 9))

16. Task • Re-create scenario of AA 965 Cali incident (1995) with approach revision due to runway change under high time pressure • Pre-departure flight subtasks: • 3 potential flight paths will be provided with 6 waypoints included in each path • Subjects formulate flight plans or evaluate pre-determined flight plans • Subjects program complete flight plan or select a plan from options presented by computer

17. Task (Cont.) • En-route flight subtasks: • Monitor on-board operations & system status • Conduct approach revision due to runway change within 6-7 min prior to crossing 3rd waypoint • Select among identical waypoint identifiers with dissimilar geographical coordinates presented through Duplicate Page of MCDU during approach revision (MCDU re-programming) Purpose - Induce confusion during plan re- programming to assess impact of LOA on recovery performance, SA and workload

18. Subjects • 24 students from MSU student population • Selection criteria: • Possess 20/20 or corrected to normal vision • Experience in using PC, mouse controller, and standard keyboard • To be compensated at $10 per hour Independent Variables • LOA of the flight deck subsystems

19. Dependent Variables • Performance - Time to task completion (TTC) - Number of errors committed in waypoint selection using Duplicate Page of MCDU • SA • Measured using commercial pilot SAGAT • Accomplished by freezing simulation at random points in time, blanking visual display screens, and administering SA queries  3 simulated freezes during each trial  Present 9 queries at each freeze, 3 queries on perception, comprehension and projection • Percentage of correct responses to be calculated for all queries

20. Dependent Variables (Cont.) • Workload • NASA-TLX to be used to collect subjective mental workload ratings at end of each trial Experimental Design • Completely between subjects design • 4 levels of independent variable LOA • Randomly assign subjects to 4 groups corresponding to LOA settings