Major U.S. Airline Accidents Involving Inadvertent Omission of a Normal Procedural Step (1987-2001)

1. Prospective Memory in Aviation, Everyday Tasks, and the LaboratoryKey DismukesHuman Systems Integration DivisionNASA Ames Research CenterMoffett Field, CAPsychonomic Society18 November 2006

2. Major U.S. Airline Accidents Involving Inadvertent Omission of a Normal Procedural Step (1987-2001) • Five of 27 major accidents in which NTSB found crew error to be a causal factor • 1988, Detroit: DC-9-82 Flaps/slats not set to takeoff position,154 killed; 1 seriously injured • 1989, Dallas-Ft Worth: B727 Flaps/slats not set to takeoff position, 14 killed; 5 seriously injured • 1995, LaGuardia: MD-82 Pitot-static heat not turned on, aircraft destroyed, no fatalities/serious injuries • 1997, Houston: DC-9 Hydraulic boost pumps not set to high before landing, aircraft badly damaged, no fatalities/serious injuries • 2001, Little Rock: MD-82 Spoilers not armed before landing, 11 killed; 110 injured • Multiple factors involved, but a central aspect was experienced pilots forget to perform a normal, highly-practiced procedural step

3. Theoretical Accounts of Prospective Memory Still in Infancy Our view of evidence: • After intention for deferred action is formed, attention turns to other tasks • Deferred intention fades from WM but remains in LTM • Retrieval requires noticing some cue associated in memory with intention • Can be environmental cue or stream-of-thought cue • Stored intention is retrieved if sufficient activation spreads to it from cue

4. Linking Real-World PM Phenomena to Underlying Cognitive Processes • Three complementary approaches: • Ethnographic • Analyses of accident/incident reports • Laboratory studies • Structure of airline operations well suited to study of skilled performance & error • Operations are highly standardized • Can observe deviations from formal SOPs • Fair consensus about appropriate/inappropriate actions

5. An Ethnographic Study(Loukopoulos, Dismukes & Barshi, in preparation) • Focused on B737—widely used in air transport • Reviewed written operating procedures, participated in classroom and flight simulation training at two major airlines • Observed large number of flights from cockpit jumpseat in normal revenue flights

6. Some Findings from Ethnographic Study • Pilots are: • Frequently interrupted • Forced to postpone planned tasks • (Each) forced to perform multiple tasks in parallel • Timing and nature of task demands is not entirely predictable and is only partly under control of the crew • These results confirmed by a study analyzing NTSB reports of the 19 major U.S. airline accidents attributed primarily to crew error 1990-2001 (Dismukes, Berman, & Loukopoulos, in press)

7. Memory Errors Reported to the Aviation Safety Reporting System(Nowinski, Holbrook, & Dismukes, 2003) • Sampled 20% of air carrier reports over 12-month period • Selected reports clearly involving some type of memory error • 74 of 75 reports involved prospective rather than retrospective memory • Does not necessarily indicate PM more common than RM failures, but suggests PM failures may be: • more consequential • more likely to be reported • and/or more common

8. Varieties of Prospective Memory in the Cockpit • In these three studies analyzed: • What tasks were being performed concurrently • Whether forgotten task was habitual • What cues would normally be present to trigger retrieval • Whether those cues were actually present • Concluded PM demands emerge in five types of situation: • Episodic tasks • Habitual tasks • Atypical tasks substituted for habitual actions • Interrupted tasks • Interleaving tasks, including monitoring

9. Varieties of PM (1) Episodic Tasks • Tasks not performed habitually • Type of PM task most often studied experimentally • Example: ATC instructs crew to report passing through 10,000' during descent • In real world, ongoing tasks often divert attention from cues that might trigger retrieval (Holbrook, Dismukes & Nowinski, 2005) • May be a major source of variance, but not studied extensively • This is the type of task studied most often in laboratory paradigms

10. Varieties of PM (2) Habitual Tasks • Most cockpit tasks are habitual • Many tasks must be performed, many with multiple steps • Normally performed in set sequence by SOP • Execution becomes largely automatic—deliberate search of memory not required • Explicit (episodic) intention not required—intention is implicit in action schema, stored as procedural memory • Performance of habitual tasks normally quite reliable • Performance undermined if normally present cues are removed • Example: Setting flaps to takeoff position is deferred because of freezing slush on taxiway • Action out of sequence removes normal cues & normal context • Habitual task becomes episodic task, but pilots may not realize need to encode explicit intention

11. Varieties of PM (2) Habitual Tasks • Most cockpit tasks are habitual • Many tasks must be performed, many with multiple steps • Normally performed in set sequence by SOP • Execution becomes largely automatic—deliberate search of memory not required • Explicit (episodic) intention not required—intention is implicit in action schema, stored as procedural memory • Performance of habitual tasks normally quite reliable • Performance undermined if normally present cues are removed • Example: Setting flaps to takeoff position is deferred because of freezing slush on taxiway • Action out of sequence removes normal cues & normal context • Habitual task becomes episodic task, but pilots may not realize need to encode explicit intention

12. Varieties of PM (3) Atypical Actions Substituted for Habitual Actions • Example: Crew often departs from an airport via a Standard Instrument Departure requiring left turn to 300 degrees at 2000 feet • Execution of procedure becomes largely automatic • If ATC on one occasion amends instruction to turn left to 330 degrees at 2000 feet: • Crew must form episodic intention to turn to 330 and must inhibit normal response to level out at 300 degrees • When busy with other tasks, crews vulnerable to habit intrusion (Reason, 1984) and level out prematurely at 300

13. Varieties of PM (4) Interrupted Tasks • Interruptions very common in cockpit • Duration lasts from seconds to minutes • Interruptions are often so abrupt and salient that pilots may do little to encode explicit intention to resume interrupted task • Common error is to deal with interruption then proceed to next task, forgetting to complete interrupted task • Cockpit does not always provide salient cues for status of interrupted tasks • Perceptually rich cockpit environment associated with many tasks remaining to be performed

14. Varieties of PM (5) Interleaving Tasks, Including Monitoring • While performing ongoing tasks pilots must periodically monitor status of other tasks, e.g.: • While running checklists, communicating, programming, etc., First Officer must monitor progress of taxi by Captain • Monitoring is important, but difficult to maintain monitoring goal in WM as dual task if event rate low • Similar to difficulty with vigilance (Parasuraman, 1986) but different: • Must interrupt ongoing task and shift attention • Attention allocated preferentially to areas with high info content over monitoring for low probability events, albeit with high consequences (see Wickens et al., 2003 for attention allocation model) • When attention shift delayed, monitoring task may slip from WM & must be retrieved, as in other forms of PM

15. Varieties of PM (5) Interleaving Tasks, Including Monitoring • While performing ongoing tasks pilots must periodically monitor status of other tasks, e.g.: • While running checklists, communicating, etc., First Officer must monitor progress of taxi by Captain • Monitoring is important, but difficult to maintain monitoring goal in WM as dual task if event rate low • Similar to difficulty with vigilance (Parasuraman, 1986) but different: • Must interrupt ongoing task and shift attention • Attention allocated preferentially to areas with high info content over monitoring for low probability events, albeit with high consequences (see Wickens et al., 2003 for attention allocation model) • When attention shift delayed, monitoring task may slip from WM & must be retrieved, as in other forms of PM

16. An Experimental Study of Interruptions(Dodhia & Dismukes, submitted) From aviation studies, hypothesized individuals forget to resume interrupted tasks largely because: • Salient intrusion of interruptions diverts attention, discouraging encoding explicit intention to resume or identifying specific reminder cues • Resumption will depend on happening to notice cues previously associated with interrupted task—very unreliable • Cues indicating opportunity to resume interrupted task may not match form of (implicit or explicit) encoded intention • “End of the interruption” is not a discrete perceptual cue but a state of affairs requiring interpretation • Poor match between cues and encoding provides little activation to retrieve intention from LTM • End of interruption often followed by other task demands that divert attention from interpreting significance of completing interruption • New task goals spread activation to task subgoals rather than to retrieving implicit goal of resuming interrupted task

17. An Experimental Study of Interruptions Experimental Paradigm • Participants answer questions resembling SAT, Category of questions changed between blocks • Instructed that when blocks were interrupted they should remember to resume interrupted block after completing interruption before starting new block • In baseline condition interruptions were abrupt—current screen replaced with screen of different color and with different category of questions • After each completed block a screen appeared for 2.5 sec: “Loading next block”, followed by next block in series

18. An Experimental Study of Interruptions Experimental Paradigm (continued) • Without explicit prompt participants had to remember to return to interrupted block • Proportion of successful resumptions of interrupted task in baseline condition: 0.48

19. An Experimental Study of Interruptions Encoding Manipulations • Hypothesis 1: Intrusion of sudden interruption discourages adequate encoding of explicit intention to resume interrupted task • Tested with encoding reminder manipulation: • Interruption began with 4 sec message: “Please remember to return to the block that was just interrupted” • Results: Performance increased from 0.48 to 0.65 (proportion of interrupted blocks resumed) • Was improvement due to reminder or to 4 sec delay before beginning interrupting task? Tested with encoding pause manipulation: • Interruption began with 4 sec blank screen • Results: Performance increased from 0.48 to 0.65 • Conclusion: Pause before beginning interrupting tasks helps remember to resume interrupted task

20. An Experimental Study of Interruptions Retrieval Manipulations • Hypothesis 2: Individuals are likely to forget to resume interrupted tasks because of mismatch between form of encoded intention and diverse cues that must be interpreted to recognize interruption has ended • Tested with retrieval reminder manipulation: • 2.5 sec “Loading next block” screen also said: “End of interruption” • Results: Performance increased from 0.48 to 0.90 (proportion of interrupted blocks resumed) • Hypothesis 3: Individuals are likely to forget to resume interrupted tasks if interrupting task is directly followed by other tasks that demand attention • Tested with retrieval pause manipulation: • Interval between end of interruption and beginning of next block increased to 8-12 sec and countdown clock appeared on screen • Results: Performance increased from 0.52 to 0.88 (proportion of interrupted blocks resumed)

21. An Experimental Study of Interruptions Summary of Results

22. An Experimental Study of Interruptions Implications • Theoretical: • Supports three hypotheses but more research needed • Consistent with model of PM in which intentions are retrieved from LTM by activation from cues processed attentively • Practical: • Pausing before dealing with interruptions to explicitly encode intention to resume interrupted task and to identify specific reminder cues • Periodically pausing after completing tasks to ask what other tasks remain to be done

23. Final Thoughts • Prospective memory probably operates in similar ways in the skilled tasks performed by other kinds of experts, e.g., medical personnel • To be effective, countermeasures to PM errors must be realistic for experts to apply while performing their task • Understanding the real-world performance requires integrating diverse research approaches, including observing real-world performance and well-controlled experimental studies • This integrative approach enhances the power of both field and lab studies

24. Thanks to my colleagues in this research: Rahul Dodhia, Jon Holbrook, Kim Jobe, Loukia Loukopoulos, and Jessica Nowinski For more information: http://human-factors.arc.nasa.gov/ihs/flightcognition