Collective Mistrust of Alarms

1. Collective Mistrust of Alarms James P. Bliss, Ph.D. Susan Sidone Holly Mason Old Dominion University

2. Collective Mistrust of Alarms - A Few Thoughts Before We Begin... • Novelty of this project • Not Automation per se: Alarms inform, do not control • Alarms convey system state to operator • May help to “push the envelope” of etiquette research • “Mistrust/distrust” may be different with alarms • Simplistic paradigm deceptive - multiple trust components involved • Information accessibility, technology improvements means operators expect more from alarm systems • Operator mental models very important Bliss, Sidone, & Mason, 2002

3. Collective Mistrust of Alarms - Introduction • Investigations of Individual Alarm Mistrust • Aviation (Bliss, 1997) • Mining (Mallett et al., 1992) • Ship Handling (Kerstholt et al., 1996) • Driving (Nohre et al., 1998). • General Findings: People Reacted Slower, Less Frequently, Less Appropriately to Unreliable Alarms. • No Studies of the Impact of Marginally Reliable Alarm Signals on Teams of Operators Bliss, Sidone, & Mason, 2002

4. Collective Mistrust of Alarms - Introduction • Teamed Alarm Reactions • Aviation • Critical Care Units • Nuclear Power Plants • Air Traffic Control Centers • To Effectively React to Alarms, Team Members Must • Share Information • Troubleshoot Systems • Determine Relative Signal Priority • Allocate and Coordinate Reaction Responsibility Bliss, Sidone, & Mason, 2002

5. Collective Mistrust of Alarms - Introduction • Team Member Interdependence Often Varies with the Task and the Environment (Thompson, 1967). • Dependent Teams React to Alarms More Appropriately, More Slowly (Bliss et al., 2002) • Implications of Teamed Alarm Reactions for Human-Automation Etiquette • Human-Alarm Trust • Human-Human Trust • Human-Human(Alarm) Trust Bliss, Sidone, & Mason, 2002

6. Collective Mistrust of Alarms - Introduction • Goals of the Current Research: • Investigate Reactions of Dependent and Independent Teams to Alarm Signals of Various Reliability Levels. • Determine How Collateral Alarm Systems Mediate Alarm Mistrust. • Approach: • Dual-Task Approach (Damos, 1991). • Independent Variables Manipulated Using a 2 X 3 Mixed Design. • Dyads Reacted to Two Separate Alarm Systems. • Temperature Alarm Reliability = 80% true alarms). • Pressure Alarm Reliability Fluctuated (40%, 60% or 80%). Bliss, Sidone, & Mason, 2002

7. Collective Mistrust of Alarms - Experimental Design • Interdependence Manipulated Between Two Groups • Dependent Team Members Required Interaction to React Appropriately; Independent Team Members Did Not. • Pressure Alarm Reliability Manipulated Within Groups • Pressure Alarms Were 40%, 60%, and 80% Reliable During Sequential Task Sessions. Temperature Alarms Were 80% Reliable. • Dependent Measures • Ongoing Task: Gauge Monitoring Accuracy, Tracking Error. • Alarm Task: Reaction Speed, Appropriateness; Response Frequency. Bliss, Sidone, & Mason, 2002

8. Collective Mistrust of Alarms - Method • Participants: 40 student dyads from Old Dominion University (18-43 yrs) worked for course credit and the chance for a monetary performance bonus. • Primary Task: Multi-Attribute Task (MAT) battery (Comstock & Arnegard, 1992) presented to each member. • Dual-Axis Compensatory Tracking • Gauge Monitoring • Resource Management • Participants Performed the MAT Back-to-Back Bliss, Sidone, & Mason, 2002

9. Collective Mistrust of Alarms – MAT Battery Bliss, Sidone, & Mason, 2002

10. Collective Mistrust of Alarms - Method • Auditory and Visual Alarms: Digitized Fire Bell From a Boeing 757/767 simulator. • Alarms Occurred 90° to the Side of the Primary Task. • Alarm Procedure: Determine Whether Corresponding MAT Gauges Are Out of Tolerance. If so, Reset Gauges and respond to the alarm. If Not, Cancel the alarm and resume the primary task. • Interdependent team members had to communicate because they shared the out-of-tolerance gauges. Independent team members monitored all gauges. Bliss, Sidone, & Mason, 2002

11. Collective Mistrust of Alarms - Procedure • Informed Consent Form • Experimental Instructions - Dependent team members told to communicate. • MAT Task Practice • Individual 120-second sessions (Each Subtask) • Combined 200-second session (MAT and Alarms) • Three experimental sessions • Ten alarms presented during each session. • Pressure alarm reliability randomly counterbalanced • Participants Knew Alarm System Reliability Before They Began • Debriefing, dismissal. Bliss, Sidone, & Mason, 2002

12. Collective Mistrust of Alarms – Results (Response Frequency) • Response Frequency to Temp Alarms • No Interaction (p>.05) • Linear main effect, F(1,38)=129.600, p<.001. Bliss, Sidone, & Mason, 2002

13. Collective Mistrust of Alarms – Results (Reaction Appropriateness) • Significant Interaction, F(2,76)=10.193, p<.001. • Main Effect for Interdependence, F(1, 38)= 4.000, p=.05. • Quadratic Main Effect for Reliability, F(1,38)=19,563, p<.001. Bliss, Sidone, & Mason, 2002

14. Collective Mistrust of Alarms – Results (Reaction Time) • No significant interaction • No Interdependence main effect • Linear Reliability Main Effect, F(1,38)=8.181, p=.007. • NOTE: No Primary Task Differences Bliss, Sidone, & Mason, 2002

15. Collective Mistrust of Alarms - Discussion • Results Similar to Past Efforts, Except for Lack of Primary Task Differences. • Multiple alarm systems may have led participants to rethink their trust levels, a reflection of workload (Bliss & Dunn, 2000). • Alarm designers should consider the effects of multiple alarm systems on operator behavior. • Recognize that complex reaction responsibilities may cause cognitive load as team members adjust trust levels. Bliss, Sidone, & Mason, 2002

16. Collective Mistrust of Alarms - Discussion • Dimensions of Trust in This Experiment • Basic Trust of the Experimenter (Human-Human) • Trust of the Primary (MAT) Task (Human-Computer) • Trust of the Alarm Task (Human-Computer) • Manipulated by the Experimenter • Trust of Teammates (Human-Human) • Questionable in this Experiment, Due to documented Unfamiliarity Bliss, Sidone, & Mason, 2002

17. Collective Mistrust of Alarms - Discussion • Past reactions to unreliable alarm systems • Fluctuations in physiological responses (Breznitz, 1983) • Degraded performance (Getty et al., 1995) • Complete Lack of Trust (Bliss, 1993) • Complete Trust • Probability Matching – Participants’ Response Rates Mirror the Perceived Reliability of the Alarm System. • These Patterns Take Time to Appear (Bliss et al., 1996). • Question: What if Researchers Apply human trust facilitators to human-alarm relationships? WHAT ARE THOSE VARIABLES? Bliss, Sidone, & Mason, 2002

18. Collective Mistrust of Alarms - Discussion • Documented Ways to Improve Alarm Responsiveness • Maximize alarm reliability (Bliss, 1993) • Advertise high alarm reliability rates (Bliss et al., 1995) • Add Redundant Sources of Alarm Information (Bliss et al., 1996) • Augment alarm stimuli and response options (Bliss, 1997). • Etiquette Related Possibilities • Give alarm systems “human” qualities (include verbiage, etc.) • Make alarm stimuli emotional (Sorkin et al.’s “likelihood alarm displays”; altering Edworthy’s parameters) • Vary teammate trustworthiness Bliss, Sidone, & Mason, 2002