Automaticity development and decision making in complex, dynamic tasks

1. Automaticity development and decision making in complex, dynamic tasks Dynamic Decision Making Laboratory www.cmu.edu/DDMLab Social and Decision Sciences Department Carnegie Mellon University Cleotilde Gonzalez Rickey Thomas Polina Vanyukov

2. Complex and dynamic tasks Executing a battle, driving, air traffic controlling, managing of a production plan, piloting, managing inventory in a production chain, etc. • Demand real-time decisions (time constraints) • Demand attentional control • Require multi-tasking: they are composed of multiple and interrelated subtasks • Demand the identification of ‘targets’ defined by multi-attributes • Demand multiple and possibly changing responses

3. Automaticity in dynamic, complex tasks • targets and distractors are often inconsistently mapped to stimuli and responses • Often, we bring pre-learned categories and mappings to a task stimulus - category category - response L ------------- letter button --------- click • Are decision makers in dynamic situations operating in controlled processing continuously?

4. Proposed model of automaticity in DDM Goals (Relevancy) Task switching (resource allocation)

5. Experiments • Automaticity develops with consistently mapped stimuli to targets, even when targets move and time is limited (Experiment 1) • The consistency of target to response mapping also determines automaticity development (Experiment 2) • Automaticity of a task component frees-up time and resources for high level decision-making (Experiment 3) • Automaticity develops differently with different degrees of pre-learned categories (Experiment 4)

6. The Radar Task

8. General method • Independent variables • stimulus mapping (CM or VM) • CM = Search for Numbers in Letters • VM = Search for Letters in Letters • cognitive load • Memory set size (MSS): Number of possible targets to remember (1 or 4) • frame size (FS): Number of blips present on the screen at a given time (1 or 4) • target present/absent (a target was present 75% of the trials) • Dependent variables • Accuracy: proportion of correct detections or decision-making responses • Time: mean target detection or decision-making time in msec • From 18 to 30 hours of practice, 3 hours per day 6 to 10 days

9. Experiment 1: Consistency of stimuli • Replicate major findings from the dual-process theory (Schneider & Shiffrin, 1977) in a dynamic task • Automaticity is acquired with practice in consistent mapping conditions, and automatic performance is unaffected by workload

10. Experiment 1: Method • CM vs. VM • Cognitive Load Variables • Memory Set Size • Frame Size • Only one possible response: pressing spacebar when target is detected

11. Experiment 1: Accuracy

12. Experiment 1: Detect Time

13. Experiment 1: Summary • Radar’s manipulations of cognitive load interact with stimulus mapping in ways that parallel Schneider & Shiffrin’s results • Automaticity develops with extended practice and consistently mapped stimuli even when targets move and time is limited • Radar task can be used to study automaticity in dynamic stimulus environments

14. There is some evidence that response mapping is not critical for automaticity to develop (Fisk & Schneider, 1984; Kramer, Strayer, & Buckley, 1991) In complex tasks mapping of targets to responses can be inconsistent Resulting in large processing costs, even when stimuli are consistently mapped to targets Experiment 2: Response Consistency

15. Experiment 2: Method • Only consistently mapped stimuli • Cognitive Load Variables • Memory Set Size • Frame Size • Response consistency varied in four levels

16. Response Mapping Conditions T T Mapped to Stimuli Fully Mapped to interface Random Mapping Partial Mapping to interface T T

17. Experiment 2: Accuracy

18. Experiment 2 : detect time

19. Experiment 2: Summary • A consistent response reduces processing requirements • Total task consistency (both, consistency of stimuli and consistency of responses) matters • There are processing costs if responses are not consistently mapped, even when stimuli are • Implications • Interface design: interface influences processing of responses • Response selection using track-up vs. north-up displays • Make response selection intuitive • Interface design, decision support tools, training • We can now systematically manipulate Radar to elucidate the effects of automaticity on high-level dynamic decision-making

20. Experiment 3: Automatic detection & high-level decision making • How would automatic detection of a component help decision-making? • Decision-making component required operators to analyze a sensor array of detected aircraft • Sensor and weapon information changed dynamically

21. Experiment 3: Method • Sensor Reading Task • Determine if Target is Hostile • Scan Sensors • > 13 (Hostile) • < 13 (Non-Hostile) • Press Ignore (5-Key) • Select Response (Weapon Systems) • Guns vs. Missiles • > 10 Missiles (6-Key) • < 10 Guns (4-Key) • Quiet Airspace Report • No targets detected • Click submit report with mouse key

22. Experiment 3: Detect Accuracy

23. Experiment 3: Decision-making Accuracy

24. Experiment 3: Detect Time

25. Experiment 3: Decision-making Time

26. Experiment 3: Summary • Consistent mapping of targets improved he accuracy of the decision-making of the task • Detect time, detect accuracy, and whole-task performance are sensitive to workload manipulations • Implications • Consistent mapping actually improved whole-task performance by freeing up time for the controlled sensor-reading tasks to run to completion • Thus, processing speed-up associated with automatic detection can have a large impact on whole-task performance

27. But…? • Is accuracy of decision-making improved simply because there is more time to process? • Effect of detection on high-level decision-making in the presence of a dual-task

28. Experiment 3b: Method • Secondary tone task: enter count of number of non-standard tones • Calibrated to standard tone at beginning of session for each participant • Non-standard tones higher/lower pitch than standard

29. Experiment 3b: results • In fact the Radar task performance was the same with and without the tone task! • Detect Time • No Effect of secondary task • Detect Accuracy • No Effect of secondary task • Decision-Making Time • No Effect of secondary task • Decision-Making Accuracy • No Effect of secondary task

30. Experiment 3b: Implications • No effect of dual task on RADAR performance • Operators are allocating resources away from tone task to maintain RADAR performance • Implications • Finding supports the hypothesis that consistent mapping improves decision-making performance by freeing up resources for other tasks • Thus, processing speed-up and low resource requirement associated with consistent mapping can have a large impact on performance in complex task

31. Experiment 4: Categorization • Since consistent mapping is the search for numbers in letters, it is possible that load-free processing is due to categorization (Cheng, 1985) • Purpose of this experiment is to establish the presence of load-free processing without categorization

32. Experiment 4: Method • Incorporate memory ensembles where no possible categorization can take place either a priori or with learning • CM vs. VM with tone • CM = {C, G, H, M, Q, X, Z, R, S} • VM = {B, D, F, J, K, N, W, P, L} • Memory ensembles were equated • Angular {H,M,X,Z,F,K,N,W} vs. Round {C,B,D,G,Q,P,R,J} • Beginning {B,C,D,F,G,H,J,K} vs. End {M,N,P,Q,R,W,X,Z} • Cognitive Load Variables • Memory Set Size (1 or 4) • Frame Size (1 or 4) • Indicated detection of target by pressing spacebar • Detect Performance • Detect Response Time

33. Experiment 4: Detect accuracy

34. Experiment 4: Decision-making accuracy

35. Experiment 4: Detect time

36. Experiment 4: Decision-making time

37. Experiment 4: Implications • Varied mapped performance is more sensitive to load than consistently mapped performance • Individuals performed better in the high-level decision-making component of Radar when stimulus mapping was consistently mapped • Implications • Categorization is NOT a necessary requirement for automaticity development • Consistent stimulus mapping is a necessary condition for the development of automatic detection

38. Summary of accomplishments • Developed Radar, a dynamic simulation where it is possible to study (i.e., to measure) automaticity • In Radar it is possible to elucidate the effects of automaticity on high-level dynamic decision-making • Established the usefulness and applications of the dual-process theory of automaticity • Deepen our understanding of the implications of automaticity development for practical real-world tasks • Brought together two main theories of automaticity: instance-based theory and dual-process theory

39. Future research • Consistency of mapping and responding is relative to the categories (i.e., similarity) that a user can form • Thus, consistent mapping can lead to automatic responses for high-level decision-making after extended practice

40. Looking towards applications • Test these hypotheses in airport luggage screening • Decide whether to hand search the luggage • There is no consistency but rather just similarity (relative to a ‘knife’ category)