Outline

1. Issues in measuring sensory-motor control performance of human drivers: The case of cognitive load and steering control Johan Engström, Volvo Technology Corporation European Workshop on Advanced Predictive Sensory-motor ControlJoudkrante, Lithuania, 2009-05-21

2. Outline • Multitasking in the vehicle • Secondary tasks – visual and cognitive • The primary driving task – visual control of steering • Effects of secondary tasks on steering control • Different effects of visual and cognitive tasks • The ”lane keeping improvement” effect of cognitive load • Possible explanation in terms of satisficing vs. optimising steering control • Testing predictions implied by this hypothesis Vehicle & Load Structures

3. Multitasking in the vehicle: Driving + secondary tasks Vehicle & Load Structures

4. Secondary tasks: Visual vs. cognitive distraction • Visual distraction • Looking off road • E.g. Visual time sharing when tuning the radio • Cognitive distraction: • Engaging in demanding cognitive (working memory) tasks • E.g. Mobile phone conversation • Most real-world tasks involve both components… Vehicle & Load Structures

5. The primary driving task: Sensory-motor control in steering Vehicle & Load Structures

6. The visual control of steering: Optical and retinal flow Wann and Wilke (2000) Retinal flow not equal to optical flow Straight driving, looking to the left side Straight driving, looking ahead Vehicle & Load Structures

7. Using retinal flow patterns to guide steering: Look where you’re going resulting heading initial heading Wann and Wilke (2000) Underrsteering Going towards target Oversteering Vehicle & Load Structures

8. Gaze angle can be used as a direct cue for steering through curves (Land, 1998) Fixate tangent point and adjust steering to keep gaze angle constant Vehicle & Load Structures

9. Combining retinal flow patterns and gaze direction: ”Spring” model (Wann and Wilkie, 2000) Stiffness Angular acceleration Damping Reliance on cues Main point: Foveal vision is essential for accurate steering! Vehicle & Load Structures

10. Effects of secondary tasks on steering control Vehicle & Load Structures

11. Visual time sharing Effects of visual distraction on lateral control Gaze angle • Looking away • Loosing visual input for steering control • Heading error builds up • Looking back • Large steering wheel correction • Speed reduction to compensate Steering wheel angle Lane position Increased lane position variance Speed Engström and Markkula (2006) Vehicle & Load Structures

12. What about purely cognitive distraction? • Large number of simulator and real-world driving studies found reduced lane keeping variance during cognitive load (Brookhuis et al., 1991; Östlund et al., 2004; Jamson and Merat, 2005; Engström et al., 2005; Mattes, Föhl and Schindhelm, 2007; Merat and Jamson, 2008). Engström, Johansson and Östlund (2005) Does talking on the mobile phone really improve steering control? SD lane position Cognitive task difficulty Vehicle & Load Structures

13. Other effects of cognitive distraction: Gaze concentration Victor, Harbluk and Engström (2005) Normal driving Cognitive task Vehicle & Load Structures

14. Other effects of cognitive distraction: Increased steering activity (number of steering reversals > 2 deg. per minute) SW reversals/min Engström et al. (2005) Vehicle & Load Structures

15. Summary: Effects of cognitive distraction related to lateral control • Improved lane keeping (!?) • Gaze concentration towards the road centre • Increased number of micro steering corrections (<2 deg) • How are these effects related? How can they be explained? Vehicle & Load Structures

16. Possible explanation Vehicle & Load Structures

17. Two key distinctions • Satisficing vs. Optimising • Top-down (endogenous) vs. Bottom-up (exogenous) attention selection Vehicle & Load Structures

18. 1. Satisficing vs. optimising Target value Comfort zone Optimising: Minimising performance error relative to a target state. Satisficing: Maintaining performance within acceptable boundaries. Vehicle & Load Structures

19. Example cost functions of optimising and satisficing in lane keeping Cost Optimising Satisficing Lane position Lane centre Vehicle & Load Structures

20. Example dynamics of satisficing and optimising X_dot Satisficing Comfort zone Optimising X Vehicle & Load Structures

21. Simulations Satisficing Optimising Vehicle & Load Structures

22. Top-down attention bias 2. Bottom-up and top-down attention selection Top-down selection Cognitive task Other visual task Vehicle dynamics Steering Bottom-up selection Vehicle & Load Structures

23. Top-down attention bias Normal driving Steering easy and automated task, bottom-up-driven -> satisficing Top-down selection Other visual task Vehicle dynamics Steering Spare top-down attentional resources used for other visual tasks Bottom-up selection visual time sharing • Lane keeping variance • Distributed gaze • Only intermittent steering Vehicle & Load Structures

24. Top-down attention bias Top-down selection Cognitive load Top-down attention allocated to cognitive task Cognitive task Other visual task No top-down-initiation of other visual tasks Vehicle dynamics Steering Gaze can be fully devoted to steering (attracted bottom-up) Bottom-up selection • Reduced lane keeping • variance • Gaze concentration Vehicle & Load Structures

25. Testable predictions • General: Improved lane keeping should only occur if the driver is satisficing in baseline condition • Specific predictions: • Improved lane keeping should not occur if the steering task is difficult (so that satisficing is not possible) • Improved lane keeping effect should not occur if the driver is motivated to optimise lane keeping • Support for prediction 1 • Cognitive load has been demonstrated to impair performance on tracking tasks (Creem and Proffitt, 2001; Strayer and Drews. 2001). • These tasks could be expected to be more difficult and/or less automated than normal driving • Prediction 2: Tested experimentally… Vehicle & Load Structures

26. Top-down attention bias Instruction to optimise steering (baseline) Top-down selection Top-down attention allocated to steering task and cognitive task Other visual task Optimising steering performance Vehicle dynamics Steering Bottom-up selection • Reduced lane keeping • variance • Gaze concentration • Increased steering • wheel control input Vehicle & Load Structures

27. Testing prediction 2: Experimental design • Simulator study in fixed based simulator (at Saab Automobile, Trollhättan) • Cognitive task: Count backwards with 7 • 48 subjects, split in 4 groups: • Incentive for group 1 and 2: Two cinema tickets instead of one if meeting some (unspecified) lane keeping criterion Vehicle & Load Structures

28. Prediction • Lane keeping improvement effect of cognitive load should only occur when the driver is not motivated to optimise lane keeping = satisficing • Interaction between cognitive load and instruction to optimise Vehicle & Load Structures

29. Preliminary results: Lane keeping (HP-filtered SD Lane Position) No cognitive task Effect only for non-instructed subjects Cognitive task Due to satisficing in baseline condition Instructed to optimise lane keeping No instruction Vehicle & Load Structures

30. Steering wheel reversal rate Cognitive task Same effect in both conditions Cognitive load less efficient optimising: more steering – same lane keeping performance No cognitive task Instructed to optimise lane keeping No instruction Vehicle & Load Structures

31. Still to be analysed… • Eye movements • Speed change • Performance on cognitive task Vehicle & Load Structures

32. Discussion • Replicated earlier findings for non-instructed drivers: • Reduced lane keeping performance • Increased steering wheel activity • Predicted effect of instructions found -> improved lane keeping only for non-instructed drivers – due to satisficing in baseline condition • Cognitive load seems to induce less efficient steering while optimising (more effort in steering, same result on lane keeping) • Cognitive task does not really improve steering ability-> the effect rather reflects ”involuntary” improvement from ”sloppy” baseline driving Vehicle & Load Structures

33. General conclusions • Caution is needed when interpreting driving performance measurements – do we compare to a baseline with satisficing or optimising performance? • In this case, changing instructions and/or driving task difficulty may cancel or perhaps even reverse the effect of cognitive load • Implies re-interpretation of many existing studies on the effects of cell phone conversation on driving performance (e.g. Strayer and Drews, 2001) Vehicle & Load Structures